Often developers will first think of databases as the solution to the challenge of where to store application data. Relational (SQL) and non-relational, document-based (NoSQL) databases are proven solutions in the enterprise. For some data, however, there is another option that can provide incredible benefits at a relatively low cost. Cloud storage is often much cheaper than managed databases and can accommodate many types of application architectures. Below, we have created an overview of the top seven storage options for developers.
S3 is a simple and effective way to store files and can be used as a pay-as-you-go alternative to Dropbox. S3 is most often used to store and access data programmatically. S3 offers simple pricing per GB stored per month. For the first 50 TB you store, S3 charges 2.3 cents per GB per month. They have highly granular and robust security features trusted by millions of companies. Lastly, S3 also comes with many tools and software packages to make adding their service to your applications simple.
Google Cloud Storage is quick and easy to set up and offers features and security controls similar to those of S3. The Google Cloud Platform interface is well designed and makes managing your storage very straightforward. They also offer pricing per GB stored per month. Depending on features, their pricing ranges from 2.6 cents to 3.6 cents per GB per month. They also provide tools and software packages that make integration simple.
Backblaze started as a backup company and became famous amongst storage enthusiasts for their design ingenuity and hard drive reports. They are now on version 6 of their storage pod, which is capable of hosting 60 drives and storing 480 Terabytes of data. They also open source the design of these pods so that anyone can homebrew their own-a sign of the confidence they have in their hardware.
Backblaze has recently begun to offer their B2 storage service, which takes their expertise in dealing with large backups and brings it to the cloud storage market. B2 is by far the cheapest storage option available, offering storage at half a cent per GB per month. Given their experience dealing with large backups, it makes sense that they can offer their B2 service at one quarter the price of their competitors. I was thrilled when Backblaze announced their entrance into the cloud storage market and look forward to using them as a storage service.
Digital Ocean's Spaces storage breaks from tradition in several ways. They are S3 compatible, meaning that their storage strictly adheres to S3's API, making migration and access very straightforward for current S3 customers without requiring too many changes to their applications. Spaces pricing, also, does something different. For $5/mo, they offer 250 GB of storage and unlimited uploads. After that, every additional GB you upload will add 2 cents per month to your bill in line with pricing we've seen.
Microsoft has been a Cloud powerhouse for quite some time and is continuing to invest heavily in this part of their business. Much like S3 and Google Cloud, they use the pay-per-GB model and offer storage at 1.84 cents per GB per month.
Another cost to consider for any cloud storage provider is transfer and bandwidth pricing. Prices vary, but most providers charge less than a cent per GB transferred to and from their servers. This accounts for the bandwidth usage required to get your files to their disks and is a factor if you're expecting to be moving high volumes of data to and from storage.
Finally, let's look at some alternative storage options that may make sense for special workloads or storage requirements:
You may already have some portable hard drives laying around. A NAS, or network attached storage, is a very similar concept, except it makes those drives accessible over your network and thus available to any connected computer. This storage option is usually convenient but can be insecure if not correctly managed. It can also be prone to drive failure. Depending on how the NAS is configured, even a single drive failure can cause your data to be lost. Most often, the default configurations will build in some redundancy allowing for 1 or more drives to fail before data becomes irrecoverable. This is not a concern with cloud storage options. Network attached storage devices can offer a lot of storage immediately on your network behind your firewall. Manufacturers like Synology offer NAS devices with security and software interfaces baked in. However, any data you store on these devices is best when backed up off-site in a cloud provider to ensure that even disasters cannot put your data at risk. The advantages of NAS and on-premise storage are speed, privacy, and ease of transferring large files, as transfer speeds are limited only by your network set up and can, therefore, be quite fast.
While it is no easy task, running a custom server can offer near unlimited flexibility and choice. Typically, it will run a Linux-based operating system which allows you to customize to your heart's content. Excellent filesystems like ZFS (zettabyte filesystem) make sure data does not degrade from random disk errors, also known as bit rot and can allow you to address an enormous amount of storage. You can choose to make your server securely accessible over the open internet, and powerful second-hand servers can be acquired cheaply on eBay. This can allow for large scale data processing applications to be run locally on powerful hardware over which you have complete control. Open source designs like the aforementioned Backblaze storage pod can be a great place to start if you are looking for something that is entirely storage-focused. While not an option for running production applications unless you are at a scale where S3 or other cloud providers can no longer meet your needs, rolling your own servers can be a great way to store personal data and learn a lot about storage in the process.
Amazon S3 is a great way to store files and access files programmatically but falls short if you want to interface with your files manually or if you want to share files with non-technical colleagues. We have solved this problem at Publist by allowing users to connect their S3 buckets with our storage platform. By doing so, users can keep all of their files in S3 and pay for storage with their existing AWS credits while also benefiting from our integrations and intuitive frontend and teams functionality.