Taking snapshots is an easy way to protect your system from data loss, whether on-prem or in the cloud. Cloud storage snapshots, like those from AWS EBS, are point-in-time copies of cloud volumes. However, keeping these copies consistent and retaining them for long periods of time can become a cost concern if you’re not careful, as copies are usually being created on a frequent basis.
The default design in backing up database storage in the public cloud is to backup to AWS EBS storage and store EBS snapshots of those backups to AWS S3 storage as shown here.
The other approach could have been EBS snapshot of EBS disks storing production database. But generally, databases don’t like this solution because it’s a crash consistent and not an application consistent backup.
Amazon EBS snapshots are created on an incremental basis. When a new snapshot is created, it only copies the block changes from the last snapshot and users are billed according to those incremental changes. For example, if the base storage size was 500 GB and only 3 GB of data has been modified since the last snapshot, then a new snapshot will only consume 3 GB and you are charged for a 3 GB snapshot. The three main parameters that define your AWS EBS snapshot costs for storage are baseline storage size, data growth rate, and the retention period of the snapshots. Here is the AWS pricing calculator.
However, it is important to understand the gotchas with public cloud snapshot pricing. The first snapshot is always going to be the largest in size as it is a full copy of the Amazon EBS volume and the consequent snapshots will be the changes from the last snapshot. This has a big impact on EBS snapshot costs.
An AWS EBS database snapshot introduces the following cloud cost components:
- AWS EBS Storage: to hold full crash consistent copy
- AWS Snapshot (local region): to store a snapshot in S3
- AWS Snapshot (remote region): to store snapshot in S3 in remote AWS region
- Data Transfer Charges: to replicate snapshots from local to remote AWS region.
Now enter NetApp’s Cloud Volumes ONTAP. Cloud Volumes ONTAP allows users to create application-aware snapshots that consume minimal EBS storage at no additional cost. When a snapshot is created, Cloud Volumes simply makes a copy of the root inode. This single 4KB copy is enough to protect all the data that is to be held in the snapshot, which makes the process very quick and extremely space efficient, regardless of whether your volume is a few megabytes in size or hundreds of terabytes. The snapshot acts as an online, read-only copy of the source data at the point-in-time in which it was created and is accessible just like a regular storage volume.
For block-level updates, Cloud Volumes will not update an existing block that is referenced by any snapshots. Instead, the updated data is written to a newly allocated block in the active filesystem, with inodes updated to now reference this new block instead of the old one. This is where the active filesystem and the snapshot start to diverge, with the old blocks required by the snapshot requiring additional storage space. The net effect of this is that the space required by the snapshot increases as the amount of data it shares with the active filesystem decreases.
We can see from this that NetApp Cloud Volumes snapshots are not only space efficient when they are created but also as data is changed. This is because instead of copying all the data in your system, they only copy any data that has changed by manipulating block pointers. Cloud Volumes ONTAP snapshots are extremely efficient in terms of both space and creation speed, more than any public cloud snapshot. Amazon EBS AWS snapshots, by comparison, must make an initial full copy of the data that acts as a baseline for subsequent incremental snapshots.
In addition to the benefits of Cloud Volumes ONTAP’s snapshots, the Cloud Volumes ONTAP storage efficiency features of thin provisioning, data deduplication, and data compression ensure that you can store your data as economically as possible. Not to mention data tiering that creates additional savings by allowing infrequently accessed data such as snapshots to be kept in inexpensive S3 storage. Together these benefits typically save a company as much as 50% to 70% on public cloud storage costs.
As an example, a 10 TB environment with a 1-week retention in EBS storage and 30-day retention of EBS snapshots in S3 can cost more than $5,000 per month!
So how do you manage your AWS snapshot costs?