Saving Your Wallet With Lifecycle Rules Of course, storing multiple copies of objects uses way more space, especially if you’re frequently overwriting data. You probably don’t need to store these old versions for the rest of eternity, so you can do your wallet a favor by setting up a Lifecycle rule that will remove the old versions after some time. Under Management > Life Cycle Configuration, add a new rule. The two options available are moving old objects to an infrequent access tier, or deleting them permanently after

S3 object versioning Many of the strategies to be discussed for data durability require S3 object versioning to be enabled for the bucket (this includes S3 object locks and replication policies). With object versioning, anytime an object is modified, it results in a new version, and when the object is deleted, it only results in the object being given a delete marker. This allows an object to be recovered if it has been overwritten or marked for deletion. However, it is still possible for someone with sufficient privileges to permanently delete all objects and their versions, so this alone is not sufficient. When using object versioning, deleting old versions permanently is done with the call s3:DeleteObjectVersion, as opposed to the usual s3:DeleteObject, which means that you can apply least privilege restrictions to deny someone from deleting the old versions. This can help mitigate some issues, but you should still do more to ensure data durability. Life cycle policies Old versions of objects will stick around forever, and each version is an entire object, not a diff of the previous version. So if you have a 100MB file that you change frequently, you’ll have many copies of this entire file. AWS acknowledges in the documentation “you might have one or more objects in the bucket for which there are millions of versions”. In order to reduce the number of old versions, you use lifecycle policies. Audit tip: It should be a considered a misconfiguration if you have object versioning enabled and no lifecycle policy on the bucket. Every versioned S3 bucket should have a `NoncurrentVersionExpiration` lifecycle policy to eventually remove objects that are no longer the latest version. For data durability, you may wish to set this to 30 days. If this data is being backed up, you may wish to set this to as little as one day on the primary data and 30 days on the backup. If you are constantly updating the same objects multiple times per day, you may need a different solution to avoid unwanted costs. Audit tip: In 2019, I audited the AWS IAM managed policies and found some issues, including what I called Resource policy privilege escalation. In a handful of cases AWS had attempted to create limited policies that did not allow `s3:Delete*`, but still allowed some form of `s3:Put*`. The danger here is the ability to call `s3:PutBucketPolicy` in order to grant an external account full access to an S3 bucket to delete the objects and versions within it, or `s3:PutLifecycleConfiguration` with an expiration of 1 day for all objects which will delete all objects and their versions in the bucket. Storage classes With lifecycle policies, you have the ability to transition objects to less expensive storage classes. Be aware that there are many constraints, specifically around the size of the object and how long you have to keep it before transitioning or deleting it. Objects in the S3 Standard storage class must be kept there for at least 30 days until they can be transitioned. Further, once an object is in the S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA, those objects must be kept there for 30 days before deletion. Objects in Glacier must be kept for 90 days before deleting, and objects in Glacier Deep Archive must be kept for 180 days. So if you had plans of immediately transitioning all non-current object versions to Glacier Deep Archive to save money, and then deleting them after 30 days, you will not be able to.