S3 is billed as secure, durable and highly scalable object storage. Files sizes can be 1B to 5TB. S3 uses universal name spaces so names must be unique. As you probably know, object storage is suitable for storing flat files but not for installing an OS or database in. A S3 folder is known as a bucket.

S3 offers read after write consistency for new objects, i.e. after you upload a file, you can read it back immediately.

HOWEVER

eventual consistency only is availbale for updates or deletes, i.e. it may take some time for your changes to propogate.

S3 is a simple key, value store:

  • key - the name of the object
  • value - the actual data
  • version ID - a unique ID if versioning is being used
  • metadata - data about data, e.g. date of file
  • ACLs - access control lists controlling who can access files

S3 offers the following

  • 99.9% availabilty
  • 11 9s durability
  • tiered storage options
  • life cycle management
  • versioning
  • encryption
  • security using access control and bucket policies

Tiers

Different S3 tiers are available offering different levels of service

  • standard - 99% availabilty, 11 9s durability
  • IA - infrequently accessed files but required rapid retrieval
  • Reduced Redundancy Storage (RRS) - only 99.9% durabilty and 99.9% availabilty. So only suitable for recreatible files or files

that can be lost

  • Glacier - for archiving, takes 3-5 hours to retrieve a file

Charging depends on

  • amount of storage used (rate goes down for large usage)
  • number of requests
  • data transfer pricing

Versioning

Versioning can be turned on at the bucket level, however, once it's enabled it can't be turned off again, only suspended. Versioning will most likely lead to higher costs as more versions of a file are kept and so more storage is used.

One useful feature of versioning is the ability to retrieve a deleted file. When a file is deleted, rather than renoving the file, S3 creates a delete marker which means the file won't be displayed (you can see this by clicking on show tab by Versions. If you delete the “delete marker” entry, the file reappears (effectively restoring it).

Replication

Cross region replication of buckets can be enabled to duplicate your files. If you enable cross region replication for your bucket(s), versioning must be enabled.

One thing to note, if you enable replication, any existing objects won't get replicated, only new ones. If you want to replicate existing objects, you'll need to upload them again.

Life Cycle Management

This can be set up to transition objects from the standard tier to IA then to glacier then to permanently deleted. Or from standard tier straight to glacier. Some rules are in place comtrolling when this can take place.

  • To transition from standard to IA, a minimum of 30 days must have passed
  • To transition from standard straight to glacier, a, minumum of one day must have passed
  • To transition from IA to glacier, a further 30 days must have passed (so 60 in total)
  • Another day has to pass before an object can be permanently deleted

If you have versioning enabled, you'll see two options in Life Cycle Management. There'll be an option for the current version and for the previous version. To delete an object you'll need to expire the current version then delete the previous version.

S3 Security

S3 security can be applied using the following methods

  • Bucket Policies - apply to the whole bucket
  • ACLs - can be applied to objects within a bucket

S3 buckets can be set up to log access requests

S3 Encryption

S3 encryption can be set up for data intransit (using SSL/TLS) and/or at rest. For encryption for data at rest, this can be done on the server side using the following options:

  • SEE-S3 - S3 managed keys
  • SSE-KMS - added protection and audit trail
  • SSE-C - customer provided keys

Encrption at rest can also be provided on the client side, i.e. the data is encrypted before it's uploaded into S3

Recent Changes

Contribute to this wiki

Why not help others by sharing your knowledge? Contribute something to this wiki and join out hall of fame!
Contact us for a user name and password