AWS storage: Key storage options in the Amazon cloud

November 12

Amazon Web Services (AWS) pioneered mass market, large-scale and cost-effective cloud storage when it entered the market back in 2006.

The hyperscaler’s Simple Storage Service (S3) set out to make storing data in the cloud as simple as possible, at least for end users. As Amazon CTO Werner Vogels commented on S3’s 15th anniversary in 2021: “It”s ironic because what we were trying to do – store data on the internet, and do it really well – was not so simple…For customers, it had to be ‘simple’, but designing and building S3 wasn’t.”

S3 is designed to provide storage functionality, without the bells and whistles of competing online services. But it was also designed to work equally well as the storage component for other services or applications. As a result, S3 is now the centre of a family of file, block and object cloud storage products.

AWS: Storage options

Since 2006, S3 has grown significantly. In 2021, there were 100 trillion objects in S3 buckets and that figure will have grown significantly since.

S3 is based on object storage, but AWS now offers a range of other options. For file, these include Amazon Elastic File System (EFS), Amazon FSx, and Amazon File Cache.

Amazon Elastic Block Store serves the needs of AWS block storage users, while the hyperscaler provides a range of specialist options. These include DataSync to connect AWS instances and on-premise storage, Snow for hybrid and edge applications, AWS Elastic Disaster Recovery and AWS backup.

Glacier – technically part of S3 – provides long-term data archiving.

Object storage and archiving: Amazon S3

Amazon S3 provides a range of storage classes. These include S3 Intelligent-Tiering, S3 Standard and S3 Express One Zone, as well as S3 for infrequent access in standard and one zone formats.

For longer-term storage and archiving, AWS offers S3 Glacier with instant or flexible retrieval, or deep archiving.

AWS Outposts provides object storage for on-premise environments, but using S3 application programming interfaces (APIs).

Each S3 class has its own data access levels, costs and geographic locations. S3 Intelligent-Tiering aims to optimise customer storage budgets by automatically moving data between AWS’s three low latency access tiers.

S3 provides 11-nines data durability, with data stored across three availability zones by default.

Pricing starts at $0.024 per GB per month for the first 50TB (based on Europe/London pricing), falling to $0.022 per GB for more than 500TB. Intelligent tiering ranges from $0.024 per GB per month, again for the first 50TB, for the frequent access tier, down to $0.005 per GB for Archive Instant Access. There is a small automation and monitoring charge of $0.0025 per 1,000 objects for Intelligent-Tiering.

Amazon S3 performance supports “at least 3,500 requests per second to add data and 5,500 requests per second to retrieve data”, according to Amazon.

Block storage: EBS

Amazon Elastic Block Store supports high-performance workloads including databases, enterprise resource planning (ERP) and Microsoft technologies.

EBS can be provisioned as General Purpose, Provisioned IOPS for high performance, Throughput Optimised for large data volumes, and Cold HDD for infrequent access and cold data workloads. Throughput Optimized is hard disk-based; EBS io2 Block Express volumes run on SSDs and support 256,000 IOPS and 4,000 MBps of throughput per volume.

Pricing ranges from $0.0928/GB per month (based on Europe/London) for general purpose SSDs and $0.0058 for provisioned IOPS (after a 3,000 IOPS free allowance) to $0.053 per GB per month of provisioned storage for Throughput Optimized HDDs and $0.0174 per GB per month for Cold HDDs.

EBS also provides snapshots in standard and archive tiers at an additional cost.

File storage: EFS and FSx

For file storage, AWS offers EFS in standard, cost-optimised and archive tiers. Standard is based on SSDs with “sub-millisecond latency performance”. Standard is designed for active data workloads, Infrequent for data accessed “only a few times each quarter” and Archive for a few times a year.

Infrequent Access and Archive tiers have tens of milliseconds latency; for EFS archive the minimum storage duration is 90 days.

Performance for OpenZFS and Windows File System is up to 21 GBps of throughput and more than a million IOPs for frequently accessed, cached data.

Costs are $0.33 per GB per month for Standard (Europe/London, regional, multi-AZ), $0.02/GB for Infrequent Access and $0.01 for Archive. However, there are separate charges for access. For Standard these are $0.03/GB for reads and $0.07 for writes, with tiering charges where that is used. One Zone (Single-AZ) fees are lower.

With FSx, AWS allows customers to choose between four file systems: NetApp ONTAP, OpenZFS, Windows File Server and Lustre.

Pricing depends on the exact file system and usage. Taking ONTAP as an example, these are made up of provisioned SSD storage, IOPS (three IOPS are included for each GB of storage), capacity pool usage and throughput capacity. Firms can also opt for backups and SnapLock (snapshot) copies on a pay-as-you-go basis.

AWS offers pricing calculators for each of the file systems.

Along with object, block and file, AWS provides a range of more specialist backup, archiving and data migration services, as well as the ability to link storage services together. FSx for Lustre, for example, can be linked to S3 for big data processing.

Source link

#AWS #storage #Key #storage #options #Amazon #cloud

Pelican News

Source Link