Storage

In this page we will look into the various factors that help you understand and analyze the costs for a storage account.

Factors that affect cost

• Volume of storage data

• The type of storage account

• The use of hot, cool and archive tiers

• Data redundancy

• Operations (read/write)

• Other features

  • SFTP

  • change tracking

  • Indexing

  • Restore

  • Encryption

  • etc

Things to Think about

What kind of application scenario do you have

Storage is a very flexible resource type in azure that can be used to support a number of different types of applications such as:

• Data lake storage for an enterprise data platform using heirachical namespace

• Blob storage as the backend for a frontdoor exposing content to a CDN

• Blob storage for internal PaaS application use cases

• Storage mounting on an App Service

• Table storage for a custom application for simple datasets

• Queue storage for back ground async processing for a custom application

• Backend for an SFTP service

• Storage for event driven applications that react to storage data changes

• Backup storage

• Storing datasets for AI

• File Sharing

• Media processing

• Long term log retention

These scenarios can vary significantly in terms of how much they will cost and how they will leverage the storage account

How many storage accounts will you have

It is highly unlikely you will have just 1 or 2 storage accounts.  Storage will probably be one of the most common Azure resource types you have.  You will use separate storage accounts for different application scenarios and its unlikely you will want a lot of sharing between different scenarios.

This is a good approach because in your storage account it is easy to get costing at the level of the storage account but its much harder to get cost at a lower level.

For many scenarios you will have storage account focused as part of a specific application where it will be easy to say “all of the costs in this storage account belong to application X”.  There will be use cases which are more complicated however such as an enterprise data platform or file sharing where you will have a storage account which contains data as a shared service used by multiple different applications.  In this scenario it will be difficult to workout what each applications contribution to the cost of the storage would be.

How will you allocate costs for data within the storage account

Following on from the above consideration, for these shared storage accounts, if you want to breakdown costs further you will need to think about how to evaluate what is inside the storage account.  This can be done either by considering the data in the storage account or by turning on logs and evaluating the logs for the storage account to get the right insights.

Because there are so many different metrics associated with the overall storage cost it is difficult to breakdown exact costs

Do I use logging or not?

If you turn on logs to do some deeper dive on cost management then you need to consider that there will be additional charges associated with logging.  Depending upon how frequent your storage account is accessed will influence how much this will cost.

What type of redundancy will I use?

You have many choices when it comes to the redundancy of your data.  Your requirements here will probably tie back to the type of application you are building.  Something like a data lake or a common file share solution is likely to require a much more resilient solution where you would accept additional cost for higher redundancy.

Other applications such as an app service mounting code from a storage account only needs LRS.  

It is easy to accidently choose the wrong option and then find your costs are not inline with your expectations.  Its also important to challenge your requirements.  If you look at the various SLA’s for each option you will see that all options offer the same availability but mainly differ in terms of durability.  Its a trade off decision here to match the redundancy option to the expected cost.  If you have lost of storage activity and capacity, its really amazing that you can tick a box and get redundancy across 2 regions and 4 data centres.  There is a cost for that however.  For many customers they would never have gone to that level when they had on-premise solutions but its easy to want the gold plated solution when its buy now and pay later.

The options are below:

What is your approach for access tiers?

Azure Storage has excellent features where you can build policies which will allow you to manage the archiving of data but its really more than that.  If you dont access data very often you can move it to the cool, cold and archive tiers where you will get a lower price for your storage.  You do trade off that there is a higher cost to access the data but if you dont need to access it often then this is a great way to save money easily.

Data Deletion Approach

• Access Tiers: Azure Storage provides different access tiers—Hot, Cool, Cold, and Archive—each optimized for specific data access patterns:

  • Hot Tier: Ideal for data that's accessed or modified frequently.

  • Cool Tier: Suited for data that's infrequently accessed or modified.

  • Cold Tier: Designed for data that's rarely accessed but requires fast retrieval.

  • Archive Tier: Best for data that's rarely accessed and has flexible latency requirements.

  Selecting the appropriate tier based on your data's access frequency can optimize costs.

  Microsoft Learn

• Redundancy Options: Azure offers several redundancy options to ensure data durability and availability:

  • Locally Redundant Storage (LRS): Replicates data within a single data center.

  • Zone-Redundant Storage (ZRS): Replicates data across multiple data centers within a region.

  • Geo-Redundant Storage (GRS): Replicates data to a secondary region for disaster recovery.

  • Read-Access Geo-Redundant Storage (RA-GRS): Provides read access to the secondary region.

  Higher redundancy levels offer greater durability but come at increased costs.

  Microsoft Learn

• Data Transactions: Operations such as read, write, and delete requests incur transaction costs. The volume and type of these operations can significantly affect your overall expenses.

  Microsoft Learn

• Data Retrieval and Egress: Retrieving data, especially from tiers like Cool, Cold, or Archive, can incur additional charges. Additionally, data transferred out of Azure data centers (egress) may result in bandwidth costs.

  Microsoft Learn

Common Optimizations

Am I paying for storage accounts I no longer use

Many companies have hundreds of storage accounts, many of which are old and no longer used.

Taking an audit of these accounts and validating that they are still used is a good way to clean up.  You might find storage accounts you are paying money for that contain lots of data that you just dont use.

Choose the right account kind

Azure currently has a v1 and v2 for general purpose storage accounts.  You may be able to get some benefits from moving to a V2 kind if you still have a V1.  Some of the key points are:

• V1 has a lower cost per GB stored for block blob

• V2 has lower transaction rates

• V2 has support for hot, cool and archive tiers

In general for most people you will be better off cost wise with a V2 account.

Lifecycle Management

Your data in storage can benefit from the life cycle management feature.  This allows you to configure rules to move data between tiers as it gets accessed less frequently.  You can also delete it.

This can have a big cost saving for little effort.  

Choose the right redundancy option

Ensuring you have chosen the appropriate redundancy option is a key element in being cost efficient.  Its so easy to just pick a redundancy option without thinking about it too much.  In this case you need to trade off between the needs for your requirements and the cost.

It makes sense to split data across storage accounts to allow you to choose the right options.

Will a reservation help

For a storage account you can get reserved capacity for Blob Storage & Files.

You would tend to use these if you are storing a lot of data and you purchase them in capacity of 100TiB and 1 PiB.  They are also relevant to the hot, cool or archive tier.

One of the good things with the reserved capacity is that it can have an appropriate scope allowing it to be shared across subscriptions.  This makes them very flexible.

Note there are some exclusions such as:

• Premium Storage

• V1 general purpose

• Table

• Queue

Review what data your actually storing

With a physical disk you have a hard limit to what you can store and there is a capital expense to purchase a new disk so people tend to be more cautious about what they store and free up space.  With Azure storage being cheap and easy to forget about, it is common that people dont really think about what they are storing.

If your storage costs are higher than expected its a good option to look into what you can just delete.

Use a Cache or CDN in the right cases

When an application is accessing a lot of data from storage then there could be a significant number of storage transactions.  You should consider options to cache the data to reduce the number of transactions on the storage account.

One good way to do this is using a CDN such as frontdoor which can cache things like images so that in addition to your users getting better performance you will reduce the storage transaction costs if your data is accessed from the CDN network cache.

How can Turbo360 help

Cost Analysis

In Turbo360 I am able to break up my costs into different scopes such as allowing each application team to have an area to easily manage their own costs.  Within this area the team can deeper dive and get insights on cost.  In the below graph I might be looking at the cost for each storage account used within my team.  I can see which of my 10 accounts is contributing the most cost.

I can also drill down, in the below example I am looking at the meters within the storage accounts cost.  I can look at the meters across all storage accounts of focus on just specific accounts.  This gives me an idea what kind of things are contributing to my cost.  In the below example you can see most of my cost related to data in the hot tier and the cost is focused on a combo of data stored and read operations.

Cost Monitoring

When using Turbo360 I am setting up budgets which will holistically monitor the costs for concepts like a team or application rather than just the physical asset.  This makes it much easier to monitor my costs and ensures that I wont forget about this.

In Turbo360 the group budget for your team and the anomaly detection will help reduce your risk so you keep on top of costs.

Admin Clean up Jobs

In addition to storage life cycle, some customers asked us about adding some jobs which can be ran on demand or on a schedule to perform clean up.  One such job is the storage workflow jobs which can be used to clean up data from your storage account at the end of a test window.

Useful Resources

In the below video we will look at an example where we were investigating a spike in storage costs.