Azure Storage – What is Lifecycle Management and implementation

Lifecycle management is a transition and retention activity. This helps to take necessary action based on the aging of blobs within storage account.

Azure Storage Lifecycle management comes with set of policies. A policy is collection of rules and each rule definition has a filter set and an action set. Filters are used to mark a subset of blobs in the storage account and rules are applied on the subsets.

Let’s take a scenario that an Azure service (Ex- Web App or Azure Data Factory) is pushing 1GB log data everyday to blob storage and you are not doing any cleanup activity, neither manually nor automatically. So over the period of time data will grow and you will pay for the data which may become irrelevant.

Points to Remember

• Lifecycle Management is not applicable to Premium Performance tires.
• Policies can be applied to all the blobs or limit blobs with filter. set
• Policies can be applied to Hot and Cool tires storage account, block and append blob and their subtypes base blob, Snapshots and Versions.
• Base blobs, Snapshots and Versions can be filtered based on date modified / created with Delete actions.
• A policy can be created by choosing all the options or their combinations.

Implementation

To start the implementation, you would need a storage account with below configurations –

• Account kind as StorageV2 (general purpose v2) with standard tire and locally-redundant storage(LRS).
• Performance/Access tier as Standard/Hot
• Replication as Locally-redundant storage (LRS)
• Enable versioning for blobs from Data Protection options

If you haven’t enabled blob versioning while creating the storage account or you have an existing storage account, then go to Storage Account > Data Protection > Check Enable versioning for blobs and Save the changes.

Data Preparation – Once storage account is ready, you need to do data preparation. Now, create a container<container01> and upload some .TXT files with some text and create snapshots of the files. You need to repeat this steps multiple times so that you can have some multiple snapshots and versions of the files to run the life cycle management policy.

Let’s create three policies for block blob to target sub type base blobs, snapshots and versions. All the policies are created based on only one condition for created/modified days ago. Conditions can be changed or added based on the requirements.

Move to Cool Tire

To create the first rule which will move base blob to cool tire if modified more than a particular ago. Follow the below configuration as shown in the screenshots and click on Add button on the wizard.

This is used when blobs are accessed occasionally and should be in cool storage for at least 30 days.

Move to Archive

To create the second rule which will move snapshots to Archive tire if created more than particular day ago. Follow the below configuration as shown in the screenshots and click on Add button on the wizard.

This is used when blobs are not accessed at all and should be in archive tire for 180 days or more.

Delete Permanently

To create the third rule which will permanently delete blob versions if created more than particular day ago. Follow the below configuration as shown in the screenshots and click on Add button on the wizard.

This is used when blobs are no longer accessed or of no use then based on certain conditions blobs can be deleted.

So, now you should see three life management rules. These rules are executed once a day so if you create a rule then it can take up to 24 to 48 hours to run the action very first time.

Output

These rules are applicable to the Base blobs, Snapshots and Versions with an action criteria. Let’s take couple of examples.

1. If you have created a file each day in the container in last one month. Today, you are going to add rule to move files to cool tire which were modified 7 days ago, then the rule will move all the files to cool tire which are modified before 7 days.
2. If you have created a snapshots in last one month. Today, you are going to add rule to move files to Archive tire which were created day ago is 7, then the rule will move all the files to Archive tire which were created before 7 days.

*Versions to Delete rule output can not be captured because files are no longer present.

When more than one action associated to blobs then Lifecycle management applies the least expensive action to the blobs which means delete is cheaper than TireToArchive. TireToArchive is cheaper than TireToCool.

So, we have seen Azure Storage lifecycle management policies and how to create the policy for the base blobs, snapshots and versions.

If you have any suggestions/feedback, please put them in the comment box.

Happy Learning 🙂