Azure Storage is one of the broadly used service. We are going to use a blob container of a Storage Account to upload files in bulk. Let’s consider a scenario where 100 or 1000 of various files should be upload in blob storage before starting some actual work or testing.
It becomes more tedious if it has to perform in multiple environments, for example – Development, QA, Staging, load, etc. Azure PowerShell template is available in VSTS DevOps, and the script will get integrated with various release pipelines.
In the world of automation, who wants to spend time on such activities again and again? Let’s suppose, we would need to upload various types of file .txt, .docx, .xlsx, image, and pptx.
So, let’s automate it.
Implementation Steps
So, we would need perform below steps in the Azure PowerShell script.
- Connect Azure
- Create a Resource Group
- Create a Storage Account and a Blob Container
- Upload files to the Blob Container
Azure PowerShell Script
The script has a variable $numberOfBatch which is used in a for loop to create that many set of files.
The script will upload files by renaming it to a GUID value to have a unique name in the blob container.
Define some variables and create a resource group.
#Define Variables
$rgname = 'Bulk-Upload'
$rglocation = 'EastUS'
$storageaccountname = 'bulkuploadstorage'
$containerName = 'bulkblobcontainer'
$fileslocation = "C:\Temp\Files\"
$numberOfBatch = 10
#Create a resource group
New-AzResourceGroup -Name $rgName -Location $rglocation -Force
Create a Storage Account and a Blob Container
#Create a storage account
#and if already exists then skip the account creation
$storageAccount = Get-AzStorageAccount `
-ResourceGroupName $rgname `
-Name $storageaccountname `
-ErrorAction SilentlyContinue
$context = $storageAccount.Context
if(!$storageAccount)
{
$storageAccount = New-AzStorageAccount `
-ResourceGroupName $rgname `
-Name $storageaccountname `
-Location $rglocation `
-SkuName Standard_RAGRS `
-Kind StorageV2
$context = $storageAccount.Context
}
$container = Get-AzStorageContainer `
-Name $containerName `
-Context $context `
-ErrorAction SilentlyContinue
if(!$container)
{
$container = New-AzStorageContainer `
-Name $containerName `
-Context $context `
-Permission blob
}
Looping for the variable $numberOfBatch times and upload files which are available at the location variable $fileslocation.
#Upload files and get the files count
#uploaded in the blob container.
if($container)
{
$files = Get-ChildItem -Path $fileslocation
for ($i = 1 ; $i -le $numberOfBatch; $i++)
{
write-host -ForegroundColor Yellow "Running Batch: " $i
foreach ($file in $files)
{
$ext = (Split-Path -Path $file -Leaf).Split(".")[1];
Set-AzStorageBlobContent `
-File $file `
-Container $containerName `
-Blob $([guid]::NewGuid().ToString() + "." + $ext) `
-Context $context
}
}
$blob = Get-AzStorageBlob `
-Container $containerName `
-Context $context `
$totalfiles = $blob.Count
write-host -ForegroundColor Green "$totalfiles `
"files are uploaded successfully"
}
Script Output
To test the behavior of the script, we could combine all the script blocks into one PowerShell (.ps1) file and run it using the Windows Command Prompt.
Use Connect-AzAccount command to connect Azure subscription
If you want to upload only a set of files from a physical location, change the variable $numberOfBatch value to zero (0).
So, we have seen how we can upload files in bulk in a blob container using Azure PowerShell.
Happy Learning 🙂
I am facing an error while running this script on on prem server in powershell. There are couple of other errors as well but starts what i got first.
Get-AzStorageContainer : Could not get the storage context. Please pass in a storage context or set the current
storage context.
At line:1 char:14
+ $container = Get-AzStorageContainer
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzStorageContainer], InvalidOperationException
+ FullyQualifiedErrorId : InvalidOperationException,Microsoft.WindowsAzure.Commands.Storage.Blob.Cmdlet.GetAzureSt
orageContainerCommand
Hope you have installed Azure Powershell module and before running the script you much have connected to Azure Subscription from commpt using Connect-AzAccount