Skip to main content

Windows Azure Drives

The Azure Drive feature gives you access to data contained in an an NTFS-formatted virtual hard disk (VHD) persisted as a page blob in Azure Storage. Important to notice is that although you can access the drive using normal IO code, it is NOT a shared disk between all your Windows Azure instances. Only a single Azure instance can mount a page blob for read/write access as an Azure Drive. However, multiple Azure instances can mount a snapshot of a page blob for read-only access as an Azure Drive.

Mounting a VHD as an Azure drive

The VHD for an Azure Drive must be a fixed hard disk image formatted as a single NTFS volume. It must be between 16MB and 1TB in size. A VHD is a single file comprising a data portion followed by a 512 byte footer.  When uploading a VHD it is consequently important to remember to upload the footer. Furthermore, since pages of a page blob are initialized to 0 it is not necessary to upload pages in which all the bytes are 0. This could save a significant amount of time when uploading a large VHD.

The Disk Management component of the Windows Server Manager can be used to create and format a VHD by performing the following steps:

  • Right click on Disk Management and select Create VHD
  • Specify a fixed size VHD and provide a location and the size in MB
  • Right click on the disk icon and select Initialize Disk
  • Specify MBR (Master Boot Record)
  • Right click on the unallocated space and select New Simple Volume
  • Follow the Wizard choosing the defaults - specifically including the quick format with NTFS

The VHD is now created and accessible as a new drive using Windows Explorer. It can be used just like any other drive. The Detach VHD entry on the Disk Management right-click menu can be used to detach the VHD making it no longer accessible as a drive. This does not delete the VHD file. Similarly, the Attach VHD menu item can be used to once again make the VHD accessible as a file system.

Once created and detached, this VHD can be uploaded as a page blob to Azure Storage. Not all tools support uploading page blobs, but the Azure storage explorer should do the trick.

Using the Azure drive on an instance

To access the drive, we need a CloudDrive object. This object can be created using either the constructor or the CreateCloudDrive extension method to CloudStorageAccount. 

CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
CloudDrive cloudDrive = cloudStorageAccount.CreateCloudDrive(cloudDriveUri.AbsoluteUri);

These assume that the CloudStorageAccount.SetConfigurationSettingPublisher() as has been invoked previously. Note that these only create an in-memory representation of the Azure Drive which still needs to be mounted before it can be used.

Some important considerations:

  • A VHD page blob can be mounted on only one instance at a time.
  • A VHD snapshot can be mounted readonly to an unlimited number of instances.

Before a VHD page blob can be mounted it is necessary to allocate some cache space in the local storage of the instance. This is necessary even if caching is not going to be used. This caching is a read cache. Local storage must be defined in the Azure Service Definition file as explained in this post and should be configured with cleanOnRoleRecycle set to false. 

<LocalResources>
<LocalStorage name="CloudDrives" cleanOnRoleRecycle="false" sizeInMB="50" />
</LocalResources>

InitializeCache() must be invoked to initialize the cache with a specific size and location. 

public static void InitializeCache()
{
LocalResource localCache = RoleEnvironment.GetLocalResource("CloudDrives");
CloudDrive.InitializeCache(localCache.RootPathlocalCache.MaximumSizeInMegabytes);
}

An instance mounts a writeable Azure Drive by invoking Mount() on a VHD page blob. The Azure Storage Service uses the page blob leasing functionality to guarantee exclusive access to the VHD page blob. An instance mounts a read-only Azure Drive by invoking mount() on a VHD snapshot. Since it is read-only it is possible for multiple instances to mount the VHD snapshot simultaneously. An instance invokes the Unmount() method to release the Azure Drive and for VHD page blobs allow other instances to mount the blob for write access.

The cacheSize parameter to Mount() specifies how much of the cache is dedicated to this Azure Drive. The cacheSize should be set to 0 if caching is not desired for the drive. Different Azure Drives mounted on the same instance can specify different cache sizes and care must be taken that the total cache size allocated for the drives does not exceed the amount of cache available in local storage. The options parameter takes an DriveMountOptions flag enumeration that can be used to force the mounting of a drive – for example, when an instance has crashed while holding the lease to a VHD page blob – or to fix the file system. Mount() returns the drive letter, or LocalPath, to the Azure Drive - for example, "d:" - which can be used to access any path on the drive. 

public void WriteToDrive( Uri cloudDriveUri )
{
CloudStorageAccount cloudStorageAccount =
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
cloudDrive = cloudStorageAccount.CreateCloudDrive(cloudDriveUri.AbsoluteUri); 

String driveLetter = cloudDrive.Mount(CacheSizeInMegabytes, DriveMountOptions.None); 

String path = String.Format("{0}\\hellodrives.txt", driveLetter);
FileStream fileStream = new FileStream(path, FileMode.OpenOrCreate);
StreamWriter streamWriter = new StreamWriter(fileStream);
streamWriter.Write("that you have but slumbered here");
streamWriter.Close(); 

cloudDrive.Unmount();
}

GetMountedDrives() provides access to a list of drive letters for all Azure Drives mounted in the instance. It is used as follows: 

public static void EnumerateDrives()
{
IDictionary<String,Uri> listDrives = CloudDrive.GetMountedDrives();
foreach (KeyValuePair<String,Uri> drive in listDrives)
{
String driveInformation = String.Format("drive: {0} - uri: {1}", drive.Key, drive.Value.AbsoluteUri);
Trace.WriteLine(driveInformation, "Information");
}
}

Labels:

Popular posts from this blog

Podman– Command execution failed with exit code 125

After updating WSL on one of the developer machines, Podman failed to work. When we took a look through Podman Desktop, we noticed that Podman had stopped running and returned the following error message: Error: Command execution failed with exit code 125 Here are the steps we tried to fix the issue: We started by running podman info to get some extra details on what could be wrong: >podman info OS: windows/amd64 provider: wsl version: 5.3.1 Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM Error: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:2655: connectex: No connection could be made because the target machine actively refused it. That makes sense as the podman VM was not running. Let’s check the VM: >podman machine list NAME         ...
Read more

Cache stampede: when our cache turned against us

While investigating some performance issues, we ran into an ASP.NET Core API that cached a fairly expensive aggregation query for 60 seconds. Under normal load, that was fine: one request rebuilds the cache, everyone else reads from it. Under peak load, dozens of requests would arrive in that same expiry window, all see a cache miss, and all fire the same expensive query in parallel. The database didn't like that. That was the moment when our caching layer stopped helping and started hurting. A burst of requests comes in at the same time, all miss the cache, and all go hammer the database or the downstream API at once. That's a cache stampede . The cache was supposed to protect our backend, and for a few hundred milliseconds it did the opposite. Why this happens IMemoryCache.GetOrCreate (and its async sibling) looks like it protects you, but it doesn't add any locking on its own. Look at the naive version: public async Task<Report> GetReportAsync(string key) ...
Read more

A complex system designed from scratch never works

A few years ago, I worked as an architect on a big mainframe rewrite. I still count it as one of my failures. Not because the technology was wrong, but because I couldn't convince the management team to simplify the approach. Years later, the organization is still struggling to get the new system up and running. I left the project at the time, because I couldn't put my name behind an approach that would take very long and cost a lot of money without a working system to show for it along the way. Gall’s Law That memory keeps coming back to me, because it's a textbook case of Gall's Law playing out in real life. Gall's Law , from John Gall's Systemantics , states it plainly: A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works, and it cannot be patched to make it work. You have to start over with a simple system that works. What does that mean in practice,...
Read more