Azure File Sync - Introduction
In our previous article, we discussed the concept of Azure File Systems and how they are analogous to network attached storage in on premises data center environments. We also configured a file share and then mounted it on an Ubuntu server via the SMB/CIFS protocol.
In this article, we’ll continue our exploration of file services in the Azure cloud and focus our discussion on Azure file sync.
- We could Use Azure File Sync to centralize our organization's file shares in Azure Files, while keeping the flexibility, performance, and compatibility of an on-premises file server.
- Azure File Sync transforms Windows Server into a quick cache of our Azure file share.
- Azure File Sync is a part of the Azure Storage service and we could use to extend the capabilities of Azure Files into our on-premise file servers. So rather than creating a direct link to our Azure file share, we could use Azure File Sync to increase the storage capacity of our on-premise file servers by locally caching frequently accessed files on-prem on our file servers.
- When those files that are not accessed as frequently, we would have a pointer mapping back to the file on our Azure file share where it is actually stored and can be accessed. If there is a request to access the particular file then it will get locally cached again.
- As of this writing, the Azure File Sync service is only supported with Windows 2012 R2 or later, and we only can use SMB, NFS, and FTPS protocols with our Azure File Sync service. We need to use a file sync agent on our Windows machine in order to set this up.
Advantage of using Azure File Sync
Given below are some use cases and advantages to file sync.
- Lift and shift: The ability to move applications that require access between Azure and on-premises systems. Provide write access to the same data across Windows Servers and Azure Files. This lets companies with multiple offices have a need to share files with all offices.
- Branch Offices: Branch offices need to backup files, or you need to setup a new server that will connect to Azure storage.
- Backup and Disaster Recovery: Once File Sync is implemented, Azure Backup will back up your on-premises data. Also, you can restore file metadata immediately and recall data as needed for rapid disaster recovery.
- File Archiving: Only recently accessed data is located on local servers. Non-used data moves to Azure in what is called Cloud Tiering.
Cloud tiering is an optional feature of Azure File Sync in which frequently accessed files are cached locally on the server while all other files are tiered to Azure Files based on policy settings. When a file is tiered, the Azure File Sync file system replaces the file locally with a pointer, or reparse point. The reparse point represents a URL to the file in Azure Files.
When a user opens a tiered file, Azure File Sync seamlessly recalls the file data from Azure Files without the user needing to know that the file is actually stored in Azure. Cloud Tiering files will have greyed icons with an offline file attribute to let the user know the file is only in Azure.
Components of Azure File Sync
To gain the most from Azure File Sync, it's important to understand the terminology.
1. Storage Sync Service
The Storage Sync Service is the top-level Azure resource for Azure File Sync. The Storage Sync Service resource is a peer of the storage account resource, and can similarly be deployed to Azure resource groups. A distinct top-level resource from the storage account resource is required because the Storage Sync Service can create sync relationships with multiple storage accounts via multiple sync groups. A subscription can have multiple Storage Sync Service resources deployed.
2. Sync group
A sync group defines the sync topology for a set of files. Endpoints within a sync group are kept in sync with each other. If, for example, you have two distinct sets of files that you want to manage with Azure File Sync, you would create two sync groups and add different endpoints to each sync group. A Storage Sync Service can host as many sync groups as you need.
3. Registered server
The registered server object represents a trust relationship between your server (or cluster) and the Storage Sync Service. You can register as many servers to a Storage Sync Service instance as you want. However, a server (or cluster) can be registered with only one Storage Sync Service at a time.
4. Azure File Sync agent
The Azure File Sync agent is a downloadable package that enables Windows Server to be synced with an Azure file share. The Azure File Sync agent has three main components:
- FileSyncSvc.exe: The background Windows service that is responsible for monitoring changes on server endpoints, and for initiating sync sessions to Azure.
- StorageSync.sys: The Azure File Sync file system filter, which is responsible for tiering files to Azure Files (when cloud tiering is enabled).
- PowerShell management cmdlets: These are PowerShell cmdlets that you to interact with the Microsoft. StorageSync Azure resource provider. You can find these at the following (default) locations:
C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.PowerShell.Cmdlets.dll C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll
5. Server endpoint
A server endpoint represents a specific location on a registered server, such as a folder on a server volume. Multiple server endpoints can exist on the same volume if their namespaces do not overlap (for example, F:\sync1 and F:\sync2). You can configure cloud tiering policies individually for each server endpoint. You can create a server endpoint via a mountpoint. Note, mountpoints within the server endpoint are skipped.
You can create a server endpoint on the system volume but, there are two limitations if you decide to do so:
- Cloud tiering cannot be enabled.
- Rapid namespace restore (where the system quickly brings down the entire namespace and then starts to recall content) is not performed.
6. Cloud endpoint
A cloud endpoint is an Azure file share that is part of a sync group. The entire Azure file share syncs, and an Azure file share can be a member of only one cloud endpoint. Therefore, an Azure file share can be a member of only one sync group. If you add an Azure file share that has an existing set of files as a cloud endpoint to a sync group, the existing files are merged with any other files that are already on other endpoints in the sync group.
Configuring Azure File Sync
Step-1: Create Azure File Share
We’ve already configured an Azure File Share named filesync to use for this demonstration.
Step-2: Create Windows VM
Next, we’ll create a Windows virtual machine comprising Windows Server 2019 to use as our server end point with file sync.
We’ll keep the VM size as the default for the windows server 2019 instance, specify a user name and password for our administrator user account and keep ports 80, 443 and 3389 open.
Click on Next: Disks section.
Here we are going to click on Create and attach a new disk. We're going to use this disk as a volume to map our file share for the file sync. This will open the below page where could adjust the disk settings including the disk size.
We’ll accept the defaults and click on ok. This will bring us back to the create virtual machine page. Since we don’t need to modify any other settings, we’ll now click on review and create.
Step-3: Create Azure File Sync service
With this we have created our virtual machine and our file share. Next, we’ll create our Azure file sync service. For this, type file sync in the search bar and select Azure file sync in the market place.
When creating a storage sync service, it’s important to keep in mind that the file sync service must be in the same region as the storage account.
In our case, we created the storage account in Central US so we’ll create the storage sync service in this region. We need to provide a name for our sync service along with the subscription and resource group information. Once done, click on the review + create button at the bottom left of the screen.
Once the storage service creation process completes, we could click on go to resource and it would bring us to the storage sync service page.
Step-4:Register Windows VM with Azure File Sync Service
As we could observe here, we don’t have any sync groups configured at the moment. Before we configure a sync group here, we need to prep our virtual machine. So we’ll navigate to the virtual machines page in the Azure portal and connect to the virtual machine using RDP.
This will bring us to the below page where we’ll need to download the RDP file to be able to initiate a connection.
We’ll download this file and double click it to open up an RDP connection using an RDB client into our Windows virtual machine. Next, we’ll enter our credentials to login to the Windows Server 2019 VM.
Once we login to the VM, we’ll go to Server Manager, select Local Server and turn off IE Enhanced Security Configuration.
Next, we will go to File and Storage Services where we can view our data disks including the additional disk that we added during the VM creation.
That disk is disk number 2. We’ll right click on this disk and select new volume.
This will start the volume creation wizard. We’ll continue to accept the defaults. Do make a note of the drive letter being assigned to the disk in the process.
In our case, the new disk will be mapped to drive letter F. Continue to select the default options further along the process. Once the volume creation completes, click on close.
Next, we will open powershell from the start up menu and inside the powershell prompt, we’ll type the below command to install the Az module for Powershell.
Module-Install -Name Az
Accept the defaults when prompted. The installation will take a while to complete.
The next thing that we need to do here is go into our File Explorer and create our endpoint in the F drive that we created earlier. We're going to be mapping as part of our sync group. We'll go to this PC. We'll go to our new volume that we created and we're going to create a new folder called the SyncFolder.
We're going to go into this folder and create a new text file and name it SyncFile.
Once we've done that, we're going to go to Internet Explorer and whenever it's prompted with the IE security configuration, we'll click Ask Me Later, and then type, install Azure file sync agent. We’ll open the URL containing the download link and click on download button.
This will give us a couple of download options. We’ll select the one for our OS type i.e., Server 2019.
Once the download completes, we’ll select the downloaded file and select run when prompted. This will launch the file sync agent setup wizard.
We simply need to select the defaults as move through the prompts. Once the installation finishes you should see the following prompt on the screen.
We will now be asked to sign in and register the server. We would select Azure Cloud as our Azure Environment and then click on sign in. This will prompt us for our credentials to the Azure cloud.
After signing in, we will be asked to choose a storage sync service where we’ll need to select the Azure subscription, Resource Group and the Storage sync service that we had created earlier.
Select the appropriate options and click on register. Once the registration completes, the file sync agent will perform a network connectivity test and if it passes, we should see the below message on the screen.
Now, we're going to go back to our Storage Sync Services by typing in storage sync and select it under Services. And then we're going to select on the storage sync service we created.
In the management page for our sync service demosyncss, we’ll click on the registered servers tab under the sync section on the left to find that our Windows server has successfully been registered.
Step-5: Crate sync group
We're going to create a sync group for our storage sync service by clicking on the sync group tab in the sync section of our storage service management page.
Click on the sync group button to create a new sync group. We're going to name our sync group as syncgroup and then select our storage account that we created earlier and the Azure File Share we created in that storage account.
Once we’ve filled in the required information, we’ll click on the create button. Once the sync group is created, it will contain a cloud end point being our file share and a server end point being our Windows Server 2019 virtual machine that we created and the data contained in the two would be in sync.
Summary
In this article, we discussed the different features and advantages of the Azure File Sync service. We then went through the different steps involved in configuring Azure file sync namely the creation of a windows server virtual machine and installing pre-requisites (Azure PowerShell modules and file sync agent), a file share and the storage service with the windows server VM and the file share as server and cloud end points respectively inside a sync group.
References
We referred to the Microsoft Azure official documentation for the writing of this article links to which have been shared below
Extend your on-premises file share capacity using Azure File Sync
What is Azure File Sync?
Deploy Azure File Sync
