A Little History
Hierarchical Storage Management or HSM is an architecture focused on placing the right data on the right storage for the right price. This solution decades ago was often called “Nearline” because data was all online but not on the same type of media. By using a Nearline or HSM approach organizations could manage more significant amounts of data at lower costs than keeping all data online all the time.
Over the last two decades, many solutions have attempted to create a seamless Nearline HSM solution matching data usage patterns with their performance requirement and an associated cost. One of the more recent solutions offering this architecture is Dell EMC Unity and the Cloud Tiering Appliance.
Cloud Tiering Appliance Technical Dive
The Cloud Tiering Appliance or CTA creates a seamless interface between the on-premise Dell EMC Unity Storage Array and an HSM Archive to Cloud. Customers can create a local tier of fast storage for a higher price per TB and a slower nearline tier of data in a cloud. The cloud can be a private cloud using a Dell EMC ECS or a public cloud provider such as AWS or Azure or a combination.
To make deploying the Cloud Tiering Appliance simple and highly available, we recommend using the Virtual Edition for VMware. This version is called the CTA/VE and its this configuration described in this post. I also recommend deploying the CTA/VE in an HA configuration ensuring availability of the service and the archive data it’s supporting.
The CTA/VE can be added and leveraged at any time by any Unity customer. Even if you had not considered the need or benefit of cloud archive when first deploying Unity this capability can be added when required. Each Unity array regardless of the model includes the license to use CTA, and there are no additional costs for CTA. There will be costs associated with VMware, Cloud Storage, and other pieces of supporting infrastructure for this solution and should be considered when undertaking this solution.
There are two Archive to Cloud solutions with CTA/VE: one for file data and one for block data. There are two unique solutions since the Dell EMC Unity provides both capabilities. Each Archive to Cloud solution, however, works very differently.
The file-based solution is most common when the term HSM is applied, and so I’ll begin with this solution first. Once the CTA/VE is configured, and an administrator designates a relationship with cloud storage an administrator must create a policy defining what files should remain local and what should be migrated to cloud. These policies include rules based upon the age of the file, size of the file, last modified, and many more. These elements can be combined to trigger the archive of a file from the Dell EMC Unity Storage Array to the Cloud target of choice. The time of trigger execution is also a policy giving administrators control over when archiving may occur. Often administrators will archive after that day’s backups are complete. Each file archived leaves behind an 8k stub file.
When reading the file back from Cloud Storage, administrators have a choice of either a Passthrough Recall (default), a Partial Recall, or a Full Recall Policy. A Passthrough Recall means the data never relocates to the Unity Storage Array but instead transfer blocks directly to the requestor. Partial or Full recall places either some or all of the data blocks back on the Unity Storage Array. Which strategy to choose depends on the reason for recalling the data.
The block-based solution is different in its requirements and applications. Snapshot copies of block-based volumes including consistency groups and thin clones can be stored in the cloud and expired upon scheduled policy. Making this possible requires a base image of the respective volume copied to the cloud and periodically updated. At any time a cloud-based snapshot can be copied back to the array and place into use as if it had always resided locally. There is no option to use the snapshot directly from the cloud.
Any of these policies can be run in simulation first to see if the expected outcome is accurate or if the policies need adjustments. The Cloud Tiering Appliance provides a practical HSM Archive to Cloud solution for file-based data and a vast snapshot repository for block-based data. This innovative approach to creating a hybrid cloud storage architecture helps optimize storage efficiency placing the right data in the right place for the right price.
Sizing
Sizing a Unity solution with or without the use of Cloud Storage requires capturing the proper environment metrics and understanding how Unity will apply them in its internal architecture. You can watch a video to learn how a Unity Storage Sizing works. Use this information to consider how much storage you would need on-premise if interested in leveraging the CTA for Cloud Storage. If you are unsure whether you have aging files and how often they are used Live Optics Dossier can be very helpful. You can learn about that here.
If you want to analyze the nature of your files and complete a sizing, contact us and let’s get started!
