Skip to content

Commit 44cd43f

Browse files
committedJul 2, 2021
update customer keys article, misc changes
1 parent 4bdb0b9 commit 44cd43f

9 files changed

+43
-27
lines changed
 

‎articles/hpc-cache/customer-keys.md

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: How to use Azure Key Vault with Azure HPC Cache to control encrypti
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: how-to
7-
ms.date: 07/20/2020
7+
ms.date: 07/01/2021
88
ms.author: v-erkel
99
---
1010

@@ -71,6 +71,8 @@ You must specify the encryption key source when you create your Azure HPC Cache.
7171
> [!TIP]
7272
> If the **Disk encryption keys** page does not appear, make sure that your cache is in one of the [supported regions](https://azure.microsoft.com/global-infrastructure/services/?regions=all&products=hpc-cache,key-vault).
7373
74+
![Screenshot of the completed Disk encryption keys screen, part of the cache creation interface in the portal.](media/customer-keys-populated.png)
75+
7476
The user who creates the cache must have privileges equal to the [Key Vault contributor role](../role-based-access-control/built-in-roles.md#key-vault-contributor) or higher.
7577

7678
1. Click the button to enable privately managed keys. After you change this setting, the key vault settings appear.
@@ -87,6 +89,15 @@ The user who creates the cache must have privileges equal to the [Key Vault cont
8789

8890
1. Specify the version for the selected key. Learn more about versioning in the [Azure Key Vault documentation](../key-vault/general/about-keys-secrets-certificates.md#objects-identifiers-and-versioning).
8991

92+
These settings are optional:
93+
94+
* Check the **Always use current key version** box if you want to use [automatic key rotation](../virtual-machines/disk-encryption.md#automatic-key-rotation-of-customer-managed-keys-preview).
95+
96+
* If you need to use a specific managed identity for this cache, select **User assigned** in the **Managed identities** section and select the identity to use. Read the [managed identities documentation](../active-directory/managed-identities-azure-resources/overview.md#managed-identity-types) for help.
97+
98+
> [!NOTE]
99+
> You cannot change the assigned identity after you create the cache.
100+
90101
Continue with the rest of the specifications and create the cache as described in [Create an Azure HPC Cache](hpc-cache-create.md).
91102

92103
## 3. Authorize Azure Key Vault encryption from the cache
@@ -104,11 +115,11 @@ This two-step process is necessary because the Azure HPC Cache instance needs an
104115
105116
The cache shows the status **Waiting for key**. Click the **Enable encryption** button at the top of the page to authorize the cache to access the specified key vault.
106117

107-
![screenshot of cache overview page in portal, with highlighting on the Enable encryption button (top row) and Status: Waiting for key](media/waiting-for-key.png)
118+
![Screenshot of cache overview page in portal, with highlighting on the Enable encryption button (top row) and Status: Waiting for key.](media/waiting-for-key.png)
108119

109120
Click **Enable encryption** and then click the **Yes** button to authorize the cache to use the encryption key. This action also enables soft-delete and purge protection (if not already enabled) on the key vault.
110121

111-
![screenshot of cache overview page in portal, with a banner message at the top that asks the user to enable encryption by clicking yes](media/enable-keyvault.png)
122+
![Screenshot of cache overview page in portal, with a banner message at the top that asks the user to enable encryption by clicking yes.](media/enable-keyvault.png)
112123

113124
After the cache requests access to the key vault, it can create and encrypt the disks that store cached data.
114125

@@ -120,17 +131,17 @@ You can change the key vault, key, or key version for your cache from the Azure
120131

121132
You cannot change a cache between customer-managed keys and system-managed keys.
122133

123-
![screenshot of "Customer keys settings" page, reached by clicking Settings > Encryption from the cache page in the Azure portal](media/change-key-click.png)
134+
![Screenshot of "Customer keys settings" page, reached by clicking Settings > Encryption from the cache page in the Azure portal.](media/change-key-click.png)
124135

125136
Click the **Change key** link, then click **Change the key vault, key, or version** to open the key selector.
126137

127-
![screenshot of "select key from Azure Key Vault" page with three drop-down selectors to choose key vault, key, and version](media/select-new-key.png)
138+
![Screenshot of "select key from Azure Key Vault" page with three drop-down selectors to choose key vault, key, and version.](media/select-new-key.png)
128139

129140
Key vaults in the same subscription and same region as this cache are shown in the list.
130141

131142
After you choose the new encryption key values, click **Select**. A confirmation page appears with the new values. Click **Save** to finalize the selection.
132143

133-
![screenshot of confirmation page with Save button at top left](media/save-key-settings.png)
144+
![Screenshot of confirmation page with Save button at top left.](media/save-key-settings.png)
134145

135146
## Read more about customer-managed keys in Azure
136147

‎articles/hpc-cache/hpc-cache-create.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ When you choose a throughput value, keep in mind that the actual data transfer r
5555

5656
The values you choose set the maximum throughput for the entire cache system, but some of that is used for overhead tasks. For example, if a client requests a file that isn't already stored in the cache, or if the file is marked as stale, your cache uses some of its throughput to fetch it from back-end storage.
5757

58-
Azure HPC Cache manages which files are cached and preloaded to maximize cache hit rates. Cache contents are continuously assessed, and files are moved to long-term storage when they're less frequently accessed.
58+
Azure HPC Cache manages which files are cached and pre-loaded to maximize cache hit rates. Cache contents are continuously assessed, and files are moved to long-term storage when they're less frequently accessed.
5959

6060
Choose a cache storage size that can comfortably hold the active set of working files, plus additional space for metadata and other overhead.
6161

‎articles/hpc-cache/hpc-cache-edit-storage.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,16 @@ description: How to edit Azure HPC Cache storage targets
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: how-to
7-
ms.date: 06/28/2021
7+
ms.date: 06/30/2021
88
ms.author: v-erkel
99
---
1010

1111
# Edit storage targets
1212

1313
You can modify storage targets with the Azure portal or by using the Azure CLI. For example, you can change access policies, usage models, and namespace paths for an existing storage target.
1414

15-
Storage target *management* tasks, like suspending or deleting a storage target, are described in [Manage storage targets](manage-storage-targets.md).
15+
> [!TIP]
16+
> Read [Manage storage targets](manage-storage-targets.md) to learn how to delete or suspend storage targets, or make them write cached data to back-end storage.
1617
1718
Depending on the type of storage, you can modify these storage target values:
1819

@@ -29,8 +30,7 @@ Depending on the type of storage, you can modify these storage target values:
2930

3031
You can't edit a storage target's name, type, or back-end storage system. If you need to change these properties, delete the storage target and create a replacement with the new value.
3132

32-
> [!TIP]
33-
> The [Managing Azure HPC Cache video](https://azure.microsoft.com/resources/videos/managing-hpc-cache/) shows how to edit a storage target in the Azure portal.
33+
The [Managing Azure HPC Cache video](https://azure.microsoft.com/resources/videos/managing-hpc-cache/) shows how to edit a storage target in the Azure portal.
3434

3535
## Change a blob storage target's namespace path or access policy
3636

‎articles/hpc-cache/hpc-cache-ingest.md

Lines changed: 16 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,29 +4,33 @@ description: How to populate Azure Blob storage for use with Azure HPC Cache
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: how-to
7-
ms.date: 10/30/2019
7+
ms.date: 06/30/2021
88
ms.author: v-erkel
99
---
1010

1111
# Move data to Azure Blob storage
1212

13-
If your workflow includes moving data to Azure Blob storage, make sure you are using an efficient strategy. You can either preload data in a new Blob container before defining it as a storage target, or add the container and then copy your data using Azure HPC Cache.
13+
If your workflow includes moving data to Azure Blob storage, make sure you are using an efficient strategy. You can either pre-load data in a new blob container before defining it as a storage target, or add the container and then copy your data using Azure HPC Cache.
1414

15-
This article explains the best ways to move data to Blob storage for use with Azure HPC Cache.
15+
This article explains the best ways to move data to blob storage for use with Azure HPC Cache.
16+
17+
> [!TIP]
18+
>
19+
> This article does not apply to NFS-mounted blob storage (ADLS-NFS storage targets). You can use any NFS-based method to populate an ADLS-NFS blob container before adding it to the HPC Cache. Read [Pre-load data with NFS protocol](nfs-blob-considerations.md#Pre-load-data-with-nfs-protocol) to learn more.
1620
1721
Keep these facts in mind:
1822

19-
* Azure HPC Cache uses a specialized storage format to organize data in Blob storage. This is why a Blob storage target must either be a new, empty container, or a Blob container that was previously used for Azure HPC Cache data.
23+
* Azure HPC Cache uses a specialized storage format to organize data in blob storage. This is why a blob storage target must either be a new, empty container, or a blob container that was previously used for Azure HPC Cache data.
2024

2125
* Copying data through the Azure HPC Cache to a back-end storage target is more efficient when you use multiple clients and parallel operations. A simple copy command from one client will move data slowly.
2226

23-
A Python-based utility is available to load content into a Blob storage container. Read [Pre-load data in Blob storage](#pre-load-data-in-blob-storage-with-clfsload) to learn more.
27+
A Python-based utility is available to load content into a blob storage container. Read [Pre-load data in blob storage](#pre-load-data-in-blob-storage-with-clfsload) to learn more.
2428

2529
If you don't want to use the loading utility, or if you want to add content to an existing storage target, follow the parallel data ingest tips in [Copy data through the Azure HPC Cache](#copy-data-through-the-azure-hpc-cache).
2630

27-
## Pre-load data in Blob storage with CLFSLoad
31+
## Pre-load data in blob storage with CLFSLoad
2832

29-
You can use the Avere CLFSLoad utility to copy data to a new Blob storage container before you add it as a storage target. This utility runs on a single Linux system and writes data in the proprietary format needed for Azure HPC Cache. CLFSLoad is the most efficient way to populate a Blob storage container for use with the cache.
33+
You can use the Avere CLFSLoad utility to copy data to a new blob storage container before you add it as a storage target. This utility runs on a single Linux system and writes data in the proprietary format needed for Azure HPC Cache. CLFSLoad is the most efficient way to populate a blob storage container for use with the cache.
3034

3135
The Avere CLFSLoad utility is available by request from your Azure HPC Cache team. Ask your team contact for it, or open a [support ticket](hpc-cache-support-ticket.md) to request assistance.
3236

@@ -42,20 +46,20 @@ A general overview of the process:
4246

4347
The Avere CLFSLoad utility needs the following information:
4448

45-
* The storage account ID that contains your Blob storage container
46-
* The name of the empty Blob storage container
49+
* The storage account ID that contains your blob storage container
50+
* The name of the empty blob storage container
4751
* A shared access signature (SAS) token that allows the utility to write to the container
4852
* A local path to the data source - either a local directory that contains the data to copy, or a local path to a mounted remote system with the data
4953

5054
## Copy data through the Azure HPC Cache
5155

52-
If you don't want to use the Avere CLFSLoad utility, or if you want to add a large amount of data to an existing Blob storage target, you can copy it through the cache. Azure HPC Cache is designed to serve multiple clients simultaneously, so to copy data through the cache, you should use parallel writes from multiple clients.
56+
If you don't want to use the Avere CLFSLoad utility, or if you want to add a large amount of data to an existing blob storage target, you can copy it through the cache. Azure HPC Cache is designed to serve multiple clients simultaneously, so to copy data through the cache, you should use parallel writes from multiple clients.
5357

54-
![Diagram showing multi-client, multi-threaded data movement: At the top left, an icon for on-premises hardware storage has multiple arrows coming from it. The arrows point to four client machines. From each client machine three arrows point toward the Azure HPC Cache. From the Azure HPC Cache, multiple arrows point to Blob storage.](media/hpc-cache-parallel-ingest.png)
58+
![Diagram showing multi-client, multi-threaded data movement: At the top left, an icon for on-premises hardware storage has multiple arrows coming from it. The arrows point to four client machines. From each client machine three arrows point toward the Azure HPC Cache. From the Azure HPC Cache, multiple arrows point to blob storage.](media/hpc-cache-parallel-ingest.png)
5559

5660
The ``cp`` or ``copy`` commands that you typically use to transfer data from one storage system to another are single-threaded processes that copy only one file at a time. This means that the file server is ingesting only one file at a time - which is a waste of the cache's resources.
5761

58-
This section explains strategies for creating a multi-client, multi-threaded file copying system to move data to Blob storage with Azure HPC Cache. It explains file transfer concepts and decision points that can be used for efficient data copying using multiple clients and simple copy commands.
62+
This section explains strategies for creating a multi-client, multi-threaded file copying system to move data to blob storage with Azure HPC Cache. It explains file transfer concepts and decision points that can be used for efficient data copying using multiple clients and simple copy commands.
5963

6064
It also explains some utilities that can help. The ``msrsync`` utility can be used to partially automate the process of dividing a dataset into buckets and using rsync commands. The ``parallelcp`` script is another utility that reads the source directory and issues copy commands automatically.
6165

‎articles/hpc-cache/manage-storage-targets.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: How to suspend, remove, force delete, and flush Azure HPC Cache sto
44
author: ekpgh
55
ms.service: hpc-cache
66
ms.topic: how-to
7-
ms.date: 06/28/2021
7+
ms.date: 07/01/2021
88
ms.author: v-erkel
99
---
1010

6.08 KB
Loading
Loading
5.63 KB
Loading

‎articles/hpc-cache/nfs-blob-considerations.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,8 @@ To work around this difference, Azure HPC Cache automatically disables NFS attri
3030

3131
This setting persists for the lifetime of the container, even if you remove it from the cache.
3232

33-
## Preload data with NFS protocol
33+
## Pre-load data with NFS protocol
34+
<!-- cross-referenced from hpc-cache-ingest.md and here -->
3435

3536
On an NFS-enabled blob container, *a file can only be edited by the same protocol used when it was created*. That is, if you use the Azure REST API to populate a container, you cannot use NFS to update those files. Because Azure HPC Cache only uses NFS, it can't edit any files that were created with the Azure REST API.
3637

@@ -44,7 +45,7 @@ If the files in your container were created with Azure Blob's REST API instead o
4445
* Empty the file (truncate it to 0).
4546
* Save a copy of the file. The copy is marked as an NFS-created file, and it can be edited using NFS.
4647

47-
Azure HPC Cache **can't** edit the contents of a file that was created using REST. This means that it can't save a changed file from a client back to the storage target.
48+
**Azure HPC Cache can't edit the contents of a file that was created using REST.** This means that the cache can't save a changed file from a client back to the storage target.
4849

4950
It's important to understand this limitation, because it can cause data integrity problems if you use read/write caching usage models on files that were not created with NFS.
5051

@@ -103,7 +104,7 @@ Azure HPC Cache can help improve performance in a workload that includes writing
103104
104105
One of the limitations outlined in the NFS-enabled blob [Performance considerations article](../storage/blobs/network-file-system-protocol-support-performance.md) is that ADLS-NFS storage is not very efficient at overwriting existing files. If you use Azure HPC Cache with NFS-mounted blob storage, the cache handles intermittent rewrites as clients modify an active file. The latency of writing a file to the back end container is hidden from the clients.
105106

106-
Keep in mind the limitations explained above in [Preload data with NFS protocol](#preload-data-with-nfs-protocol).
107+
Keep in mind the limitations explained above in [Pre-load data with NFS protocol](#pre-load-data-with-nfs-protocol).
107108

108109
## Next steps
109110

0 commit comments

Comments
 (0)
Please sign in to comment.