Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. In Azure, you can create a model using the Machine Learning Service or use a model from an Open Source platform like TensorFlow or Pytorch.
If you’re implementing Machine Learning(ML) Operations within your organization to bring ML models to production, then you can benefit from Azure Machine Learning as you will be able to:
- Leverage Azure compute resources
- Share data, notebooks, and environments
- Keep track of the versions of your model
Azure Machine Learning can be integrated with other services to support your project, like Azure Synapse Analytics to process and stream data with Spark. You can also integrate Azure SQL Database, Storage Blobs, and App Services to deploy and manage ML-powered apps.
Your entire ML project can be managed with Azure Machine Learning Service. It provides a workspace that allows for collaboration for many users. You can share the results of the runs from the experiments and use versioned assets for jobs like environments and storage references.
Through Azure Machine Learning, you will be able to manage your complete project lifecycle, including tasks definitions, data preparation, training, validation of your models, deployment of models, monitoring and publishing models.
Before working with Azure Machine Learning, we have to create a workspace and then add compute resources to the workspace.
The workspace is the top resource for all your machine learning activities and a centralized place to view and manage artifacts.
The image below shows a high-level architecture of Azure Machine Learning and the components included in the workspace:
That said, we will use Bicep to deploy the Azure Machine Learning Workspace.
The Bicep file will include the following components:
- Application Insights
- Container Registry
- Key Vault
- Machine Learning workspace
- Storage account
We will define a few parameters for the deployment’s name, environment, and location. We will also define a parameter bool that specifies whether to reduce telemetry collection and enable additional encryption.
The code below shows the parameter definition:
@description('Specifies the name of the deployment.')
param name string@description('Specifies the name of the environment.')
param environment string@description('Specifies the location of the Azure Machine Learning workspace and dependent resources.')
param location string = resourceGroup().location@description('Specifies whether to reduce telemetry collection and enable additional encryption.')
param hbi_workspace bool = false
Now we will define the variables section.
The code below shows the definition of the variables to be utilized. We will use variables to specify the name of the resources to be created:
var tenantId = subscription().tenantId
var storageAccountName_var = 'st${name}${environment}'
var keyVaultName_var = 'kv-${name}-${environment}'
var applicationInsightsName_var = 'appi-${name}-${environment}'
var containerRegistryName_var = 'cr${name}${environment}'
var workspaceName_var = 'mlw${name}${environment}'
var storageAccount = storageAccountName.id
var keyVault = keyVaultName.id
var applicationInsights = applicationInsightsName.id
var containerRegistry = containerRegistryName.id
Now let’s define the resources.
We will define the following resources:
- Storage Account
- Key Vault
- Application Insights
- Container Registry
- Machine Learning Service Workspace
The code below shows the definition of the resources.
resource storageAccountName 'Microsoft.Storage/storageAccounts@2021-01-01' = {
name: storageAccountName_var
location: location
sku: {
name: 'Standard_RAGRS'
}
kind: 'StorageV2'
properties: {
encryption: {
services: {
blob: {
enabled: true
}
file: {
enabled: true
}
}
keySource: 'Microsoft.Storage'
}
supportsHttpsTrafficOnly: true
}
}resource keyVaultName 'Microsoft.KeyVault/vaults@2021-04-01-preview' = {
name: keyVaultName_var
location: location
properties: {
tenantId: tenantId
sku: {
name: 'standard'
family: 'A'
}
accessPolicies: []
enableSoftDelete: true
}
}resource applicationInsightsName 'Microsoft.Insights/components@2020-02-02' = {
name: applicationInsightsName_var
location: (((location == 'eastus2') || (location == 'westcentralus')) ? 'southcentralus' : location)
kind: 'web'
properties: {
Application_Type: 'web'
}
}resource containerRegistryName 'Microsoft.ContainerRegistry/registries@2019-12-01-preview' = {
sku: {
name: 'Standard'
}
name: containerRegistryName_var
location: location
properties: {
adminUserEnabled: true
}
}resource workspaceName 'Microsoft.MachineLearningServices/workspaces@2021-07-01' = {
identity: {
type: 'SystemAssigned'
}
name: workspaceName_var
location: location
properties: {
friendlyName: workspaceName_var
storageAccount: storageAccount
keyVault: keyVault
applicationInsights: applicationInsights
containerRegistry: containerRegistry
hbiWorkspace: hbi_workspace
}
dependsOn: [
storageAccountName
keyVaultName
applicationInsightsName
containerRegistryName
]
}
Note that we define a few dependencies in the workspace to ensure all the rest of the resources needed to deploy the workspace exists before deploying the workspace resource type.
Now we will define the parameters file. This file will contain only a few parameters: the name of the deployment, the name of the environment, and the location.
The code below shows the definition of the parameters file.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"name": {
"value": "azinsiderb"
},
"environment": {
"value": "azinsiderb"
},
"location": {
"value": "eastus"
}
}
}
Now we will proceed to deploy this resource. We previously created a resource group called ‘AzInsiderBicep’. We will use the following code to deploy this Bicep file:
$date = Get-Date -Format "MM-dd-yyyy"
$deploymentName = "AzInsiderDeployment"+"$date"New-AzResourceGroupDeployment -Name $deploymentName -ResourceGroupName AzInsiderBicep -TemplateFile .main.bicep -TemplateParameterFile .azuredeploy.parameters.json -c
Note that we use the flag -c to preview the deployment before executing it.
The figure below shows the preview of the deployment.
Once the Bicep file is valid, we will proceed to execute the deployment. The image below shows the output from this deployment.
You can go to the Azure Portal and see the resources deployed as shown below:
From this point, you can start working on your own workspace and add compute instances and compute clusters as needed for your model.
Here’s the complete Bicep code to deploy an Azure Machine Learning Workspace:
@description('Specifies the name of the deployment.')
param name string@description('Specifies the name of the environment.')
param environment string@description('Specifies the location of the Azure Machine Learning workspace and dependent resources.')
param location string = resourceGroup().location@description('Specifies whether to reduce telemetry collection and enable additional encryption.')
param hbi_workspace bool = falsevar tenantId = subscription().tenantId
var storageAccountName_var = 'st${name}${environment}'
var keyVaultName_var = 'kv-${name}-${environment}'
var applicationInsightsName_var = 'appi-${name}-${environment}'
var containerRegistryName_var = 'cr${name}${environment}'
var workspaceName_var = 'mlw${name}${environment}'
var storageAccount = storageAccountName.id
var keyVault = keyVaultName.id
var applicationInsights = applicationInsightsName.id
var containerRegistry = containerRegistryName.idresource storageAccountName 'Microsoft.Storage/storageAccounts@2021-01-01' = {
name: storageAccountName_var
location: location
sku: {
name: 'Standard_RAGRS'
}
kind: 'StorageV2'
properties: {
encryption: {
services: {
blob: {
enabled: true
}
file: {
enabled: true
}
}
keySource: 'Microsoft.Storage'
}
supportsHttpsTrafficOnly: true
}
}resource keyVaultName 'Microsoft.KeyVault/vaults@2021-04-01-preview' = {
name: keyVaultName_var
location: location
properties: {
tenantId: tenantId
sku: {
name: 'standard'
family: 'A'
}
accessPolicies: []
enableSoftDelete: true
}
}resource applicationInsightsName 'Microsoft.Insights/components@2020-02-02' = {
name: applicationInsightsName_var
location: (((location == 'eastus2') || (location == 'westcentralus')) ? 'southcentralus' : location)
kind: 'web'
properties: {
Application_Type: 'web'
}
}resource containerRegistryName 'Microsoft.ContainerRegistry/registries@2019-12-01-preview' = {
sku: {
name: 'Standard'
}
name: containerRegistryName_var
location: location
properties: {
adminUserEnabled: true
}
}resource workspaceName 'Microsoft.MachineLearningServices/workspaces@2021-07-01' = {
identity: {
type: 'SystemAssigned'
}
name: workspaceName_var
location: location
properties: {
friendlyName: workspaceName_var
storageAccount: storageAccount
keyVault: keyVault
applicationInsights: applicationInsights
containerRegistry: containerRegistry
hbiWorkspace: hbi_workspace
}
dependsOn: [
storageAccountName
keyVaultName
applicationInsightsName
containerRegistryName
]
}
I recommend you the following resources as the next steps:
Join the AzInsider email list here.
-Dave R.