AzureFeeds

In Azure Batch, to monitor the node performance like CPU or Disk usage users are required to use Azure monitor.  The Azure Monitor service collects and aggregates metrics and logs from every component of the node. Azure Monitor provides you with a view of availability, performance, and resilience. When you create an Azure Batch pool, you can install any of the following monitoring-related extensions on the compute nodes to collect and analyse data.

Previously users have leveraged Batch Insights to get system statistics for Azure Batch account nodes, but it is deprecated now and no longer supported.

The Log Analytics agent virtual machine (VM) extension installs the Log Analytics agent on Azure VMs and enrols VMs into an existing Log Analytics workspace. The Log Analytics agent is on a deprecation path and won’t be supported after August 31, 2024. Migrate to Azure Monitor Agent from Log Analytics agent – Azure Monitor | Microsoft Learn

Azure Monitor Agent (AMA) now replaces the Log Analytics agent.

Install and Manage the Azure Monitor Agent – Azure Monitor | Microsoft Learn

Important!

Currently Azure Monitor in Batch Pool is only supported for Batch accounts that are created with Pool allocation Mode in User subscription mode only. Batch accounts that are created with Pool allocation Mode in Batch Service are not supported. As in Batch Service mode, nodes will be created in Azure subscriptions and users do not have access to these subscriptions, enabling data collection for these nodes is not possible. 

This article will focus on how you can use to install and configure the Azure Monitor Agent extension on Azure Batch pool nodes.

Note : Extensions cannot be added to an existing pool. Pools must be recreated to add, remove, or update extensions. Currently the Batch pool with user assigned Managed Identity and extension is only supported by ARM template and REST API call. Creating a pool with extension is unsupported in Azure Portal. Creating a pool with user assigned Managed Identity is unsupported in Az PowerShell module and Azure CLI.

To use the templates below, you’ll need to follow below prerequisites:

• To create a user-assigned managed identity. A managed identity is required for Azure Monitor agent to collect and publish data.


• To configure data collection for Azure Monitor Agent, you must also configure or deploy Resource Manager template data collection rules and associations.

Step 1: Create a pool with AMA extension

Below is sample JSON template to create a pool with AMA extension enabled for Windows server.

{
  “name”: “poolextmon”,
  “type”: “Microsoft.Batch/batchAccounts/pools”,
  “properties”: {
    “allocationState”: “Steady”,
    “vmSize”: “STANDARD_D2S_V3”,
    “interNodeCommunication”: “Disabled”,
    “taskSlotsPerNode”: 1,
    “taskSchedulingPolicy”: {
      “nodeFillType”: “Pack”
    },
    “deploymentConfiguration”: {
      “virtualMachineConfiguration”: {
        “imageReference”: {
          “publisher”: “microsoftwindowsserver”,
          “offer”: “windowsserver”,
          “sku”: “2019-datacenter”,
          “version”: “latest”
        },
        “nodeAgentSkuId”: “batch.node.windows amd64”,
        “extensions”: [
          {
            “name”: “AzureMonitorAgent”,
            “publisher”: “Microsoft.Azure.Monitor”,
            “type”: “AzureMonitorWindowsAgent”,
            “typeHandlerVersion”: “1.0”,
            “autoUpgradeMinorVersion”: true,
            “enableAutomaticUpgrade”: true,
            “settings”: {
              “authentication”: {
                “managedIdentity”: {
                  “identifier-name”: “mi_res_id”,
                  “identifier-value”: “/subscriptions/xxxxx/resourceGroups/r-xxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/usmi”
                }
              }
            }
          }
        ]
      }
    },
    “scaleSettings”: {
      “fixedScale”: {
        “targetDedicatedNodes”: 1,
        “targetLowPriorityNodes”: 0,
        “resizeTimeout”: “PT15M”
      }
    },
    “currentDedicatedNodes”: 1,
    “currentLowPriorityNodes”: 0,
    “targetNodeCommunicationMode”: “Default”,
    “currentNodeCommunicationMode”: “Simplified”
  }
}

Below is sample JSON template to create a pool with AMA extension enabled for Linux server.

{
  “name”: “poolextmon”,
  “type”: “Microsoft.Batch/batchAccounts/pools”,
  “properties”: {
    “allocationState”: “Steady”,
    “vmSize”: “STANDARD_D2S_V3”,
    “interNodeCommunication”: “Disabled”,
    “taskSlotsPerNode”: 1,
    “taskSchedulingPolicy”: {
      “nodeFillType”: “Pack”
    },
    “deploymentConfiguration”: {
      “virtualMachineConfiguration”: {
        “imageReference”: {
          “publisher”: “canonical”,
          “offer”: “0001-com-ubuntu-server-jammy”,
          “sku”: “22_04-lts”,
          “version”: “latest”
        },
        “nodeAgentSkuId”: “batch.node.ubuntu 22.04”,
        “extensions”: [
          {
            “name”: “AzureMonitorAgent”,
            “publisher”: “Microsoft.Azure.Monitor”,
            “type”: “AzureMonitorLinuxAgent”,
            “typeHandlerVersion”: “1.0”,
            “autoUpgradeMinorVersion”: true,
            “enableAutomaticUpgrade”: true,
            “settings”: {
              “authentication”: {
                “managedIdentity”: {
                  “identifier-name”: “mi_res_id”,
                  “identifier-value”: “/subscriptions/xxxxxx/resourceGroups/r-xxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/usmi”
                }
              }
            }
          }
        ]
      }
    },
    “scaleSettings”: {
      “fixedScale”: {
        “targetDedicatedNodes”: 1,
        “targetLowPriorityNodes”: 0,
        “resizeTimeout”: “PT15M”
      }
    },
    “currentDedicatedNodes”: 1,
    “currentLowPriorityNodes”: 0,
    “targetNodeCommunicationMode”: “Default”,
    “currentNodeCommunicationMode”: “Simplified”
  }
}

Once the pool is created you can verify if the extension is installed on pool from portal.

Step 2: Create Log analytics workspace

You are required to have a log analytics workspace where the data will be sent.

Step 3: Create a data collection rule (DCR)

Create a DCR to collect data by using the Azure portal by following below document. You can refer to below document on how to create a DCR. Below document also talks about the types of data you can collect from a VM client with Azure Monitor and where you can send that data. 

Collect data from virtual machine client with Azure Monitor – Azure Monitor | Microsoft Learn

Once the VM is associated to DCR, you can check the computers connected in Log Analytics workspace.

To verify that the agent is operational and communicating properly with Azure Monitor check the Heartbeat for the VM.

To verify that data is being collected in the Log Analytics workspace, check for records in the Perf table.

To verify that data is being collected in Azure Monitor Metrics, select Metrics from the virtual machine in the Azure portal. Navigate to the VMSS from portal and select Virtual Machine Guest (Windows) or azure.vm.linux.guestmetrics for the namespace and then select a metric to add to the view.