AzureFeeds

In our previous blog, we discussed how to debug live issues in the Azure WebJobs Storage Extensions SDK. This approach is particularly effective when issues can be consistently reproduced. However, for intermittent problems, live debugging may not be the most efficient solution. In such cases, integrating custom logging can provide deeper insights and facilitate troubleshooting.

In this blog post, we will provide a step-by-step guide on how to implement custom logging within the Azure WebJobs Storage Extensions SDK. This will help you capture valuable information and better understand the behavior of your applications.

If you encounter any issues while using the Azure WebJobs Extensions SDK, the best way to report them is via GitHub Issues. You can report bugs, request features, or ask questions there. When reporting a bug, it’s most efficient to use the default bug template and provide detailed steps to reproduce the issue. Ideally, a minimum reproducible project is preferable. We also recommend creating a pull request (PR) to contribute to the community. Therefore, knowing how to live debug the WebJobs Extensions SDK or adding custom logging is useful.

Before we dive into adding custom logging in the Azure WebJobs Storage Extensions SDK, it’s important to configure the logging level in the host.json. This configuration allows storage extension logs to flow seamlessly to Application Insights, enabling effective monitoring and troubleshooting.

A typical host.json configuration might look like this:

{
    “version”: “2.0”,
    “logging”: {
      “applicationInsights”: {
        “samplingSettings”: {
          “isEnabled”: true,
          “excludedTypes”: “Request”
        },
        “enableLiveMetricsFilters”: true
      },
      “logLevel”: {
        “Microsoft”: “Debug”,
        “Function”: “Information”
      }
    }
}

To capture detailed logs for the category that starts with Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener, you can set the logging level specifically for Microsoft components. This can be done by adding “Microsoft”: “Debug” in the logging.logLevel section of the host.json file. Also, it’s wise to consider shrinking the scope to only the necessary components to avoid overwhelming your logs with excessive information.  More logging level configuration is located at:  public doc

Then in the application insights, you’ll be able to see the storage extension logs like:  

6/3/2025, 7:32:28.633 AM Poll for blobs newer than ‘0001-01-01T00:00:00.000’ in container ‘mycontainer’ with ClientRequestId ‘9f5fb8df-1e44-41f8-a884-xxxxxxxxx’ found 5 blobs in 1103 ms. ContinuationToken: False.

This logging statements maps to the line of source code at:  here

Next, let’s explore how to add custom logging in the storage extension SDK. For other extensions like service bus or event grid or others, the methodology is the same.

Suppose we are troubleshooting a very intermittent blob trigger not firing issue, based on the storage extension SDK logs in application insights, we noticed the ContainerScan polling is expected to return 11 blobs but only 1 was found. This could be due to 2 reasons: either the listBlob storage API is returning incorrect results, or the LastModificationTime of the blobs wasn’t updated correctly, causing some blobs to be missed.

Our goal is to implement custom logging that prints out all the blobs and their LastModificationTime returned by the listBlob API. This will help us determine whether the issue lies with the ListBlob API, the blobs themselves, or something else.

Step 1:  Download the source code of the azure webjob storage extension SDK from github releases, or else you can use the equivalent git clone or fork the repository. At the time when this blog is being written, the storage extension SDK is at 5.3.4 version: Release Microsoft.Azure.WebJobs.Extensions.Storage_5.3.4 · Azure/azure-sdk-for-net · GitHub

Step 2:  In VS studio, build the sdkstorageMicrosoft.Azure.WebJobs.Extensions.Storage.BlobssrcMicrosoft.Azure.WebJobs.Extensions.Storage.Blobs.csproj, ensure there is No build error.

Step 3:  Now we can add custom logging in the storage extension SDK. For example:

In ScanBlobScanLogHybridPollingStrategy.cs:

In ScanBlobScanLogHybridPollingStrategy.Logger.cs

Step 4:  In VS studio, rebuild the sdkstorageMicrosoft.Azure.WebJobs.Extensions.Storage.BlobssrcMicrosoft.Azure.WebJobs.Extensions.Storage.Blobs.csproj, ensure there is No build error.

Step 5:  Copy the artifactsbinMicrosoft.Azure.WebJobs.Extensions.Storage.BlobsDebugnetstandard2.0Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.dll that we built in the step 4 to the remote function app via Kudu console at C:homesitewwwroot.azurefunctions to override the release version (No debugging symbols) of the dll. 

Note: you’ll need to use web deploy to deploy function app to Azure (WEBSITE_RUN_FROM_PACKAGE = 0) because we’ll have to replace the storage extension dll compiled with debugging symbols on the remote function app.

Step 6:  Restart the function app and you’ll see the custom logging in the Application Insights.

Happy debugging with custom logging!