AzureFeeds

Azure Monitor logs are trusted by hundreds of thousands of organizations to monitor mission-critical workloads. But with such a diverse customer base, there’s no one-size-fits-all solution. That’s why we’re excited to announce a series of major advancements to Auxiliary Logs, the Azure Monitor plan that is designed for high volume logs, making it more cost-effective and improving the tools for use.

Auxiliary Logs works in tandem with all other Azure Monitor tools including the more powerful Basic and Analytics Logs plans. Together, they are the one-stop-shop for all the logging needs of an organization.

Auxiliary Logs is also one of the underlying services behind Microsoft Sentinel and the newly announced Sentinel data lake enabling customers to use the same data for both observability and security. This unified approach reduces both direct costs and total cost of ownership by eliminating data duplication and fragmentation in skillset.

General Availability of Auxiliary Logs and reduced pricing

Auxiliary Logs were introduced last year and have gained a lot of traction since. There are many customers that ingest data into Auxiliary Logs, with several teams ingest more than a Petabyte of logs per day.

Over the last few months, we have moved Auxiliary Logs to General Availability status, made them available in all regions, and made numerous enhancements to the service. We’re also announcing a significant price reduction for Auxiliary Logs, making them even more accessible for high-volume scenarios. For detailed pricing information and charges, visit the Azure Monitor pricing page.

Auxiliary Logs were first introduced with support for Custom Logs only, security data was added following and is now also supported. Additional table support will be available soon.

Learn more about table plans here.

Enhanced Query Capabilities

We have worked to make queries on Auxiliary Logs faster and more powerful, including:

1. Expanded KQL Support: All KQL operators on a single table are now supported, including the lookup operator to Analytics tables.


2. Performance Boosts: Built on Delta Parquet, Auxiliary Logs now benefit from improved encoding and partitioning to make queries much more efficient, though indexed technologies like Basic Logs and Analytics Logs performs better.


3. Extended Time Range: Queries are no longer limited to the last 30 days – you can now query across any time period. 


4. Cost Estimation Preview: Get a cost estimate before running your query.

General Availability of Summary Rules

We are also announcing the General Availability of Summary rules. Summary rules have quickly become a key resource for optimizing data ingestion and analysis, having been adopted by a significant number of customers during the preview period. Summary rules enable users to efficiently summarize high-ingestion-rate streams across Analytics, Basic, or Auxiliary plans, supporting robust analysis, dashboarding, and long-term reporting via summarized Analytics tables. Unlike conventional ETL processes, raw data remains in its original tables, allowing for detailed investigations as needed.

Key enhancements include:

• Increased rule count limit per workspace


• Enabling users to retry bins affected by incidents (coming soon)


• Expanded regional availability

Customers can now utilize summary rules on a greater scale with increased confidence.

Learn more about summary rules here.

Search Jobs: More Power, More Flexibility

Search jobs allow users to scan vast amount of data asynchronously and ingest the results into Analytics table for further investigation. Based on customer feedback, we’ve made the following improvements:

1. Enabling more results to be loaded, up to 100 million records.


2. Improved user interface that streamlines the search job execution.


3. Provide a cost prediction before running a search job.


4. Increase concurrently and remove additional limits.


5. Add support for all KQL operators on a single table with the lookup operator to Analytics tables (coming soon).

Learn more about search jobs here.

Public Preview of KQL Transformations for Auxiliary Logs

Last, but not least, we’re excited to announce the public preview of KQL-based transformations for Auxiliary Logs. This milestone brings Auxiliary Logs to feature parity in terms of ingestion-time transformations with other Azure Monitor log tiers, eliminating the previous limitation of ingesting only raw custom logs into Auxiliary storage.

With this new capability, you can now apply filtering and transformation logic at ingestion time, enabling a more strategic and cost-effective approach to managing high-volume, low-fidelity logs. By using Data Collection Rules (DCRs) with Kusto Query Language (KQL) expressions, you can:

• Filter out noise to reduce data volume.


• Parse and shape fields to prepare logs for efficient downstream consumption.


• Split data across multiple tables or tiers, for cost-performance optimization.

What makes this especially powerful is that transformations apply to both custom and standard log streams directed to custom tables in the Auxiliary tier. For example, you can now route a portion, or the entirety, of specific platform logs to a custom table in Auxiliary storage, applying transformations as needed.

Applying custom transformations and filtering to data ingested into the Auxiliary tier will incur a log processing charge . For detailed pricing information and charges, please refer to the Azure Monitor pricing page. Processing charges will apply starting August 1^st.

Learn more about ingestion-time transformations here.

What’s next?

Stay tuned for more updates as we continue to evolve Azure Monitor to meet your growing observability needs.

We look forward to hearing your feedback and seeing how you take advantage of these new capabilities.