AzureFeeds

In the era of big data, managing storage efficiently is vital for maintaining the performance, security, and cost-effectiveness of cloud resources. Azure Storage Insights offers a robust solution for monitoring and managing storage accounts, providing detailed metrics, logs, and diagnostic information to enhance visibility into storage operations.

With Storage Insights, you can monitor key performance indicators (KPIs) such as latency, throughput, capacity utilization, transactions and what not! It also aids in swiftly identifying and resolving issues by delivering actionable insights and alerts. By leveraging these capabilities, organizations can optimize their storage performance, enhance security, and reduce operational costs.

Whether overseeing a single storage account or multiple accounts across various regions, Storage Insights equips you with the tools necessary to ensure optimal storage health and performance. This introduction will explore the essential features and advantages of using Storage Insights for monitoring Azure storage accounts.

🔍What is Azure Storage Insights?

Storage Insights is a solution in Azure Monitor that provides pre-configured dashboards and metrics visualization for your storage accounts. Whether you select the Storage feature directly from a storage account or from Azure Monitor, Storage insights present a consistent experience. It aggregates telemetry from blob containers, file shares, queues and tables.

It enables you to:

• Real-time monitoring


• Track usage trends


• Monitor performance


• Set up alerts and get notified


• Analyse access patterns


• Health analysis and optimization


• Enhanced security and compliance


• Inventory details and many more features…

It’s interesting features!

• Let’s one change metrics as needed, modify or set thresholds that align with your limits, and save as workbook.


• Charts in the workbook can be pinned to Azure dashboard or exported as per the requirement.


• It is easy to setup as it does not require you to enable or configure anything, the storage metrics from storage accounts are collected by default.


• Style the table’s appearance by adjusting its margin, width, or padding. Include a search bar for quick resource lookup within the table, set a limit on the number of rows displayed, customize columns and conditionally display the table.

Practical Applications

1. Monitoring Storage Account Usage:  A company wants to track the usage of their storage accounts to ensure they are optimizing their resources.


2. Cost Management: An organization needs to manage costs associated with multiple storage accounts.


3. Security Auditing: A security team tasked to ensure compliance and identify any security issues in storage accounts.


4. Performance Monitoring: A development team is experiencing performance issues with their applications relying on Azure Storage.


5. Backup and Disaster Recovery: A company needs to ensure their data is backed up and recovery processes are in place.


6. Compliance and Regulatory Reporting: An organization must comply with industry regulations that require reporting on data access and retention.


7. Data Lifecycle Management: A company wants to implement policies for managing data throughout its lifecycle.


8. Inventory of Storage Accounts: A large organization has multiple storage accounts across various subscriptions.

Pricing

Azure Storage Insights itself does not have a separate pricing tier—it leverages Azure Monitor, so the costs are based on the Azure Monitor’s essential features configured or enabled.

👩‍💻Customize Storage Insight Workbook

Before we begin with details on how we can customise workbook, you must have the appropriate permissions to view or edit a workbook.

Prerequisites You Can’t Ignore!

Workbook permissions are based on the permissions the user has for the resources included in the workbooks.

Standard Azure roles that provide access to workbooks:

• Monitoring Reader includes standard /read privileges that would be used by monitoring tools (including workbooks) to read data from resources.


• Monitoring Contributor includes general /write privileges used by various monitoring tools for saving items (including workbooks/write privilege to save shared workbooks). Workbooks Contributor adds workbooks/write privileges to an object to save shared workbooks.

For custom roles, you must add microsoft.insights/workbooks/write to the user’s permissions to edit and save a workbook.

How to view Storage Insights?

There are two ways to navigate to Storage Insights page, as shown below. Please follow link for more detailed information.

1. View from Azure Storage Account
   Access it from the Storage Account -> Insights

It shows Overview, Failures, Performance, Availability, and Capacity to open the respective workbook; each delivering a set of interactive storage-related information tailored to that category.

• Overview workbook – displays the most essential details related to storage capacity, availability, transactions and latency.


• Failure workbook – displays client-side errors, successful requests, and performance charts for transaction response types. Not only that but one can setup alerts as well as check log errors for investigation.


• Performance workbook – includes metrics for end-to-end latency and server latency, with performance charts and tables breaking down latency by API.


• Availability workbook – shows the current health state of Azure Storage service and availability trends for each data service in the account.


• Capacity workbook – shows the total amount of storage used for each data object in the account and the number of data objects stored.

      2. View from Azure Monitor

Access it from the Monitor -> Storage accounts

It shows Overview and Capacity sections under which you can view transaction, latency, and capacity details from multiple storage accounts in your subscription, and help identify performance, capacity problems, and failures.

• Overview workbook – displays interactive storage metrics like transactions, latency and errors for selected storage accounts within selected subscription from drop-down list on top.


• Capacity workbook – shows the amount of total storage used in the account and capacity used by each data service in the account to help identify over and under-utilized storage.

How to edit Workbook?

To edit an Azure Workbook, follow these steps using the Azure Portal:

1. Navigate to the Storage Insights view from Azure Monitor(suggested)


2. Enter edit mode by clicking on the Edit button at the top. You’ll now see options to modify or add sections.In this image you can see edit button, three dots displaying add, move, clone and remove options, save and done editing buttons, pencil icon to go in edit mode, code icon to see template and download icon to export view. You also get option to pin and refresh from three dots next to download icon.


3. Modify existing sections by –
   
   
   
   • Click the ✏️ on any section (text, query, chart, etc.)
   
   
   • Update the metric or query table, visualization type, or parameters
   
   
   • Click Apply to preview changes.
   
   
   
   


4. Add new sections by clicking Add to insert different type of tables like metric table, query table, parameters table, text table or links/tabs table. You get option to group tables together if resulting into final table which displays data. 


5. Customize layout and formatting as needed.




• Editing Metric table gives options to add metrics, add filter, data visualization, size of table, format and time range for data population.

• 
  
  
  
  • Editing Query table lets you choose data source for query, visualization.
  
  
  
  

• 
  
  
  
  • Editing Parameter with options like name, type, query.
  
  
  
  

• 
  
  
  
  • Use Column setting to create rich, interactive and easily interpretable reports using key features column renderers, link actions (configure actions for cells that act as links, such as navigating to another view or opening a URL), threshold (assign icons or open different workbooks based on cell values), custom formatting (number and data formatting).
  
  
  
  

• 
  
  
  
  • Setting and styling table from Advance Settings and Style page. 
  
  
  
  

Advance Setting helps to provide addition features like pin and export table, add search bar, limit to number of rows table and show table conditionally.Style helps in defining the look of the table by changing its margin, width or padding.

• 
  
  
  
  • Use Markdown for Text. You can include headings, lists, links, and even dynamic parameter values.
  
  
  
  

6. Preserve your changes by Save or Save As button from top bar.

7. It has autosave feature too. If you accidentally close the browser, you can recover unsaved changes from the autosave as it saves your edits locally in your browser every few minutes.

8. Use the Preview tab to see how your changes look.

9. Once done, click Done Editing.

✍️Curated Customised Workbook

We have enhanced the capability of Storage insights by updating the default workbook to give more visual information related to detailed subscription/resource group information of storage account, blob information on count, capacity and backup, fileShare capacity utilisation at storage level and cost estimation to help monitor and optimize our storage resources effectively.

Visuals are generally in the form of grid view and to fetch data metrics, and graph queries are used with merge operations on tables. Provided workbook template can be integrated in any environment to give a detailed overview of Azure Storage Accounts.

What it shows?

• It lets you filter the view by selecting subscription, storage account and time range.


• Number of storage account for which information is presented shown on top in titles.


• Workbook is curated to show 10 sections, as mentioned below:
  
  
  
  • Resource Group Section: Here one can view the structural hierarchy of storage account i.e., to which resource group and subscription it belongs, count of storage account under each resource group.
  
  
  • Overview Section: Its gives grid-view information on storage transactions with timeline, E2E and server latency, account availability and errors. One can easily hover and click on specific storage account from table to navigate to that resource.
  
  
  • Capacity Section: This section shows the total amount of storage used for each storage data object in the account.
  
  
  • Redundancy Zone: Here you get to see data on to which zone, tier and account kind storage service belongs.
  
  
  • Count By Type: This section shows how many data objects are stored in the account in a grid view and count sum up at subscription level.
  
  
  • Blob Count: This tab shows blob count information as per the chosen way – either by Type or by Tier.
    
    
    
    • If Type is selected it shows blob count split into ADLS, page blob and block blob.
    
    
    • If Tier is selected count of blob as per Hot, Cool, Archive and Cold tier will be shown.
    
    
    
    
  
  
  • Blob Capacity: Similar to Blob Count, this section also gives 2 data views – Type and Tier for same variables.
  
  
  • Blob Backup: This gives grid view information backup vault, policy, policy rule and associated storage account for type microsoft.dataprotection/backupvaults/backupinstances.
  
  
  • FileShare Info: This section provides capacity utilisation information of fileShares at storage level with their count and capacity.
  
  
  • Cost Estimation: Here defender for storage cost estimation is shown.
  
  
  
  

Go through repo to understand how to use it. You can enhance its functionality as your requirement by going into edit mode once template is deployed locally via Azure portal.

⏳Limitations of Azure Storage Insights

• Scope and Scale:




•  Limited to Storage Accounts: Storage Insights primarily focuses on Azure Storage accounts, therefore it’s not suitable for other types of storage solutions or third-party storage services. Also, the number of selected storage accounts has a limit of 200.


• Data Volume: When dealing with numerous storage accounts, the analysis might take longer to complete, and there could be performance impacts.




• Limited Customization Options: It may not be as flexible as other specialized monitoring tools. Users might find it challenging to tailor the insights to specific needs beyond the provided metrics.


• Latency in Data Collection: There can be a delay in data collection and reporting, which means it might not be suitable for real-time monitoring needs.


• Complexity: For users unfamiliar with Azure Monitor, storage metrics or workbooks, there can be a steep learning curve to effectively utilize Storage Insights.

 

To conclude, Azure Storage Insights transforms raw telemetry into actionable intelligence. By leveraging its dashboards, metrics, and alerting capabilities, you can ensure your storage infrastructure is resilient, performant, and cost-efficient.

Hope this helps! 😊