AzureFeeds

It’s nice to meet you virtually! We are a team consisting of Tamara Gaidar, a data scientist, and Fady Copty, a researcher. We both work at Microsoft Security. In the upcoming blog posts, we will share insights our team has gained from building AI agents that can be leveraged for any agentic-ai application. We hope you enjoy reading and that it adds value to your day-to-day job. Cheers!

TL;DR

In this blog we provide a practical guide for transitioning agentic AI projects from research to production. We will be writing it in several parts.

In part one we will explore how to effectively design agentic AI systems – AI agents that can reason over and generate unstructured data – by emphasizing the importance of clearly defining the problem and understanding why AI is needed. We will outline a structured approach that includes engineering agent workflows, managing data through chunking and cleanup, and ensuring high-quality input/output handling.

Introduction

Agentic AI presents a significant opportunity to reason on and generate unstructured data. While the scope of this opportunity is evident, identifying and defining the exact problems to be solved with AI can be challenging.

Imagine you’re tasked with analyzing thousands of pages of financial reports, legal contracts, or customer feedback. Traditional automation tools fall short – too rigid, too brittle. Agentic AI: a paradigm where autonomous agents, powered by large language models (LLMs), collaborate to reason over unstructured data and generate meaningful outcomes.

But building these systems is hard. Where do you start? How do you ensure they’re reliable, scalable, and production-ready?

This blog walks you through a practical, end-to-end approach to designing and deploying agentic AI systems – from defining the problem to choosing the right models and tools.

What Is Agentic AI?

Agentic AI refers to systems composed of multiple AI agents that can:

• Autonomously perform tasks,


• Collaborate through defined workflows,


• Reason over complex, unstructured data.

Unlike traditional AI pipelines, agentic systems are modular, dynamic, and inspired by how humans solve problems – breaking them into subtasks, applying domain knowledge, and iterating based on feedback.

We view agentic AI as a combination of engineering and data science techniques that allow solving complex problems with high accuracy. In this document, we present methods to help you solve various problems that arise from research to production.

Start with the Problem: Why Do You Need AI Here?

Before jumping into tools or models, ask: Why do I need AI here?

• Is the data too unstructured for rule-based systems?


• Would solving this manually require 10x more resources?


• Is reasoning or summarization involved?

Use these questions to align with your product team and define both functional and non-functional requirements. If you’re in an incubation phase without a product team, apply de-risking methodologies to identify challenges early and iterate quickly.

Designing the Agentic Workflow

Agentic workflows involve two key steps:

1. Engineering the input/output (e.g., chunking the report into manageable sections).


2. Defining agent dependencies (e.g., one agent handles tables, another summarizes text).

For example, we want to build an agentic AI system that reads a financial annual report and produces a 2-page executive summary:

• One agent splits the report into chapters.


• Another extracts financial tables and sends them to a specialized analysis agent.


• A summarization agent compiles the outputs into a coherent summary.

Data Engineering for Agentic AI

One significant distinction between generative AI and traditional AI is the utilization of pre-trained models. These models offer a robust foundation for addressing the problems at hand. However, it is crucial to recognize the following:

• We must still engineer the data used as input for the models and the output generated by the models


• Data collection for evaluation purposes remains essential 


• Data collection for fine-tuning purposes may be necessary


• Agentic-AI introduces new data requirements for intermediate agentic steps 

Below, we will explore several data-engineering methods specific to agentic-AI that we have observed

Input Chunking

Chunking is more than splitting text – it’s about encoding subject-matter expertise (SME) into the workflow. For code analysis, this might mean chunking based paths on a call graphs. For financial reports, it could mean separating narrative sections from tables.

This approach creates dependencies between agents – like one identifying chunks and another analyzing them – so it’s crucial to clearly define each agent’s expected inputs and outputs.

Input cleanup

Agents rely on assumptions about their inputs, hence we must ensure that the input data meets those assumptions.

Be cautious of pitfalls in your data that can introduce unwanted bias. For instance, an agent analyzing code could pick up a comment in the code that describes desired behavior that does not exist in the code. Ensure that you clean up your data before calling an LLM.

Sampling and aggregation

If your data is repetitive (e.g., you are reverse engineering code and you see a 1,000 similar repetition of a core function), use sampling and aggregation to reduce cost and improve efficiency.

Output curation and enrichment

Just like inputs, outputs must be curated:

• Ensure strict formatting (e.g., valid JSON).


• Add confidence scores or labels.


• Ask the LLM to explain its reasoning and add evidence from the data

This is especially important when outputs feed into downstream agents.

Designing with SME Patterns – But Know Your Tools

A great starting point for building agentic workflows is to draw inspiration from how subject-matter experts (SMEs) approach a task. However, it’s critical to align those patterns with the actual strengths and limitations of large language models (LLMs). You’ll need to experiment with your data to find the balance – where agents deliver results that are reliable enough to support downstream dependencies.

If an agent becomes a key part of your workflow, make sure its output is trustworthy. This might mean adding a secondary agent to audit its results or including a human-in-the-loop for validation.

Also, don’t overlook what LLMs can do that humans can’t. They can simultaneously act as a writer, reviewer, and target audience. Leverage these unique capabilities to enhance your workflow and unlock new possibilities.

⚠️ Caution: Don’t blindly mimic SME workflows. Use them as inspiration, not gospel.

Summary

We have described a structured approach to designing an agentic AI workflow.  

Stay tuned for the next post 😊