AzureFeeds

Purpose of the document provide overview of AI Foundry and how it can be set up and organised for at scale for an enterprise.  This document should be considered as guidance and recommendations, but individual organisations should treat and consider other factors such as security, policy, governance and number of business units.

AI Foundry Resource:

Azure AI Foundry is Microsoft’s unified platform for building, customising, and managing AI applications and agents—designed to accelerate innovation and operationalise AI at scale.

It brings together:

• Data, models, and operations into a single, integrated environment.


• A developer-first experience with native support for GitHub, Visual Studio, and Copilot Studio.


• A low-code portal and code-first SDKs/APIs for flexibility across skill levels.

Key capabilities include:

• Model Catalogue: Access and customise top-tier LLM models (e.g. OpenAI, Hugging Face, Meta, Phi, Mistral, etc)


• Agent development: Build multi-agent systems using prebuilt templates and orchestration tools.


• Enterprise-grade governance: Identity based authentication, Role-based access (RBAC), quota management, and compliance tooling.


• AI Foundry offer centralised management; project workspaces will remain the primary environment for AI developers.

Organization of AI Foundry Resource:

The new version of AI Foundry Resource and its high-level component view:

AI Foundry Resource serves as the foundational building block that defines the scope, configuration, and monitoring of deployments. AI Foundry Projects act like containers (child or sub resource of AI Foundry Resource), helping to organize work and resources within the context of an AI Foundry Resource. AI Foundry Project also provide access to Foundry’s developer APIs and tools.

Organise & Set up AI Foundry:

The following considerations can guide the design and establishment of an AI Foundry within an enterprise:

• Team Structure: Teams such as Data Science, AI Innovation, and Generative AI are structured and collaborate around specific business use cases.

o   AI Foundry Resource per Team: Separate resources are aligned to individual team who works multiple project / products

o   AI Foundry Resource per Product/Project: Separate resources are aligned to individual customer projects or products.

o   Single AI Foundry Resource: A single resource supports multiple teams or projects, depending on scale and maturity.

• Environment Setup: Environments are integral to the development lifecycle of Generative AI use cases, supporting the transition from experimentation to operationalisation through model deployment. Typical environment stages include:

o   Development, Testing, Production

Each environment should include an instance of the AI Foundry resource to effectively support the full lifecycle of Generative AI deployment

Team Structure:

To address manageability and governance needs, organisations typically implement one or a combination of the following AI Foundry setup patterns.

• AI Foundry Resource per Team (Business Unit or Group within org): Each team is provisioned with a single AI Foundry Resource instance. Team members can use this shared resource to work on multiple use cases. Within this setup, each AI Foundry Project represents a distinct use case. These projects act as containers that organize all relevant components—such as agents and files—specific to that application.

While projects inherit baseline security settings from the parent resource, they also support their own access controls, data integrations, and governance configurations, enabling fine-grained management at the project level.

Each team ie Data Science, AI Innovation, and Generative AI is provisioned with a dedicated AI Foundry Resource instance. This instance acts as the primary workspace, allowing teams to create and manage multiple projects within a cohesive environment. Teams are typically organised by business unit or line of business, ensuring alignment with specific organizational goals.

Centralized Governance: The AI Foundry Resource serves as the central control for each team, enabling unified access to data, shared resources, and consistent policy enforcement across all associated projects.

Access Management: Security configurations and access controls defined at the resource level are inherited by all associated projects, ensuring consistency and simplifying team level administration. While these global settings are inherited, individual projects can define their own RBAC rules to address specific security and collaboration needs.

Shared Connections: Connections established at the AI Foundry Resource level—such as links to data sources, tools, Azure OpenAI, or other Foundry resources—are accessible across all projects within the resource. It improves the team members productivity by having easily access, explore, and reuse the connections.

Project Level Isolation: For projects handling sensitive data or subject to regulatory constraints, isolated connections can be configured at the project level to prevent sharing with other projects under the same Foundry Resource instance.

Cost & Consumption Tracking: This approach streamlines cost management at the team level. As experimentation and trial runs can scale rapidly, consolidating activities within a single AI Foundry Resource per team helps maintain clear ownership and keeps operations well organised.

This setup is recommended for enterprise-scale deployments where teams share similar needs—such as consistent data access, comparable experimentation workflows, or common asset usage—offering greater flexibility, seamless collaboration, and strong governance across the organization.

• AI Foundry Resource per Product/Project: Using an AI Foundry Resource at the product level is recommended when there is a need to fully isolate data and assets within a specific customer’s project or product. This setup is tailored for product-centric collaboration and sharing, ensuring that only the relevant product team or group has access. It enables controlled reuse of assets strictly within the boundaries of that product, supporting secure and focused development.

Isolation & Ownership governance: All data, assets, and resources are isolated under a product scope, ensuring exclusive access and controlled reuse within the product by the designated team or group. For example, when multiple teams are involved in developing a single product, instead of provisioning separate AI Foundry Resources for each team, a single AI Foundry Resource can be established. Within this shared resource, individual AI Foundry projects can be created to support the development of sub-products, maintaining isolation while promoting coordinated collaboration.

 Access Management: In addition to product scope data isolation, Role-Based Access Control (RBAC) can be configured at the individual project level, allowing the product team to tightly manage permissions and maintain secure access to assets.

Cost & Consumption Tracking: Budgeting, billing, and usage can be monitored and managed at the product level. Enables transparency and cost optimisation per product or project.

Sharing Limitations: Challenges in sharing assets and connections outside the AI Foundry resource. For instance, fine-tuned models and their associated datasets are often tightly coupled to a specific product context. May require additional governance or integration mechanisms for cross-product collaboration.

This setup is ideal when high levels of isolation, data security, and asset control are required. It supports projects that demand clear ownership, regulatory compliance, and product-specific governance

• Single AI Foundry Resource: This setup is ideal for non-project-specific or non-team-specific experimentation, where resources are shared across users without being tied to a particular initiative. It simplifies management, reduces admin overhead, and is best suited for sandbox environments.

Ownership & Governance: Designed for general-purpose use with no team or project ownership. Enables user to run experiment without needing dedicated resources.

Cost & Resource Efficiency: Costs are not attributed to any specific team or project. Helps minimise Azure resource sprawl and reduce management overhead.

Simplified Management: Operates as a single unit for accessing all data and assets. Reduces the complexity of maintaining multiple isolated environments.

Potential Challenges: Lack of isolation can lead to clutter and resource mismanagement. Difficult to manage access, data governance, and asset lifecycle as usage grows. Consolidating all projects / teams under a single AI Foundry resource can lead to disorder and governance challenges over time.

This set up is recommended for Sandbox environments where flexibility and ease of access are prioritised over control and isolation.

Environment Setup:                        

Following environment deployment approaches are common:

• Single environment deployment: A single AI Foundry Resource is deployed without separating production and non-production data. This model is best suited for sandbox or experimental use cases where data isolation is not a priority and simplicity is preferred.


• Multi environments (e.g., Dev, Test, Prod) are established to segregate data and access controls. This setup supports both inner and outer loops of the GenAIOps lifecycle, enabling smooth promotion of code and assets from development to production. Recommended for enterprise-grade implementations requiring structured governance and lifecycle management.


• Isolated environment deployment: Environments are strictly separated based on data sensitivity. For example, the development environment accesses only non-production data, while the production environment handles live and historical data. This model ensures compliance, data security, and controlled access, making it suitable for regulated industries or sensitive workloads.

                                  Multi Environment Deployment

The proposed multi environment approach aligns with the recommended model of assigning an AI Foundry Resource per team. Each environment contains separate subscriptions for different teams (Team A and Team B), which house individual AI Foundry resources and projects. These resources are connected to additional services for data, monitoring, and AI, ensuring the integration of security and content safety measures. By adopting a GenAIOps approach, any Generative AI or agent-related development can be efficiently promoted across environments—from development through to production—ensuring a smooth and consistent lifecycle.

The shared subscription serves as a centralised platform for deploying AI Foundry assets such as models, domain knowledge, common data sources, and MCP servers that are universally applicable within a specific environment (ex: development). This centralised shared subscription approach ensures that governance, security, and control measures, such as policies prohibiting the use of certain LLM models, are comprehensively managed. Models and policies within the shared subscription can be uniformly applied across various projects.

This setup not only facilitates strict governance and uniform policy across all projects but also enables inter-team collaboration within the same environment. For example, Team A in the development environment can leverage AI Foundry models, common “AI services” within the shared subscription and can connect with Team B’s resources for additional AI functionalities (thro “connected resources”) such as other specific “AI services”. Access to these models for application purposes is mediated through an APIM gateway, which serves as a single-entry point for all LLM models consumption in the given environment. Each environment is recommended to have its own dedicated shared subscription to maintain organised and secure management of AI assets.

Regions:

AI Foundry Resource instances can be deployed across multiple regions based on organizational requirements such as data residency, compliance, and security. Associated resources can also span multiple regions to support workloads while still being centrally managed under a single AI Foundry Resource instance. Furthermore, LLM models can be deployed in various regions to accommodate different data zones, global standard, Batch and PTU. These regional and global deployed models can be accessed via APIs and keys, allowing seamless integration across geographies.

Cost:

Azure AI Foundry Resource integrates multiple Azure services, and its pricing structure varies based on the chosen setup and the number of AI Foundry projects created under a single resource instance. Costs are influenced by architectural decisions, resource usage, and the provisioning of associated components and services.

It’s important to account for all related cost when planning deployments in Azure AI Foundry. To ensure cost efficiency and scalability, it is recommended to perform sizing and cost estimation based on the specific use cases and workloads being considered.

IaC – Templates:

Automate the AI Foundry Resource by using IaC templates

• ARM templates or Bicep templates to automate environment provision and secure deployments


• Terraform templates
  
  

Conclusion

In summary, Microsoft’s Azure AI Foundry offers a comprehensive and unified platform for organisations aiming to operationalise GenAI at scale. By providing a flexible structure that accommodates various team, product, and project requirements, AI Foundry empowers enterprises to tailor their setup according to business needs, security considerations, and governance standards.  Selecting the right organisational model—whether by team, product, or through a single resource—enables alignment with business objectives, cost management, and collaboration. The recommended practices for environment setup, cost estimation, and automation with Infrastructure as Code streamline adoption and ongoing management. Ultimately, by following these guidelines and considering the unique context of each enterprise, organisations can maximise the value of AI Foundry, accelerating innovation whilst maintaining robust security, compliance, and operational efficiency.