AzureFeeds

Introduction

In Kenya’s evolving higher education landscape, access to accurate information about funding opportunities remains a critical challenge for students and their families. When the New Funding Model (NFM) was introduced in May 2023, many applicants found themselves navigating unfamiliar processes with limited guidance. This knowledge gap inspired our team to develop a solution that would bridge this divide using cutting-edge AI technology.

Our project, RAG-Powered Virtual Assistant for the Higher Education Fund (HEF), emerged as the overall winner at the Microsoft Data + AI Hack Kenya 2025. By leveraging Microsoft Fabric’s powerful Eventhouse capabilities alongside Azure OpenAI services, we created an intelligent assistant that provides instant, accurate responses to queries about Kenya’s Higher Education Fund—democratizing access to critical information and empowering students across the country.

Meet the Team

Our cross-disciplinary team brought together technical expertise and firsthand experience with Kenya’s education funding systems:

• Ephraim Mwereza (LinkedIn) – Technical Lead
  As the technical architect behind the project, Ephraim led the development of the RAG system, integrating Microsoft Fabric Eventhouse with Azure OpenAI services to create a scalable and efficient solution.


• Amaziah Marvel – User Research/Outreach Lead
  As a nursing student at Kenya Medical Training College (KMTC) and a prospect for the New Funding Model, Amaziah provided fresh perspectives on the challenges faced by first-time applicants, particularly those from remote regions seeking funding information. His contributions to user research ensured our solution addressed real needs.


• Emmanuel Eddy Mwereza – Product/Student Experience Lead
  Currently pursuing a degree in Procurement at Jomo Kenyatta University of Agriculture and Technology (JKUAT) under the New Funding Model (NFM), Emmanuel brought invaluable perspective having previously completed a diploma under the Differentiated Unit Costs (DUC) model. His firsthand experience with both funding systems provided crucial insights into user needs and pain points.

Project Overview

The Higher Education Fund Virtual Assistant addresses a pressing need in Kenya’s education sector. Following the introduction of the New Funding Model in 2023, many students and guardians struggle to understand the complex processes for applying for and receiving higher education funding.

Our solution provides a 24/7 conversational AI assistant that answers questions about the Higher Education Fund using official information from trusted sources like the Kenya Universities and Colleges Central Placement Service (KUCCPS), Higher Education Loans Board (HELB), and the Universities Fund. By implementing a Retrieval-Augmented Generation (RAG) architecture powered by Microsoft Fabric’s Eventhouse as a vector database, we created a robust system that delivers accurate, contextually relevant information instantly.

The project aims to:

• Make higher education funding information accessible to all Kenyans, regardless of location


• Reduce the workload on administrative staff at educational institutions


• Provide consistent, accurate information based on official documentation

Project Journey

Identifying the Problem

Our journey began with a deep analysis of the challenges facing prospective higher education students in Kenya. Through conversations with students, parents, and education administrators, we identified several key issues:

1. Information Gap: The transition to the New Funding Model created widespread confusion


2. Geographical Barriers: Many students in remote areas couldn’t easily access information centers located in major cities.


3. Resource Constraints: Support staff at educational institutions were overwhelmed with repetitive queries regarding higher education.

Emmanuel’s unique perspective—having experienced both the old and new funding models—helped us understand the transition challenges, while Amaziah’s insights into the difficulties faced by students in remote areas shaped our accessibility focus.

Choosing the Right Approach

After evaluating several potential solutions, we determined that a RAG-based virtual assistant would best address these challenges. RAG combines the strengths of retrieval-based systems (which find relevant information) with generative AI (which produces natural language responses). This approach ensures answers are both accurate (grounded in official documentation) and conversational (easy for users to understand).

Technical Implementation Challenges

Building the system presented several challenges:

1. 
   

   Hybrid Document Processing: Merging structured data from PDF documents with dynamic content scraped from official websites posed difficulties in maintaining formatting and semantic coherence

   
   


2. Response Quality: Ensuring responses were both accurate and helpful for users with varying levels of technical understanding


3. Performance Optimization: Creating a system that could provide near-instant responses despite complex backend processes

Microsoft Fabric’s Eventhouse emerged as the perfect solution for our vector database needs, offering exceptional performance for similarity search at scale.

Technical Details

Our virtual assistant implements a full RAG pipeline using Microsoft Fabric and Azure OpenAI services:

Architecture Overview

The system operates in two main phases:

1. Document Processing and Indexing:




• PDF documents containing official HEF information are stored in a Microsoft Fabric Lakehouse


• Supplementary data scraped from official websites (e.g., HELB, KUCCPS, Universities Fund) is cleaned and structured


• All text sources are normalized and split into manageable chunks of 1000 characters with a 30-character overlap


• Azure OpenAI’s text-embedding-ada-002 model converts these chunks into vector embeddings


• Both original text and embeddings are stored in Microsoft Fabric Eventhouse




2. Query Processing:




• User questions are converted into embeddings using the same model


• Similarity search in Eventhouse identifies the most relevant document chunks


• Retrieved content is combined with the original question in a prompt to GPT-4o


• The model generates a natural language response that directly addresses the user’s query

Key Technologies

• Microsoft Fabric Lakehouse: For secure, scalable storage of source documents


• Microsoft Fabric Eventhouse: As a high-performance vector database


• Azure OpenAI GPT-4o: For generating natural, contextually appropriate responses


• Azure OpenAI Embeddings (text-embedding-ada-002): For creating vector representations of text


• Kusto Query Language (KQL): For efficient vector similarity searches

All operations were run via Python notebook, with code hosted in our GitHub repository.

Visuals and Media

 1: Architectural view of the application

 2: Processing the files and indexing the embeddings

 3: RAG – Getting answers

Video Demo 

Results and Outcomes

The RAG-Powered Virtual Assistant for the Higher Education Fund has demonstrated significant impact even in its initial deployment:

Quantitative Results

• Response time under 2 seconds for most queries


• Accuracy rate of over 95% when compared to official documentation


• Ability to handle hundreds of concurrent queries, demonstrating excellent scalability

Qualitative Feedback

• Students report finding the system intuitive and helpful for navigating the complex funding landscape


• Education administrators note a reduction in repetitive queries, allowing them to focus on more complex cases


• The solution has been particularly praised for its accessibility benefits to students in remote areas

Our project was selected as the overall winner of the Microsoft Data + AI Hack Kenya 2025, with judges highlighting the practical impact, technical implementation, and innovative use of Microsoft Fabric’s Eventhouse as a vector store.

Lessons Learned

Building this system taught us several valuable lessons:

Technical Insights

• Vector Database Selection Matters: Microsoft Fabric’s Eventhouse proved to be an exceptional choice for our vector database needs, offering superior performance compared to alternatives we evaluated


• Chunk Size Optimization: Finding the right balance for document chunking is crucial—too small and you lose context, too large and similarity matching suffers


• Prompt Engineering: The quality of the final output depends significantly on how well the prompt is structured for the GPT model

Process Insights

• Start with User Needs: Our focus on solving a real problem faced by students drove better design decisions


• Iterative Testing: Regular testing with real queries helped us refine the system continuously


• Documentation Quality: The accuracy of our system was directly related to the quality and comprehensiveness of the source documents

Collaboration and Teamwork

Our team’s diverse backgrounds were instrumental to the project’s success:

• Ephraim led the technical implementation, architecting the RAG system and integrating Microsoft Fabric with Azure OpenAI.


• Emmanuel provided crucial insights into the user experience with his dual experience with both funding models.


• Amaziah led user research efforts, particularly focusing on the needs of students from remote areas.

Our collaboration process involved regular brainstorming sessions, continuous feedback loops, and collaborative testing to ensure the system addressed real-world queries accurately. The combination of technical expertise and firsthand experience with Kenya’s education funding systems made our team particularly effective at addressing this challenge.

Future Development

While our virtual assistant already provides significant value, we have ambitious plans for future enhancements:

1. Multi-language Support: Expanding to include Swahili and other local languages to further improve accessibility


2. Voice Interface: Adding speech recognition and synthesis for users with limited literacy or visual impairments


3. Integration with Existing Systems: Creating APIs to integrate with university portals and the HEF application system

We’re also exploring ways to adapt this architecture to other public service domains in Kenya, where information accessibility remains a challenge.

Conclusion

The RAG-Powered Virtual Assistant for the Higher Education Fund demonstrates how cutting-edge AI technologies can be applied to solve real-world problems in Kenya’s education sector. By combining Microsoft Fabric’s powerful data capabilities with Azure OpenAI services, we’ve created a solution that democratizes access to critical information about higher education funding.

Our project shows that technical innovation can have meaningful social impact, particularly when addressing specific challenges faced by underserved communities. As we continue to develop and expand this system, we remain committed to our vision of making education more accessible to all Kenyans through technology.

Call to Action

We encourage readers interested in building similar RAG applications with Microsoft Fabric to explore these valuable resources:

• Microsoft Fabric Documentation


• Optimizing Vector Similarity Searches at Scale


• Azure Data Explorer for Vector Similarity Search

These resources were instrumental in our project development and will help you leverage the power of Microsoft Fabric and Azure OpenAI for your own innovative solutions.