Empowering Healthcare AI: A POC for Policy Query with GenAI-powered Chatbot

Aqila Systems developed a Proof of Concept (POC) for a leading consulting organization to showcase its ability to address challenges in creating a GenAI-powered healthcare chatbot. Leveraging Retrieval-Augmented Generation (RAG) and Azure OpenAI, the POC achieved 80% accuracy in query resolution, offering precise and contextual responses linked to policy documents.

CATEGORY: Artificial Intelligence

July 2024

The client,  a global leader in consulting services, was building an in-house GenAI-based healthcare chatbot. Faced with technical and resource challenges, they sought a POC from Aqila Systems to explore a scalable solution and evaluate the potential for collaboration.

Problem and Goals

The client needed a solution to address the following:

• Technical Challenges: Overcoming inaccuracies and hallucinations in AI-generated responses.

• Efficient Query Handling: Ensuring precise responses to healthcare policy-related queries.

• In-House Development Support: Exploring backend architecture that could be expanded into a full-scale product.

The POC aimed to demonstrate:

1. The potential of GenAI and RAG techniques to address these challenges.

2. A backend architecture capable of generating accurate, referenced responses from a knowledge base of selected policy documents.

Solution

Aqila Systems delivered a backend-focused POC that featured:

• Backend Architecture: A robust system integrating Azure OpenAI and RAG techniques for semantic search and query resolution.

• Streamlit-based Front-End Demo: A lightweight open-source interface to showcase chatbot functionality.

• Advanced Features:

  • Topic Modeling: Efficient categorization and organization of policy documents.

  • Prompt Engineering: Enhanced accuracy through optimized prompts to minimize hallucinations.

  • Knowledge Base Integration: Linking responses to specific sections and pages of policy documents to ensure transparency and traceability.

Aqila also provided guidance on improving prompt engineering and mitigating hallucinations, enabling the client to refine their in-house development approach.

Result

While the POC was well-received, the client opted to continue MVP development internally. Key outcomes of the POC included:

• 80% Query Resolution Accuracy: Demonstrating the potential for a highly accurate, LLM-powered chatbot solution.

• Backend Excellence: Showcased Aqila Systems’ capability in AI architecture and problem-solving.

• Actionable Insights: Provided recommendations for improving response reliability and reducing hallucinations in AI-generated outputs.

Conclusion

The POC successfully demonstrated Aqila Systems’ expertise in developing backend architectures for GenAI-powered solutions. It offered the client actionable insights and a clear path to refining their in-house development. This engagement underscores Aqila’s ability to address complex AI challenges and deliver high-value solutions, even in exploratory phases like proof-of-concept projects.