Building a public-facing chatbot powered by Large Language Models (LLMs) comes with some unique challenges, ensuring responses are accurate, grounded, and aligned with ethical guidelines and business regulations.
In this session, we’ll explore how we leveraged Azure OpenAI to create a customer service chatbot that "plays nice"
Some key takeaways:
Grounding responses with confidence: How we leveraged external data sources, content moderation, and prompt engineering to ensure the chatbot consistently delivers accurate and grounded responses.
Preventing harmful interactions, techniques for mitigating risks of the chatbot engaging in inappropriate or harmful conversations, including ethical considerations.
Automating Evaluation Pipelines: How we developed automated systems to evaluate chatbot responses, ensuring measurable improvements and avoiding regressions when updating prompts or models.
Architectural decisions and Azure services that we used to create a scalable and resilient chatbot solution.
Lessons learned from real world usage
Jakob Ehn works as a Cloud architecture and DevOps specialist at Active Solution. For more than 20 years, has has been building software solutions, educating developers and teams on Microsoft technologies through training, books, and conference talks.
Jakob is a Microsoft Azure MVP (former ALM/DevOps MVP). He is a regular speaker at various conferences and user groups around the world, such as NDC, Techorama, DevSum, CloudBrew, UpdateConf and SweTugg.
