Azure OpenAI Starters | Kickstart Your AI-Application Development

One of the things that I love about GitHub is that it’s easy to find working code for areas that I want to learn. For OpenAI, it has become a great source of working GPT-enabled applications. This article is a compilation of my most-used repositories, created by the wonderful GitHub open-source community. This is not a comprehensive list, but at least a filtered list of the many repos that I’ve tried running and deploying.

Why Azure OpenAI? The main reason is Microsoft’s strong data, privacy, and security statement. This article puts it this way.
Your prompts (inputs) and completions (outputs), your embeddings, and your training data:

• are NOT available to other customers.
• are NOT available to OpenAI.
• are NOT used to improve OpenAI models.
• are NOT used to improve any Microsoft or 3rd party products or services.
• are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data).
• Your fine-tuned Azure OpenAI models are available exclusively for your use. The Azure OpenAI Service is fully controlled by Microsoft; Microsoft hosts the OpenAI models in Microsoft’s Azure environment and the Service does NOT interact with any services operated by OpenAI (e.g. ChatGPT, or the OpenAI API).

Disclaimer: The views expressed in this blog are solely mine do not necessarily reflect the views of my employer. These are my personal notes based on my understanding of the code and trial deployments to Azure.

If anything is wrong with this article, please let me know through the comments section below. I appreciate your help in correcting my understanding.

Chatbot UI (Mckay Wrigley)

Chatbot UI by Mckay Wrigley (repo)

Many organizations today are preventing their employees from accessing chat.openai.com. This is because ChatGPT will by default use the chat history for further improving the model. Many organizations are concerned about employees unintentionally sending confidential information to the public ChatGPT.

But, there is no questioning how ChatGPT is able to help the productivity of many employees. So the simple solution is to deploy an internal ChatGPT, using Azure OpenAI (a “pure” deployment without any customization.)

Chatbot UI is the quickest starter code that I’ve found to do this. This starter…

• can be configured to use OpenAI or Azure OpenAI.
• has a neat extra feature for saving your most-used prompts.
• is mobile responsive.
• is developed using NextJS and the OpenAI official SDK.
• can be tried out at chatbot-ui.com.

Deployment Notes

Chatbot UI can easily be deployed using docker. If you don’t want to modify the code, you may simply use:

docker run -e OPENAI_API_KEY=xxxxxxxx -p 3000:3000 ghcr.io/mckaywrigley/chatbot-ui:main

Personally, I deployed this to a free-tier Azure Static Web App (see my forked repo). Azure Resources of Chatbot UI

Azure OpenAI on your Data (Microsoft)

Azure OpenAI on Your Data by Microsoft (repo)

Probably the TOP use case of organizations today is the ability to chat and ask questions about their internal documents (PDFs, Word documents, PowerPoint presentations, etc.). To quickly deploy something with this use case, Microsoft has developed Azure OpenAI on your Data (Preview).

This starter…

• is not for a “pure” ChatGPT deployment, but about chatting with your data.
• can be configured to search your data only, or also use GPT pre-trained data.
• can configured to use normal keyword search, semantic search or vector search (all using Azure Search.)
• can save the chat history of multiple users into an Azure Comos DB.
• is mobile responsive.
• is developed using TypeScript (frontend) and Python (backend), both are deployed to a single Azure App Service.

This starter is actually a Microsoft product feature. But since the code is in the repo, there’s no stopping an engineering team from fully customizing it, or creating something from scratch with this as a reference. I have a lot to share about using this feature, which I will write in another dedicated post, but for now, let’s focus on how to use this starter.

Do also note that Microsoft has announced M365 Copilot. If your documents are in SharePoint or OneDrive, then consider simply using Business Chat.

Deployment Notes

The best way to deploy this is to follow the official step-by-step guide.

Azure OpenAI Playground

This will create (or use an existing)…

1. Azure Storage account (where your documents are stored)
2. Azure Search with a Search Index and Indexer
3. (Optional) Azure Search Vector DB, for using vector search

And then, you can deploy this to an Azure App Service with the option of also deploying a Cosmos DB to store chat history. Azure Resources of Azure OpenAI on your Data (blurred rows are my other resources, not relevant to this)

Azure Chat (Microsoft)

Azure Chat by Microsoft (repo)

Azure Chat is a “solution accelerator” by Microsoft. Solution Accelerators are starter kits developed with the intent of helping customers deploy an solution to production as quickly as possible. This repository is proven to be a very good starter for many workshops and hackathons.

This starter…

• can also be used as a “pure” ChatGPT.
• allows users to chat with their data, limited to a single file that the user will upload through the app.
• is voice-enabled (speech-to-text.)
• is not (yet) mobile responsive.
• is developed using TypeScript.

Deployment Notes

The step-by-step deployment guide is in the repo itself. This accelerator currently requires every Azure resource to be newly created – including a new Azure OpenAI resource (Workaround: modify the ./infra/main.bicep file to use existing = {}.) But once that’s done, it is easy to make changes to the code and redeploy.

Azure Resources of Azure Chat

Semantic Kernel Chat Copilot (SK Community)

Semantic Kernel Copilot Chat by the SK Community (repo)

See my last post to learn more about Semantic Kernel.

This starter made me fall in love with Semantic Kernel (SK). It is the most sophisticated of the starters in this post.

This starter…

• can also be used as a “pure” ChatGPT.
• allows users to upload multiple documents for chat.
• allows users to attach multiple OpenAI Plugins! The SK Planner will take care of the execution.
• supports multiple Vector DBs! The supported DBs are: Qdrant, Postgres, Volatile, or Azure Search.
• is not (yet) mobile responsive.
• is developed using TypeScript (frontend) and C# (backend), deployed to separate Azure resources.

Deployment Notes

The SK Chat Copilot is also the most sophisticated to deploy. If you noticed, the above starters all have the web application deployed to a single Azure resource. The SK Copilot, instead, uses a frontend TypeScript Static Web App and a backend C# API app deployed into separate Azure resources. I prefer this because this is a very common enterprise architecture requirement.

Azure Resources of SK Copilot Chat

2023-10-13 Update: I just learned that, as of release 0.7 (last week), the frontend static web app is now deployed together in the same Azure App Service as the backend API 😢. This was done to simplify deployment of this solution accelerator. The good news (workaround) is that the frontend app static files are fairly easy to separate and deploy to a different service as needed… with the implication that doing so will require modifications to the deployment scripts and deviate from the main community’s version.

Start Building Your Apps Today!

All of the above are starters are actively being developed by its respective communities on GitHub. So it is expected for you to see enhancements when you start using the starters.

GPT-enabled applications is a rapidly evolving area with many engineers around the world building new innovations around it. This is not the time to wait until a “mature solution” is available. This is the time to adapt and innovate — to be one of the first… or get left behind. I hope this post helps you get started in building AI applications today.