Learning Notes | Integration of OpenAI with Enterprise Apps | Part 2 - Development with Semantic Kernel

These notes are outdated. Please find my latest learning notes here. These notes were written during version dotnet-0.13. A lot a has changed since then. I am keeping these notes for my personal reference but please head to my latest.

Last week, I wrote my high-level thoughts on infusing OpenAI with enterprise applications, taking an HR virtual assistant as a sample use case. This post explores how the Orchestrator Service can be built using Semantic Kernel.

Disclaimer: The views expressed in this blog are solely mine do not necessarily reflect the views of my employer. These are my personal notes and understanding I learn SK. For official information, please refer to the official documentation on Semantic Kernel.

If anything is wrong with this article, please let me know through the comments section below. I appreciate your help in correcting my understanding.

What is Semantic Kernel (SK)?

Semantic Kernel (SK) is a “lightweight SDK that lets you mix conventional programming languages with the latest Large Language Model (LLM) AI prompts. It has templating, chaining, and planning capabilities out-of-the-box.” Although SK is relatively new (the first commit was late Feb 2023), it is developed by a large software company, Microsoft, with developers in mind. While LangChain is more mature, it can be hard for someone who knows little about ML engineering to pick up easily. SK on the otherhand simplified the framework into something that developers could easily understand.

SK is primarily for C# developers. Python is available but still in preview support (see feature parity document. If you are a skilled Python developer or ML Engineer, you may consider going straight to my next post on LangChain.

In developing an orchestrator service, SK organizes what needs to be done nicely into skills, memories, and connectors.

flowchart LR
    A>Ask] --> K{Kernel}
    K --> P((Planner))
    P -->|Gather| S[(Skills)]
    P -->|Gather| M[(Memories)]
    P -->|Gather| C[(Connectors)]
    S -->|Inform| T((Steps))
    M -->|Inform| T
    C -->|Inform| T
    T --> L[/Pipeline/]
    L --> G[Get]

Source: SK is a kit of parts that interlock

From this diagram alone, we can immediately picture how the C# project will be organized.

The SK Planner

The planner is one of the ingenious core skills of SK. It takes an Ask as input, which it also refers to as the GOAL. The goal is then broken down into steps, forming a plan that is executed sequentially.

flowchart RL
    G((Goal)) -->|Work Backwards| D[Step N]
    D --> C[...]
    C --> B[Step 2]
    B --> A[Step 1]
    A --> S((Start))

Source: What is Planner?

For example, if the goal is “Apply for vacation leave on the last 2 weeks of December”, the steps would be something like this:

1. Get {user} details
2. Get {fromDate} and {toDate} for the last 2 weeks of December
3. Check {user} eligibility for {computed} days of leave from {fromDate} to {toDate}
4. Apply for leave

The developer who knows the exact flow to achieve the goal can choose to implement it directly. Alternatively, the SK planner can be used to figure out the steps using GPT and prompt engineering. This is the prompt that the planner uses to figure it out:

Create an XML plan step by step, to satisfy the goal given.
To create a plan, follow these steps:
0. The plan should be as short as possible.
1. From a <goal> create a <plan> as a series of <functions>.
2. Before using any function in a plan, check that it is present in the most recent [AVAILABLE FUNCTIONS] list. If it is not, do not use it. Do not assume that any function that was previously defined or used in another plan or in [EXAMPLES] is automatically available or compatible with the current plan.
3. Only use functions that are required for the given goal.
4. A function has an 'input' and an 'output'.
5. The 'output' from each function is automatically passed as 'input' to the subsequent <function>.
6. 'input' does not need to be specified if it consumes the 'output' of the previous function.
7. To save an 'output' from a <function>, to pass into a future <function>, use <function.{FunctionName} ... setContextVariable: ""<UNIQUE_VARIABLE_KEY>""/>
8. To save an 'output' from a <function>, to return as part of a plan result, use <function.{FunctionName} ... appendToResult: ""RESULT__<UNIQUE_RESULT_KEY>""/>
9. Append an ""END"" XML comment at the end of the plan.

[EXAMPLES]
[AVAILABLE FUNCTIONS]
  <!-- functions here... -->
[END AVAILABLE FUNCTIONS]

<goal>Summarize the input, then translate to japanese and email it to Martin</goal>
<plan>
  <function.WriterSkill.Summarize/>
  <function.LanguageHelpers.TranslateTo translate_to_language=""Japanese"" setContextVariable=""TRANSLATED_TEXT"" />
  <function.EmailConnector.LookupContactEmail input=""Martin"" setContextVariable=""CONTACT_RESULT"" />
  <function.EmailConnector.EmailTo input=""$TRANSLATED_TEXT"" recipient=""$CONTACT_RESULT""/>
</plan>

[END EXAMPLES]

[AVAILABLE FUNCTIONS]
{{$available_functions}}
[END AVAILABLE FUNCTIONS]

<goal>{{$input}}</goal>

In short, the planner skill takes the following as input: goal, available functions & skills, and excluded functions & skills, and then calls GPT to figure out the sequence of steps to achieve the goal. Whoever came up with this prompt is a GENIUS.

Semantic Skills and Native Skills

A skill is the terminology that SK uses to categorize related single-purpose functions. For example, the Book Creator sample uses a skill that is called ChildrensBookSkill which contains two functions BookIdeas and CreateBook.

• BookIdeas is the function used to generate the book idea titles after you enter a topic (try running the sample or watching the video.)
• CreateBook is the function used to generate the book itself.

SK also supports both semantic skills and native skills.

• Native skills are C# (or Python) code written in quite similarly to how you would write Azure Functions.
• Semantic skills are prompt-engineered skills. These skills are written and stored in plain text files in a recommended folder structure. SK has figured out a clean-code way to organize prompts into a folder of text files. See this sample.

How to Get Started

The instructions on how to get started are shared here. Here are my top tips on how to learn further:

1. Read the official docs
2. Clone the repo
3. Run the sample in debug mode
4. Join the Discord community and ask questions.

In learning, you will notice that few SK samples show people using Jupyter or Polyglot notebooks; most are about implementing SK in a web service, unlike LangChain, where most blogs and YouTube videos show how people work on a Jupyter notebook.

Next Steps

That’s it for now. My next step is to try LangChain hands-on to understand better how it compares with SK.