RSS and Multi-FAQ Chatbot: RSS Implementation Learnings (Part 3)
This post is part of a series of blog posts about a chat bot RazType™ that I implemented recently. Given that chat bots are advertised as a modern application that’s super easy to do, I decided to implement a prototype to learn through hands-on experience. There are 5 parts to this series:
- High Level Design (Part 1)
- Choices and Decisions (Part 2)
- RSS Implementation Learnings (Part 3) — this post
- FAQ Implementation Learnings (Part 4)
- Adding the Facebook Channel (Part 5)
Parts 4 and 5 are code heavy. For the readers who prefer getting the full source rather than going through snippets, you may find the source code here.
Quick Design Recap
As mentioned in the High Level Design (Part 1), the simple requirement is to create a simple menu based chat bot that returns RSS feed results according to the following dialog conversation flow:
From a bot service perspective, this meant that I will be doing (and learning from) the following:
- Return RSS results based on topics and sub topics (waterfall conversation steps)
- Display the results in a rich card carousel
Base Code
Before I could start, I needed a suitable base code. I didn’t want to start from scratch so I tried to look for existing code samples. I eventually found the following GitHub repos from Microsoft:
Both links above showed C# and NodeJS samples. After looking through the various samples, I realized that there is no I resulted in using the sample code base found in: https://github.com/Microsoft/BotBuilder-CognitiveServices/tree/master/Node/samples/QnAMaker/QnAMakerSimpleBot
Why this? Mainly because I will need to implement QnA as well later.
Waterfall Conversation Flow
With the sample code base, creating the menus and waterflow conversation was fairly easy. Following this documentation page, I wrote the following constants
1
2
3
4
5
6
7
8
9
10
11
12
const mainMenu = {
"News": { item: "newsMenu" },
"Entertainment": { item: "entertainmentMenu" },
"MYX": { item: "myxMenu" },
"Lifestyle": { item : "lifestyleMenu" },
"Sports": { item: "sportsMenu" },
"Push" : { item: "pushMenu" },
"E! News": { item: "eNewsMenu" },
"Choose Philippines" : { item: "choosePhilippinesMenu" },
"TrabaHanap": { item: "trabahanapMenu" },
"Government": { item: "governmentMenu" }
};
And with this, I wrote the following main menu dialog:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
bot.dialog("mainMenu", [
function(session) {
builder.Prompts.choice(
session, "What information are you looking for?", dialogConstants.MainMenu,
{ listStyle: builder.ListStyle.button }
);
},
function(session, results) {
if(results.response) {
session.beginDialog(dialogConstants.MainMenu[results.response.entity].item);
}
}
])
.triggerAction({
matches: /^restart$|^reset$|^home$|^main$|^main\s*menu/i,
});
Retrieving the RSS Feeds
How to read RSS feeds in Node.JS? There are multiple RSS node modules available to do this, but I thought it’d be cool if I can use Azure Logic Apps instead. And after checking, I found that it does have an RSS connector.
Integrating with Azure Logic Apps allows me to use the same service for additional integration points in the future. For example, if I wanted to connect to a social media like Twitter or an internal repository like SharePoint, I will not need to bother looking for multiple node modules but instead let Logic Apps handle the integration. I am thinking of the following architecture with a future roadmap:
HTTP Request
This is how you can create an Azure Logic Apps resource through the Azure portal. After creating, I added an HTTP Request with the following payload
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{
"properties": {
"chatmessage": {
"type": "string"
},
"intent": {
"type": "string"
},
"subintent1": {
"type": "string"
},
"subintent2": {
"type": "string"
},
"subintent3": {
"type": "string"
}
},
"type": "object"
}
My thinking was to get my code pass the chat message and the resolved intent. Given that there are sub-menus in the chat flow requirement, I added 3 extra sub intents (hence subintents 1, 2 and 3 above).
Switch Condition
Initially, after adding the HTTP Request, I used switch conditions according to the intent and subintent. The initial plan was to create an switch case per menu item.
But after some refactoring, I realized that it is easier to handle RSS urls by code – since environment variables makes it easier to reconfigure.
RSS Connector and HTTP Response
And finally, I used the RSS connector and pointed to the corresponding intent/subintent RSS URL.
Well, I thought that was the last step. I quickly realized later that an HTTP Response is actually needed after retrieving the RSS results. This was done through Bot Service application settings. So I added the following step:
Calling the HTTP Request from Node.JS
For Node.JS experts, this step may be easy. But being a beginner actually took me some time to figure it out. After searching with many trials and errors, I ended up with the following Node.JS code to do an Asynchronous HTTP Post:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
function processLogic(chatmessageValue, intentValue, subintent1Value, subintent2Value, subintent3Value, callback) {
var logicAppUrl = process.env.LogicAppPOST;
var options = {
method: "POST",
url: logicAppUrl,
json: {
chatmessage: chatmessageValue,
intent: intentValue,
subintent1: subintent1Value,
subintent2: subintent2Value,
subintent3: subintent3Value
}
};
request(options, function(error, response, body) {
console.log('STATUS: ' + response.statusCode);
console.log('HEADERS: ' + JSON.stringify(response.headers));
callback(body);
}).on('error', function(e) {
console.log("HTTP REQUEST ERROR: " + e.message);
});
}
Displaying Feeds in a Rich Card Carousel
As you may have noticed in my code above, I passed the response to a cardFactory.getRssCardsAttachment method in order to generate a rich card carousel. I referred to the RSS connector reference page to understand the properties of the RSS connector response.
Now this involved some trial and error, as it also depended on the properties that are available from the RSS source itself. In my case, I found that I could only make use of the title, subtitle and text properties. This resulted in the following code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
function getRssCardsAttachment(session, rssArray, defaultTitle, siteUrl, imageUrl) {
if (rssArray.length == 0)
{
return getGoToSiteCardAttachment(
session, defaultTitle,
"Sorry, no content feed is available at this time.",
"",
siteUrl, imageUrl
);
}
var cards = [];
var max = rssArray.length < maxRssResults ? rssArray.length : maxRssResults;
for (var i = 0; i < max; i++)
{
var rss = rssArray[i];
var card = new builder.ThumbnailCard(session)
.title(rss.title)
.images([
builder.CardImage.create(session, imageUrl)
]);
if (rss.subtitle != null && rss.subtitle.trim() != "")
card.subtitle(rss.subtitle);
if (rss.summary != null && rss.summary.trim() != "")
card.text(convertToPlainText(rss.summary));
if (rss.primaryLink != null && rss.primaryLink.trim() != "")
card.buttons([
builder.CardAction.openUrl(session, rss.primaryLink, "Read More...")
]);
else
card.buttons([
builder.CardAction.openUrl(session, siteUrl, "Read More...")
]);
cards.push(card);
}
return cards;
}
To make my carousel look nicer, I just searched for image URLs that could describe the particular menu. I made this URL configurable, placed it in App Settings, and passed this URL as I called the cardFactory. And so, what remains is to send the rich card message:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
var mycallback = function(response) {
var cards = cardFactory.getRssCardsAttachment(
session, response,
"ABS-CBN News",
siteUrl, process.env.NewsImageUrl);
var msg = new builder.Message(session);
msg.attachmentLayout(builder.AttachmentLayout.carousel)
msg.attachments(cards);
session.send(msg);
session.replaceDialog("newsMenu", { reprompt: true });
};
logicAppHandler.getRssFeed(session.message.text, rssUrl, mycallback);
And this is what the result looks like:
