Auto Mapper and Record Types - will they blend?

Give your Form Flow based chat bot new smarts with LUIS

FormFlow is an attractive API offering for building chat bots in the Microsoft BOT Framework in that it greatly simplifies the conversation data management you otherwise need to deal with yourself, to have a useful dialogue with a human.

With FormFlow, you set up predefined processes (or guided conversations) that you would like your human to navigate through, e.g.

Bot: Hello and welcome to Customer Support!

Bot: I'm Botty. What is your name?

Human: Susan.

Botty: Great to meet you, Susan! How can I help you today? 

You can ask me to 1. Get me a Sandwich, 2. Book an appointment or 3. Rant incessantly.

Susan: Please get me a sandwich

Botty: Are you sure you can handle the heat?

Susan: Yes

Your Hot sandwich is coming right up!

The above is accomplished by creating a serializable poco with your Fields, accompanied by two helper methods to turn the poco into a Form:

[Serializable]
public class CustomerSupportExample
{
    [Prompt("I'm Botty. What is your name?")]
    public string CustomerName;

    [Prompt("Great to meet you, {CustomerName}! How can I help you today? You can ask me to {||}")]
    public MenuOptions? Menu;

    [Prompt("Tell me more about this appointment :-)")]
    public string AppointmentDetails;

    [Prompt("Any favourite topic?")]
    public string RantTopic;
    
    public static IForm BuildForm()
    {
        return new FormBuilder()
            // Always greet the Human with a friendly message ... 
            .Message("Hello and welcome to the customer support")

            // Ask our Human what her name is
            .Field(nameof(CustomerName))

            // Add our menu field
            .Field(nameof(Menu))

            // Show our Sandwich field ... 
            .Field(nameof(Sandwich),
                // ... but only if the Human is hungry
                customerSupport => customerSupport.Menu == MenuOptions.Sandwich)

            // Show our Rant topic field ... 
            .Field(nameof(RantTopic),
                // ... but only if the Human is in the mood
                customerSupport => customerSupport.Menu == MenuOptions.Rant)

            // Make sure the human really wants Jalapeños on her sandwich ...
            .Confirm("Are you sure you can handle the **heat**?",
                // ... if a Sandwich is what the Human craves
                customerSupport => 
                    customerSupport.Menu == Options.Sandwich &&
                    customerSupport.Sandwich == SandwichOptions.JalapenoSandwich)

            .OnCompletion(async (context, customerSupport) => {
                if (customerSupport.Menu == MenuOptions.Sandwich) {
                    await context.PostAsync($"Your {customerSupport.Sandwich} sandwich is coming right up!");
                    return;
                }

                if (customerSupport.Menu == MenuOptions.Rant) {
                    await context.PostAsync($"*goes on a rant about {customerSupport.RantTopic} ...*");
                    return;
                }
            })
            
            .Build();
    }
    
    public static IDialog MakeRootDialog()
    {
        return Chain.From(() => FormDialog.FromForm(BuildForm))
            .Do(async (context, supportCase) =>
            {
                try
                {
                    await supportCase;
                }
                catch (FormCanceledException e)
                {
                    await context.PostAsync(e.InnerException == null
                        ? $"You quit on {e.Last}"
                        : "Sorry, I've had a short-circuit. Please try again");
                }
            });
    }      
}

This is all well and good. But what if Susan were to answer "Hi Botty, I'm Susan!". Then, our dear Bot would continue to call her just that:

Bot: I'm Botty. What is your name?

Human: Hi Botty, I'm Susan!.

Botty: Great to meet you, Hi Botty, I'm Susan!. How can I help you today?

Not quite as awesome. Language Understanding Intelligent Service - LUIS - to the rescue!

Working with LUIS

In LUIS, you can create language applications that can help you classify human speech. Starting at luis.at, let's create a New App, give it a useful name - e.g. Customer Support Example - pick Bot as our usage scenario and pick Education as application domains (since we're learning all about chatbots!).

Let's Add App and start working on our Entities and Intents.

We know that we are going to add an Intent that identifies the Human's name. However, before we add the Intent, we must first add the Entities we want to be able to identify.

Add a new Entity, name it CustomerName.

Add a new Intent, name that CustomerPresentedItself and give it a sample utterance, e.g. "I'm Susan" (as a response to "What is your name").

Once you've added the utterance, click on susan and classify the name as CustomerName:

Submit.

Next utterance: "Hi Botty, I'm Susan!". 

Again, click on susan, classifying the name as CustomerName. 

Here, you'll also need to select CustomerPresentedItself as the Intent.

"I'm Caroline; nice to meet you"

"My name is George"

"Maria; how do you do?"

Now enter "I'm Sophie" and see that LUIS understands what you're on about!

We now have an intelligent enough service to use in our bot. 

Push Publish - Publish web service and make a note of the URL (mine looks like this: https://api.projectoxford.ai/luis/v2.0/apps/13996b17-f4f5-4bf3-ae21-1202237c7728?subscription-key=762a29ac2749493181a04941275b60d4&verbose=true).

Integrating our model

Back in our BOT, we'll add a NuGet reference to RestSharp (I had to update some packages first).

Then, we create our LUIS rest client

namespace BotDemoForBlog
{
    using System;
    using System.Collections.Generic;
    using RestSharp;
    using RestSharp.Deserializers;

    public interface ILuisClient {
        Result UnderstandIntent(string input);
    }

    public abstract class LuisClient : ILuisClient
    {
        private readonly string _applicationId;
        private readonly string _subscriptionKey;
        private readonly bool _verbose;
        protected readonly string BaseUrl = "https://api.projectoxford.ai/luis/v2.0/apps/{applicationId}";

        protected LuisClient(string applicationId, string subscriptionKey, bool verbose = true)
        {
            _applicationId = applicationId;
            _subscriptionKey = subscriptionKey;
            _verbose = verbose;
        }

        public Result UnderstandIntent(string input)
        {
            var client = new RestClient { BaseUrl = new Uri(BaseUrl) };

            var request = new RestRequest(Method.GET);
            request.AddUrlSegment("applicationId", _applicationId);
            request.AddQueryParameter("subscription-key", _subscriptionKey);
            if (_verbose)
                request.AddQueryParameter("verbose", "true");
            request.AddQueryParameter("q", input);

            var response = client.ExecuteAsGet(request, Method.GET.ToString());
            return response.Data;
        }
    }

    public class Result
    {
        public string Query { get; set; }
        public TopScoringIntent TopScoringIntent { get; set; }
        public List Intents { get; set; }
        public List Entities { get; set; }
    }

    public class TopScoringIntent
    {
        public string Intent { get; set; }
        public float Score { get; set; }
    }

    public class Intent
    {
        [DeserializeAs(Name = "intent")]
        public string Name { get; set; }
        public float Score { get; set; }
    }

    public class Entity
    {
        [DeserializeAs(Name = "entity")]
        public string Name { get; set; }
        public string Type { get; set; }
        public int StartIndex { get; set; }
        public int EndIndex { get; set; }
        public float Score { get; set; }
    }

    public class PersonNameLuisClient : LuisClient {
        public PersonNameLuisClient() : base(
            verbose: true,
            applicationId: "13996b17-f4f5-4bf3-ae21-1202237c7728",
            subscriptionKey: "762a29ac2749493181a04941275b60d4") {
        }
    }
}

The additional types above - Result, TopScoringIntent, Intent and Entity - were generated by Visual Studio's Paste Paste JSON As Classes command, which you'll find beneath the Edit menu. 

Here, I had to do two changes - instead of Arrays, I changed the collection types in Result to Lists of T (so that RestSharp could instantiate them through their default constructor).

Recognizer

Next step, is to create a new Recognizer class that can understand your Human's textual input:

namespace BotDemoForBlog
{
    using System.Collections.Generic;
    using System.Linq;
    using Microsoft.Bot.Builder.FormFlow;
    using Microsoft.Bot.Builder.FormFlow.Advanced;

    public class RecognizeResponseToNameInquiry : IRecognize where TFormData : class
    {
        private readonly RecognizeString _baseRecognizer;
        private readonly ILuisClient _luisClient;

        public RecognizeResponseToNameInquiry(RecognizeString baseRecognizer, ILuisClient luisClient) {
            _baseRecognizer = baseRecognizer;
            _luisClient = luisClient;
        }

        IEnumerable IRecognize.Matches(string input, object defaultValue)
        {
            if (!input.Contains(" ")) {
                /* Presume that the Human answered with whatever she would 
                 * like to be called (first name, family name, handle) */
                return new []{ new TermMatch(0, input.Length, 1, input) };
            }

            var predictionResult = _luisClient.UnderstandIntent(input);
            var topScoringIntent = predictionResult.TopScoringIntent;
            if (topScoringIntent.Intent != "CustomerPresentedItself" ||
                // Scoring from 0-1; higher than .5 (50% confidence is a good enough guess)
                !(topScoringIntent.Score > .5)) {
                return null;
            }

            return predictionResult.Entities
                //.Select(entity => new TermMatch(entity.StartIndex, entity.EndIndex, entity.Score, entity.Name));
                // -- Consume the entire input by declaring [index 0 to input.Length] as source for the entity
                .Select(entity => new TermMatch(0, input.Length, entity.Score, entity.Name));
        }

        // Let our base recognizer handle the rest of the required functionality
        string IRecognize.Help(TFormData state, object defaultValue)
            => _baseRecognizer.Help(state, defaultValue);
        object[] IRecognize.PromptArgs() => _baseRecognizer.PromptArgs();
        IEnumerable

Field

Finally, we need to create a Field type, which we'll use to tie the above functionality into our CustomerSupportExample.

namespace BotDemoForBlog
{
    using Microsoft.Bot.Builder.FormFlow.Advanced;

    public class NameField : FieldReflector where T:class
    {
        private RecognizeResponseToNameInquiry _luisRecongizer;

        public NameField(string name, bool ignoreAnnotations = false) : base(name, ignoreAnnotations)
        {
        }

        public override IPrompt Prompt
        {
            get
            {
                if (_luisRecongizer == null)
                    _luisRecongizer = new RecognizeResponseToNameInquiry(
                        (RecognizeString)_recognizer, new PersonNameLuisClient());

                TemplateBaseAttribute annotation = _promptDefinition;
                var prompter = new Prompter(
                    annotation,
                    Form,
                    _luisRecongizer);
                return prompter;
            }
        }
    }
}

Tying it up

Tie it all up in the BuildForm method of your CustomerSupportExample:

...

// Ask our Human what her name is
.Field(new NameField(nameof(CustomerName)))

...

Summary

In this article, we've looked at how to integrate LUIS into our Chat bot to understand a response to the inquiry What is your name?. As we've gathered, there's quite a bit to language understanding! Next up, we'll take a look at how to intelligently parse enumerations!