Cognitive Virtual Assistants

Let’s deep dive into the main topic of this book, cognitive virtual assistants. The term assistant can be defined as someone or something that can assist in performing a task or activity. Virtual means something that exists in the virtual world such as software. The term cognitive relates to human cognition: “the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses.” It covers aspects of intellectual functions and processes such as attention, generation of knowledge, evaluation, reasoning, problem-solving, decision-making, comprehension, and formation of language.

In other words, cognition is a human ability that helps us to acquire knowledge, perform all our mental functions, gain knowledge from an environment, and create new knowledge from that.

Some cognitive virtual assistants have some of these capabilities, but they may not be as accurate as a human being today. However, there are certain areas in which they are better than humans as they don’t have the limitation of processing power. In addition, they have an ability to scale and communicate with millions of users simultaneously, which is not possible for a human being.

Let’s now define what a cognitive virtual assistant (CVA) is. A CVA is a software agent that performs tasks either for humans or for machines based on text, voice, and visual input. It has an ability to understand the input provided, interpret it, perform step-by-step operations, and probe a user for missing information. As a result, it can either provide some information or execute an intended task. CVAs have the capability to understand conversation in a natural language and maintain the context of conversation to achieve the end objective.

Some CVAs can be configured to process or interpret voice commands, images, and videos, and then perform an action accordingly. CVAs can respond in various formats depending on a mode of communication, i.e., text when the mode of communication is a messaging system, or voice to respond over voice channels. CVAs can also be embedded in devices such as speakers, navigation systems, and so on.

Nowadays, CVAs are available on multiple devices and interfaces and can be accessed via mobile handheld devices, laptops, and PCs. CVAs can also be integrated with various messaging platforms such as Facebook Messenger, Skype, Telegram, etc.

Use Cases for Cognitive Virtual Assistants

The following are use cases for cognitive virtual assistants.

Google Ecosystem

Google provides a set of enterprise-ready artificial intelligence (AI) services, applications, and tooling. It is a collection of cloud services running on the Google Cloud Platform. For example, Google uses the Natural Language API to identify insights from a large amount of unstructured text data. Google AutoML leverages the latest AI technologies to find an optimal model architecture for specific datasets to perform specific tasks. This section provides details of these AI services, which can be used for the development of various AI-based use cases.

Example: Human-to-Human Interactions

In a chatbot system, a conversation is defined as communication between parties in which news or information is exchanged, questions are asked or answered, etc. We will start with an example of a sample conversation between users and demonstrate how a chatbot system recognizes the intent of the user.

Though these activities look complicated and beyond the capabilities of automated systems, with the advanced conversational capabilities being made available by machine learning technologies, it is possible to create such complex use cases. In subsequent sections, we will take a step-by-step approach to creating a conversation system using the Google Dialogflow engine.

Building Your First Bot Using Google Dialogflow

Let’s go through the steps for creating a conversation interface using Google Dialogflow. Let’s start by creating a Google Dialogflow account. Please note that you should have a Google account before proceeding with the following steps. If you do not have a Google account already, you can sign up for one using the details at https://accounts.google.com/SignUp. Once you have a Google account, you can proceed to the next steps.

Creating a Google Dialogflow Account

Follow these steps to create a Google Dialogflow account:

1. 1.

   Go to https://dialogflow.com/ and click the “sign for free” button.

    
2. 2.

   The site will ask you to log in with a Google account.

    
3. 3.

   Select a Google account for setting up a Dialogflow account. After selecting an account, the site will ask you for the password for the selected account.

    
4. 4.

   After entering the password, you will be redirected to the confirmation screen where you need to accept the terms of service, as shown in Figure 2-1 .

    

After reviewing and accepting the terms of service, you will be redirected to the welcome page of Google Dialogflow, as shown in Figure 2-2.

Using the Dialogflow Agent

The Dialogflow agent is a virtual agent that controls the end-user conversation. It implements natural language understanding technology to deduce the meaning of a human query. It converts natural language text or voice into a structured format that is used as input to an application, service, or third-party API.

An agent can also be defined as a collection or repository of use cases for a particular domain. Ideally, only one agent should be configured for use cases related to one domain. We need to create separate agents for use cases related to other domains. A domain in this context is an industry or vertical domain such as IT, healthcare, banking/financial services, etc.

An agent can also use the Google knowledge base if Dialogflow is not able to answer the user’s query. The knowledge base acts as a searchable document store that can search documents by automatically extracting the intent from a user conversation and matching it to a document in the knowledge base. This feature is in beta at the time of writing of this book. Google Dialogflow can be enriched by enabling other features such as sentiment analysis, spellcheckers, and knowledge bases using agent configurations.

An agent can also use either a rule-based system or an ML+rule-based system for intent recognition. For example, we can use a rule-based system if fewer variations are available and can use an ML system otherwise. A rule-based system does not need a large amount of data to train a machine learning model; it simply uses rules that we configure to find the intent from the conversation.

Now, we will proceed to create an agent and configure it for the travel domain.

Click the Create Agent option in the top-left menu to add an agent, as shown in Figure 2-2. Provide the agent name, agent language, project description, and default time zone. Then click the Create button next to the agent name, as shown in Figure 2-3. We will simply name our agent Travel as we are going to create a use case for the travel industry.

We will be redirected to the screen shown in Figure 2-4.

Defining the Intents and Entities

Before creating the intents, let’s first discuss some of basic concepts related to Google Dialogflow.

Use Case

A use case is a complete end-to-end conversation flow for a scenario. It includes all the possible paths that the system can take to provide a relevant response to the query asked by a user. A simple example is something like Tell me about travel options from Pittsburgh to Boston. The conversation system will recognize the intent, entities, and actions, and it may also trigger your webhook APIs to provide responses to your queries by integrating with third-party systems that may have this data. In Google Dialogflow, a single intent or collection of intents can be referred to as the use case.

Intent

An intent is the verb or action that is part of the conversation with the user. Dialogflow categorizes end-user inputs into the relevant intent. In each agent, a user can define multiple intents, with the combined intents controlling the flow of a conversation. For each intent, we can specify how to respond to the user based on the use case. As an example, an intent for a query from a user saying Tell me about travel options from Pittsburgh to Boston would be Travel since the user is looking for some information related to travel.

For example, in Figure 2-5, one of scenarios shows that if the user says Tell me about travel options from Pittsburgh to Boston, Dialogflow triggers the Travel intent. Also, it extracts the useful parameters from the user input. Thus, the first step in a conversation workflow is the ability of the system to extract the intent. The intent is the most important element in the conversation flow; if the system is unable to extract the intent and match it, the workflow wouldn’t meet its end objective of solving the question or query of the user. As we proceed through the chapters in this book, we will learn how to fine-tune the system to extract the intents clearly, handle scenarios where there may be ambiguity in intent, and use the best practices for configuring our system so that the accuracy of our agent in handling users’ intents is satisfactory.

Intents contain the information covered next.

Training Phrases

For each intent, the training phrases are the set of sample utterances from the end users of our use case. Google Dialogflow uses these phrases to train built-in machine learning algorithms for intent classification. These examples should be captured from end users who are using the system or will use the system. The data from previous chats and paths that user queries took will come in handy to train Google Dialogflow. Chat logs, manual standard operating procedures, call logs, web logs, web form requests, e-mails, etc., will contain information on how the user conversation flow happens and what queries customers are asking related to this use case and domain. The training phrases are also called utterances. This is the end-user data that is used to train the machine learning models so that the model can then accurately determine the intent when a new question is asked by a user by matching it to the already configured training phrases. Readers should note that the training phrases or utterances are not matched one to one with the query that the user is asking, but these are used to train a model, and thus the agent will be able to correctly figure out the intent even if a user’s query is different than any of the training phrases configured in the system. Google Dialogflow uses machine learning and natural language understanding and processing models to figure out the intent. Thus, it is important for a cognitive virtual assistant developer to understand use cases, intents, training phrases, and entities in depth to be able to configure the use case correctly and provide the correct response to new queries.

Action Field and Parameters

When an intent is matched, the Action field value is defined by us, and this value is passed on to the fulfilment webhook requests or APIs. In other words, the action is used to trigger the logic in the workflow.

Parameters, entities, values, and responses are defined under an action, and they are used to extract the required information from the user to complete the workflow, as shown in Figure 2-6.

We can define various parameters as required, which will ensure that the agent will ask the user for this information in a step-by-step fashion as explained in our use case example and will proceed to fulfilment only once all the required information has been extracted in the conversation. After getting all the required data, the agent can then query an external system through a webhook API to provide information or lodge a fulfilment request in the external system. In the case of our travel example, the required information about the source destination, location destination, date and time of travel, and mode of travel are important elements that are needed for the agent to complete the use case.

As an example, the parameter values from a user query Tell me about travel options from Pittsburgh to Boston are Pittsburgh and Boston. We can add parameters to the intent by clicking the New Parameter link. You need to provide the following details for each parameter, as shown here:

• Required: Select this only if it is a mandatory parameter for an intent.

• Parameter Name: Set the name of the parameter. Example: source-city and destination-city.

• Entity: Define the system or developer entity for this parameter. Example: @sys.geo-city.

• Value: Define a variable where the parameter value will be stored. Example: $source-city and $destination-city.

• Is List: Specify whether this parameter should contain multiple values or not.

• Prompts: Regularly ask questions if relevant or required input was not provided. This field is used only if the Required field is selected. Example: Please Enter Source City.

The following example is configured with two parameters (i.e., source-city and destination-city) to extract the source and destination city from the sample query, as shown in Figure 2-7. Each of the parameter has been defined with the system entity @sys.geo-country. So, whenever there is an occurrence of the source and destination city in a query, these parameter values (i.e., Pittsburgh and Boston) will be extracted and stored in the variables $source-city and $destination-city, respectively. If the required parameters are missing, the chatbot will ask for more information with Please enter source city and please enter destination city. Once this information is entered, these variables can be used to generate dynamic responses in order to have an interactive conversation with the user. Here, the entity type defines how these parameter values will be extracted from end-user inputs. We will discuss entities in the subsequent section in detail; for the moment, just think of entities as object types and entity values as objects.

Responses

Google Dialogflow supports built-in response handlers where we can configure a response for an intent. So, whenever the end-user expression matches the relevant intent and entities, the corresponding response will be displayed to the user. We can define different types of responses other than just text formats such as image, audio, speech, etc. The user can use parameter reference variables in the response field to generate a dynamic response. We can even define multiple responses for an intent to make our agent more human-like, where the agent will choose one of the responses at random.

In regard to the previous example, based on the earlier user query (i.e., Tell me about travel options from Pittsburgh to Boston), we captured two parameters: $source-city and $destination-city. We will configure these parameters in the following response text in the “Responses” section, as shown in Figure 2-8:

It’s nice you want to travel from $source-city to $destination-city.

Here, the captured value of $source-city and $destination-city will be Pittsburgh and Boston, respectively, as per the user expression. Hence, the user will see the response It’s nice you want to travel from Pittsburgh to Boston. We can also make our conversation more human-like by creating multiple responses like this:

Have a safe trip, you want to travel from $source-city to $destination-city.

Figure 2-9 shows the basic flow of intent matching.

Entities

Entities are nouns that combine with the intent to identify a particular action. An entity also specifies how parameter values will be extracted from user expressions. In the previous example, the system entity (@sys.geo-city) is used to extract the parameter values (Pittsburgh and Boston). Each parameter has an entity type, which shows what type of data is to be extracted and stored in the parameter value.

System Entities

Google Dialogflow provides a set of predefined system entities such as @sys.email, @sys.number, @sys.phone-number, etc. Dialogflow enables system entities automatically if a training phrase has one of these entities, as shown in Figure 2-10.

System Entity Extension

Dialogflow provides a set of predefined entities to extract the information from end-user expressions. But for most use cases, it may require extra values that are not predefined by Dialogflow. We can extend system entities for this purpose. In the Training Phase section on the intent page, we can add values to a system entity by annotating a training phase.

Regexp Entities

Dialogflow provides features for defining regular expressions as entity entries. This is useful if the user wants to extract or recognize entity entries from user expressions that follow certain patterns such as for a phone number, e-mail address, country, etc. For example, if we want to define a regular expression to extract an e-mail address, then we can define the regular expression ^[\w.]+@[a-zA-Z_]+?\.[a-zA-Z]{2,3}$, as shown in Figure 2-11.

Referencing Entity Values and Synonyms in the Training Phrases

Since we’ll be defining the entities and values related to the variations, we can add entity values or synonyms in the variations to create a relation mapping between the intents and the entities. Returning to the previous example, say we want to add the variations related to travel options. We have an entity called @sys-geo-country with entries such as Pittsburgh and Boston, so we can add these values as part of the variations. Figure 2-12 shows some examples of such variations.

Batch Operations

Dialogflow provides the ability to copy, move, and delete multiple entities using batch operations , as shown in Table 2-1 and Figure 2-13.

Table 2-1

Batch Operations

Operation	Description
Copy	This is used to transfer the selected entities into the destination agent without removing any in the current agent.
Move	This is used to transfer the selected entities into the destination agent and remove any from the current agent.
Delete	This is used to delete the selected entities permanently from the agent.

Testing the Intents

After defining the intents and variations, we can test them in the Google Dialogflow “Try it now” panel available on the right side of the agent configuration page, as shown in Figure 2-14. And we can see, the intent is triggered based on the variations added, and the correct response has been populated.

We have explained the basic concepts of Google Dialogflow that are necessary for training a conversation system. Let’s now create a use case in a step-by-step manner.

We will start with the identification of a use case. For this example, we are continuing with the use case from the travel domain that we have used in the previous examples.

The chatbot system can be integrated with a travel web site where users can log in and start conversing with our chatbot system. As an example, the user wants to travel from one place to another. The user is looking for travel options such as train, flight, etc., and the cognitive bot will capture the intent of the user and decipher what it is.

You can download the agent zip file that has the use cases configured for the travel domain. Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_2 to download the Create_First_Bot.zip file.

Once you have downloaded the agent, you can import it into Google Dialogflow.

Let’s see how to import a precreated agent into Google Dialogflow.

1. 1.
   Go to the Settings page and then click Export and Import; Dialogflow will redirect you to the page shown in Figure 2-15.

   

    
2. 2.
   Click IMPORT FROM ZIP, upload the Create_First_Bot.zip file that you downloaded, and type IMPORT. Then click Import, as shown in Figure 2-16.

   

    
3. 3.
   Once you click Import, the agent will be imported to Google Dialogflow. Then go to the Intents page, where you can see the intent for the travel use case, as shown in Figure 2-17.

   

    

Once we have identified a use case to work upon, we will move on to the intent, training phrases, entities, and response configuration.

Working with Intents

The following sections explain how to work with intents.

Default Intent

By default, Dialogflow has two intents, named Default Welcome Intent and Default Fallback Intent, as shown in Figure 2-18.

• Default Welcome Intent : This intent is triggered when the user starts a conversation with the bot. The intent gives a response to the user with the information about what your agent does. This is the first response from the bot to the user.

• Default Fallback Intent: This intent is triggered if no condition is matched in any of the intents in the hierarchy. If the workflow matches the condition of any other intent, this intent is not executed. Thus, if none of the intents is matched, the agent can respond by gracefully exiting the conversation, as shown in Figure 2-19.

The user can add training phrases explicitly to the default fallback intent that will act as negative examples. In some situations, there may be similar sentences, but they may not match with the intent; these negative example variations will ensure that the intent is matched correctly and excludes the negative training variations.

Create and Train a New Intent

Let’s create and train a new intent named Travel_info. The entity parameters are as follows:

• Entity type: Travel_option

  • Entity entries: Train, Flight

• Entity type: Airlines

  • Entity entries: American Airlines, JetBlue

With the previous information, next we need to define utterances or training phrases that the user might ask when using the bot, as shown here:

• I want to book a flight from Chicago to San Francisco

• I want to explore tourism options in Las Vegas

• Help me with travel options from Las Vegas to Brooklyn

• Travel options from Chicago to Boston

• I want to travel from Los Angeles to San Francisco

Ideally, in a practical scenario, these phrases should be collected from end users to provide actual variations to the system for training.

Let’s move on to creating an intent with training phrases, entities, and responses.

• Log in to the Google Dialogflow console.

• Select the agent named Travel that we created earlier.

• In the top-left menu, click the + sign next to Intents. Provide Travel_info as the intent name and click the Save button, as shown in Figure 2-20.

  

• Click the drop-down menu next to “Training phrases” and then click the ADD TRAINING PHRASES link to add possible user utterances, as shown in Figure 2-21.

  

• Enter training phrases or user utterances one by one and press Enter. These are the sample variations that we will feed into the Google Dialogflow engine for training. Google Dialogflow will automatically annotate entities to relate variations with intents, as shown in Figure 2-22.

  

As per best practices, please ensure the following criteria are met while configuring training phrases:

• Add at least ten examples/variations or more as training phrases for an intent. The more variations, the better the training, and that results in better responses from the bot.

• Two different intents shouldn’t have similar types of training phrases. This is required to avoid any kind of overlap during intent recognition.

Dialogflow provides limited types of system entities. To match any custom data, the user can create a developer entity such as Travel_option.

Follow these steps:

1. 1.

   Go to the left menu of the agent and click the Entities tab. Then click CREATE ENTITY.

    
2. 2.

   Fill in the entity name at the top of the page. This defines the type of information the user wants to extract from the query. Enter Travel_Option as the name here.

    
3. 3.

   Click the first row and enter an entity value and synonym. Here, enter Flight with a synonym of Flight.

    
4. 4.

   Like previously, fill in the next rows as per the requirements, as shown in Figure 2-23. Here, enter Train with a synonym of Train.

    

After entering these values, your screen will look like Figure 2-23.

The “Allow automated expansion” option is used to match the entity values that are explicitly defined in the developer entities. If the user mentions some value that is not included but similar to an entity, then this feature recognizes the value as a parameter in the entity.

The “Fuzzy matching” option is used if the user wants to enable the matching of incorrectly spelled entity entries from user expressions. Google Dialogflow uses built-in machine learning and natural language processing algorithms for fuzzy logic. You can select these options or leave them unchecked.

Defining an Action and Parameters

Now we will configure an action and parameters so that we can extract the relevant entities from the conversation and prompt the user appropriately.

Configure the parameters for our use case as per Table 2-2. Then we will discuss the configuration of these parameters.

Table 2-2

Action and Parameters

S. No.	Required	Parameter Name	Entity	Value	Is List	Prompts Text
1	Yes	source-city	@sys.geo-city	$source-city	No	Please enter source city.
2	Yes	destination-city	@sys.geo-city	$destination-city	No	Please enter destination city.
3	Yes	Travel_options	@Travel_option	$travel_option	No	That’s nice. Please tell me how you want to travel.
4	Yes	airlines	@Airlines	$airlines	No	Please mention airline.
5	Yes	date	@sys.date	$date	No	Please mention date.

For each parameter, configure the following properties:

• Required: Check this for all parameters. This restricts users to providing values for all the parameters.

• Parameter Name: Provide the name of all parameters as mentioned in Table 2-2 in the column “Parameter Name.”

• Entity: This defines which entity entries will be stored in a particular parameter. For our use case, define the entity as per Table 2-2.

• Value: This is a variable that stores values of entities from a user’s query. For our use case, define variables as per Table 2-2 in the “Value” column.

• Is List: Define whether a parameter can have multiple values. For our use case, each parameter will have only a single value. Hence, don’t select this option for any parameter.

• Prompts: A text message will be prompted to the user if values of parameters are not provided by the user. Please note that this will be enabled if you select the Required option for a parameter.

Once configured, your system should look like Figure 2-24.

We can configure the prompts for each parameter by clicking the “Define prompts” link in the Prompts column for each of the required parameters, as shown in Figure 2-24.

This opens a new window to define your own prompt text for the parameter Travel_option, as shown in Figure 2-25. Define a prompt for the parameter Travel_option as That’s nice. Please mention how you want to travel. and click the Close button.

For this use case, configure the prompt text for all the parameters as per the details shown in Table 2-2. The “Action and parameters” fields will look like Figure 2-26. After defining all the parameter prompts, click the Save button next to the intent name to save the changes to the “Action and parameters” configuration.

The required parameters ensure that the agent will ask the queries to probe the users for additional information until all the required information is captured.

Once all the required parameters are filled in, then the parameter values can be used to either provide the final response to the user as configured in the system or use webhooks to query external APIs to provide the response; if the agent fails to match the intent and entities, then the fallback response is provided to the user.

Defining the Responses

Now, we can configure the response by clicking the drop-down button next to the Response section for the intent Travel_info. Define the response message as follows:

Your $airlines flight ticket has been booked. Below is your itinerary details. Source City: $source-city Destination City: $destination-city Date: $date Departure Time: 10:00 AM EDT Arrival Time: 11:30 AM EDT PNR Number: 12345671

In the response message, parameter values are used to generate dynamic responses (i.e., $airlines, $source-city, and $destination-city).

For this use case, we have hard-coded the values for the departure time, arrival time, and passenger name record (PNR) number to show how a response should be configured. Click the Save button to save the response message, as shown in Figure 2-27. Here, we can add any type of message such as images, speech, or messages from other third-party messaging platforms.

We have configured intents, entities, parameters, and responses for the Travel_info use case. Google Dialogflow automatically starts training the agent as soon as an intent and its complete details have been configured.

Next, we need to test whether our configured use case is working well. For that, go to the test console, and at the top-right corner of the Dialogflow console, enter the query I want to travel from Log Angeles to San Francisco. Then click the Enter key, as shown in Figure 2-28.

Dialogflow will send the query to the agent, which in turn classifies the user expression to the relevant intent (i.e., the Travel_info intent) followed by identifying the entity entries of type @sys.geo-city (a system entity). Since the user hasn’t specified the mode of travel and other parameters in the initial query, the Dialogflow agent will prompt the user with the defined prompt text for the other parameter values, as shown in Figure 2-29, Figure 2-30, Figure 2-31, and 2-32.

Integrating with the Built-in Bot Framework Web Demo

Google Dialogflow provides a built-in bot known as Web Demo to which we can link our agent. To enable this integration, go to the top-left menu and click Integrations; then enable the Web Demo connector by clicking the toggle button, as shown in Figure 2-33.

Once enabled, we can click the Web Demo icon. This takes us to the Web Demo configuration screen. Now, we can see the Web Demo URL that has been generated automatically for this agent, as shown in Figure 2-34.

Trying Your First Bot

Copy the highlighted URL as shown in Figure 2-34 and open it in the browser of your choice; this will open the conversation window, as shown in Figure 2-35. Note that the URL for your system will be different than the URL shown here.

Now, we can try the use cases that we have configured in our Google Dialogflow agent, as shown in Figure 2-36, Figure 2-37, and Figure 2-38.

Conclusion

In this chapter, we designed our first cognitive chatbot using Google Dialogflow. We also discussed basic concepts such as intents, entities, parameters, etc., with an example to give you a glimpse at the building blocks that are essential to creating conversation use cases in Google Dialogflow.

Input and Output Contexts

In previous chapter, we configured a simple use case where we triggered a single intent (i.e., Travel_info) with actions and parameters defined as source-city, destination-city, Travel_options, airlines, and date with probing questions to extract the entity values from the conversation, as shown in Figure 3-1.

Now, let’s take a step forward and imagine a scenario where a user might ask further questions based on the response of the previous intent. So, when the agent responds with Are you looking for any other bookings? the user might say cab booking or accommodation to continue a conversation flow. We’ll create separate intents for these services, and the challenge is connecting these intents as part of the conversation flow with the first intent.

Dialogflow provides a feature called input/output context that controls the flow of conversation and branching. Contexts can be set for intents and can be reused further in other intents. Whenever an intent is created, we get two options: input and output contexts. For the first intent, we set the output context, and then every other intent that is to be linked to the first intent needs to have its input context defined to be the same as the output context of the first intent. This way, these two intents can be associated with each other. We can define multiple input and output contexts to configure more complex use cases.

To understand this, let’s continue with the previous example wherein the agent gives the Are you looking for any other bookings? response. Here the output context of the intent Travel_info is set to Travel. So, every intent that has the context Travel that you set as its input is a potential intent that could be triggered. If the user gives the response cab or hotel booking, then to continue the flow of conversation, we need to set the input context in the cab and accommodation intents to be the same as the output context set in the Travel intent. This continues the conversation flow of the respective service, cab booking or hotel booking, as shown in Figure 3-2.

As shown in Figure 3-2, if the user asks a travel-related query, it will trigger the Travel_info intent, and then its output context will be set to Travel. Here, the Cab and Accommodation intents are going to activate as their input contexts are defined as the Travel context. Now, if the user asks cab- or accommodation-related questions, the respective intent will be triggered based on the user query matching with the training phrases defined in the intents. But if a user’s query does not match any of the active intents, then it will go to the fallback intent.

You can download the agent that has the use case configured for the travel domain using the input and output contexts. Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_3 to download the Travel_IO_context.zip file.

Import the downloaded agent into Google Dialogflow, as explained in the Chapter 2.

Let’s configure the use case where we will be showcasing how to link the cab intent with the Travel_info intent using the input and output contexts. As a first step, we’ll define the output context as Travel for the first intent, Travel_info, that we created in the previous chapter, as shown in the Figure 3-3.

Configure the cab intent with the following information, as shown in Figure 3-4 and Figure 3-5:

• Intent: Travel_Cab

• Entity type: Cab_Vendor

• Entity entries: Uber

• Training phrases:

  • I want to book a cab from San Francisco airport to Parker guest house at 5pm

  • I want to travel to airport using Lyft

  • Show me some cab details

  • Cab services

Now, define the input and output contexts for the Travel_Cab intent. Click the drop-down menu next to the Contexts label. Then, enter the input context as Travel, which is the same output context of the Travel_info intent, and enter the output context as Cab. Here, defining the output context for the intent Travel_Cab is optional, as shown in Figure 3-6. It is defined to include subsequent cases in upcoming sections.

To book a cab, the agent will need some mandatory information from the user such as the source, destination, day and time of travel, and preferred cab vendor. This information can be asked of the user as per the configuration in the “Action and parameters” section of the intent Travel_Cab, which is shown in Figure 3-7.

The values for the parameters Cab_vendor, Pick-Up-Location, etc., are being stored in variables that can be used in the Responses section to make the bot more interactive. For example, the Text Response entry in Figure 3-8 will generate custom messages based on the values of parameters.

Click the Save button next to the intent name to save the changes. Now, you can try your chatbot configuration in the “Try it now” panel, as shown in Figure 3-9, Figure 3-10, and Figure 3-11.

Follow-Up Intents

In “Input and Output Contexts” section, we discussed how multiple intents can be combined to form a conversation flow using contexts. However, Dialogflow provides a feature known as follow-up intents. A follow-up intent is like a child intent of the associated parent intent. Since it will be follow-up intent, all input and output contexts will be set automatically. The follow-up intent will be triggered only after the conversation has already matched with the parent intent in a previous conversation.

Now, what’s the difference between the two if both of them are doing same thing?

If we use the follow-up intents feature, then the child intent would be triggered only after triggering the root intent. Please note that the child intent can’t be triggered from any other intent available in the agent even if its input context has been set by that intent. However, if we use the input and output contexts, then follow-up intents can be triggered from any other intent available in the agent if the input context to the follow-up intent has been set by that intent, as shown in Figure 3-12.

As shown in the figure, if the user expression matches the training phrases of Intent A, then only their follow-up intents will be matched further, either Follow-up Intent 1 or Follow-up Intent 2. And their input contexts will be automatically set as the output context of the parent intent. Once the follow-up intent is matched, it will give a response.

You can download the agent that has the use case configured for travel using follow-up intents.

Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_3 to download the Travel_Follow_up.zip file.

Import the downloaded agent into Google Dialogflow, as explained in Chapter 2.

Next, we will configure same intents as defined in earlier sections (i.e., Travel_info and Travel_Cab) and connect them using the follow-up intents feature of Dialogflow. Configure the intent Travel_info in the same manner as defined in Chapter 2.

Now, right-click the intent Travel_info and select the “custom” option from the drop-down menu, as shown in Figure 3-13.

This will create a custom intent with the name Travel_info-custom, as shown in Figure 3-14. Rename this custom intent to Travel-cab.

The output context (Travel-followup) of the Travel_info intent and the input context (Travel-followup) of the Travel-cab intent will be set automatically, as shown in Figure 3-15 and Figure 3-16.

Now, consider the scenario where the user is looking for either cab bookings or accommodations just after confirming the travel. In other words, either the cab booking intent or the accommodation intent should be triggered after the Travel_info intent.

The follow-up intents feature of Dialogflow can be used here to create a child intent (i.e., Travel_accommodation, with Travel_info being the parent intent). Follow the same method as defined earlier in this section to create the follow-up intent. Right-click the Travel_info intent and select “custom” in the drop-down menu; a new follow-up intent with the name Travel_info-custom will be created. Rename it to Travel-accommodation and click the Save button next to it, as shown in Figure 3-17.

Next, you can see that the input context of the intent Travel-accommodation has automatically been set to Travel-followup, which is the output context of the parent intent Travel_info, as shown in Figure 3-18.

As we can see, the new entity type (i.e., the regexp entity) has been defined in this intent. It will be used to extract the duration of stay at a hotel from the user’s query. It will identify patterns such as three nights, two days, etc., as shown in Figure 3-19.

Next, configure the mandatory parameters such as Room_Type, Duration, Hotel_name, from-date, and to-date in the section “Action and parameters” of the intent Travel-accommodation. The trickiest part here is how to showcase values that have been captured in the root intent as part of the Responses section of the current intent. We can refer those values with a combination of the output context of that intent and the variable name where the information has been stored.

In the following example, destination-city has been captured from the root intent. For this, a combination of the input context (i.e., Travel-followup) and variable name destination-city is used as #Travel-followup.destination-city, as shown in Figure 3-20.

Now, go to the top-left menu and click Integrations. Enable Web Demo, and it will show up as a link to your chatbot agent. Open it in a browser, and you can start conversation, as shown in Figure 3-21.

In Figure 3-22, the user mentioned that he is looking for cab services, and accordingly the chatbot responded to the user’s query.

But this time, the user has also asked for accommodation services, and the chatbot responds accordingly, as shown in Figure 3-23 and Figure 3-24.

Once all the mandatory parameters are captured, the chatbot responds. Here, the destination-city San Francisco has been captured, as explained earlier using a combination of the context and variable name.

Handling Multiple Intents (Intent Priority)

Dialogflow comes with a unique implementation to handle multiple intents, called intent priority. By default, all intents have a normal priority. If multiple intents have been recognized from a query, then we can set the priority, and the highest priority intent will be triggered.

For example, in the query I am traveling from the United States to the United Kingdom and I want to book a cab from the airport to the hotel. So, two intents (i.e., Travel_info and Travel_Cab) should be recognized, but the order of their execution matters. As per the defined use case, the Travel_info intent should be triggered first, and then Travel_Cab intent should be triggered. In this case, during configuration, the priority of the Travel_info intent should be set to higher than the Travel_Cab intent.

Consider a case where we have only two intents, Travel_Cab and Travel_info, as shown in Figure 3-25. For these intents, configure the options in “Action and parameters,” the entities, and the responses, as mentioned in the previous sections.

Now, we will use the “Try it now” panel to see how our conversation system is responding with the default configuration (i.e., all intents at normal priority).

The user has entered the query I want to travel from US to UK and I also want to book cab. The chatbot has recognized both intents simultaneously, but because of a high confidence score of the Travel_Cab intent, it has triggered this intent before the Travel_info intent, as shown in Figure 3-26, where it is asking questions from the Travel_Cab intent instead of the Travel_info intent. Ideally, the Travel_info intent should be triggered first, before the Travel_Cab intent.

We can solve this problem by increasing the priority of the intent Travel_info from Normal to Highest. So, whenever two intents are being recognized from a single user’s query and Travel_info is one of them, then it will always be triggered first.

To increase the priority of the intent Travel_info, click the circle icon (in red) just before the name of intent, as shown in Figure 3-27. Change the priority of the Travel_info intent to Highest.

Now, open the “Try it now” panel and enter the same query (i.e., I want to travel from US to UK and I also want to book cab). This time you can see the Travel_info intent has been triggered first, as shown in Figure 3-28. Please note that the Travel_info intent will always be triggered even if the user enters the query I want to book cab and I also want to travel from US and UK.

Conversation List

Click the Training tab in the left sidebar menu. You will be redirected to the page that shows the list of conversations. The agent conversation log will automatically appear in the conversation list. You can also upload log data from other sources by uploading the text data, as shown in Figure 3-29.

Click the UPLOAD button. Then click the Choose File button to upload the text file containing the conversation log, as shown in Figure 3-30.

Once the file is successfully uploaded, then click the Back button to view the uploaded list of conversation logs, as shown in Figure 3-31. It shows the list of all the requests with the date, total number of requests, and number of requests that are not matched to an intent. For each request, the matched intent will be shown below the user query, as shown in Figure 3-31.

For requests not matched to a specific intent, the default fallback intent will appear, as shown in Figure 3-32.

For each request, you can choose to perform an action to improve your agent.

You can add the request to the currently matched intent by following these steps:

1. 1.

   Click the check mark icon at the top right.

    
2. 2.

   The check mark will turn green; click the Approve button to update the intent, as shown in Figure 3-33.

    

The user can also add the request to a different intent.

Click the current intent name, and you will see the list of associated intents to the agent, as shown in Figure 3-34.

Select the intent you want to add, as shown in Figure 3-35.

The check mark icon changes to green; click the Approve button, as shown in Figure 3-36.

After adding a request to an intent, you will see the selected intent, as shown in Figure 3-37.

Agent Settings

We can apply various agent settings to an agent.

1. 1.

   Go to the Google Dialogflow page.

    
2. 2.

   Click the Settings button next to the agent in the left sidebar menu. This opens the agent’s settings page, as shown in Figure 3-38.

    

Here is a breakdown of the settings:

Description: Give a description of your agent.

Default Time Zone: Select the time zone of your agent.

These are the options under Google Project:

• Project ID: This is the project ID of the Google Cloud Platform project linked to the agent, as shown in Figure 3-39.

• Service Account: This is the service account for authentication.

One option appears under API Version:

• Select the API version the for agent, which here is V2 API, as shown in Figure 3-40.

These are the settings under Delete Agent:

• Delete This Agent: This completely deletes the agent and cannot be undone, as shown in Figure 3-41. Remove all the users before deleting the agent.

These are the settings under Languages:

• The default language is set to English. You can add multiple languages.

• Click “+Add locale” to add a location (if available).

• Click Save after adding a language and locale, as shown in Figure 3-42.

Under ML Settings, choose one of the options according to the agent, as shown in Figure 3-43.

• Hybrid (Rule-based and ML): This is for agents with a small number of examples/templates of intents.

• ML only: This is for agents with a large number of examples of intents.

These are the spelling-related options:

• Automatic Spell Correction: Enable this to correct the spelling in queries, as shown in Figure 3-44.

• Automatic Training: Enable or disable automatic agent training each time the agent is modified, as shown in Figure 3-44.

• Agent Validation: Enable this to automatically validate the agent after performing agent training, as shown in Figure 3-44.

These are the options on the Export and Import tab :

• Export as ZIP: Exports the agent as a zip file, as shown in Figure 3-45.

• Restore from ZIP: Overwrites the current agent with the supplied zip file, as shown in Figure 3-45.

• Import from ZIP: Adds intents and entities to the current agent from the supplied zip file. If any existing intents or entities have the same name as those in the zip file, they will be replaced.

On the Environments tab, you can publish and create versions of your agent for a specific environment, as shown in Figure 3-46. Multiple environments can be set up depending on your needs, like Development, Testing, and Production.

These are the options under Improve Speech Recognition Quality:

• Enable Enhanced Speech Models and Data Logging: Toggle this button to enable enhanced speech models and logging, as shown in Figure 3-47.

• Enable Auto Speech Adaptation: Toggle this button to enable auto speech adaptation, as shown in Figure 3-47.

These are the text to speech options:

• Enable Automatic Text to Speech: Toggle this button to enable the automatic conversion of text to speech for every conversation, as shown in Figure 3-48.

• Output audio encoding: Select the encoding type (16-bit linear PCM, MP3, Ogg Opus).

• Voice Configuration: There are several options here:

  • Agent Language: Select the agent language.

  • Voice: Choose a voice synthesis model.

  • Speaking Rate: Adjust the voice speaking rate.

  • Pitch: Adjust the voice pitch.

  • Volume Gain: Adjust the audio volume gain.

  • Audio Effects Profile: Select the audio effects profiles that you want applied to the synthesized voice.

• Experiment with Different Voice Setting: Type some text and hit Play to test different voice settings.

On the Share tab, you can send an invite to new people to share and grant access to your agent, as shown in Figure 3-49.

On the Advanced tab, under Sentiment Analysis, toggle this button to enable sentiment analysis and provide a sentiment score for each user query, as shown in Figure 3-50.

Multilingual Chatbots

Consider a scenario where you need to build a chatbot for an organization whose users are spread across different geographic locations and converse in different languages. How would you build a conversation system that can handle such diversity?

The preferred approach is to build an end-to-end system in multiple languages for higher accuracy. In this scenario, we need to provide training phrases for intent classification, entity variations, and parameter prompts if required in the target language. This is how Google Dialogflow implements a multilingual conversation system.

You can download the agent that has the use case configured for the travel domain in different languages.

Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_3 to download the Travel_multilingual.zip file.

Import the downloaded agent into Google Dialogflow, as explained in Chapter 2.

Go to the top-left menu and click the + sign next to the default language (i.e., en), as shown in Figure 3-51. This will open the Language tab on the agent’s settings page. Set the language to French and click the Save button to save the changes, as shown in Figure 3-52.

Now, a new language, i.e., French, has been added to your agent. Next, you need to configure both intents (i.e., Travel_info and Travel_Cab) with the training phrases in French, as shown in Figure 3-53.

Next, define the prompt text for parameters in French only, as shown in Figure 3-54. Also, change the response in the Responses section to French, as shown in Figure 3-55.

Now, go to Integrations, open the Web Demo URL, and enter the query Je veux voyager des États-Unis au Royaume-Uni, which is the French translation of I want to travel from US to UK. The agent recognizes the intent as Travel_info for this query and responds accordingly, as shown in Figure 3-56.

Prebuilt Agents

Prebuilt agents are agents trained on common topics such as weather, traffic, unit conversion, and so on. These agents have a predefined set of intents along with training phrases, entities, and parameters. So, if you want to leverage use cases related to these common topics, then you can use them in a new agent or copy them into an existing agent.

As a continuation of our travel use case, if the user is also looking for the weather conditions of the location where he is planning to travel, then we can use prebuilt agents to configure this in our existing agent. We will start with the configuration of a prebuilt agent, which will create a new agent by default, and then we will move intents from this newly created agent to the existing one.

Go to the left menu bar and click Prebuilt Agents. Select the Weather prebuilt agent, and click the IMPORT button, as shown in Figure 3-57 .

You can also click the VIEW DETAILS link just below the IMPORT button to see some of the sample user utterances that have been configured in some of the intents of the Weather agent, as shown in Figure 3-58.

Wait for a few seconds, and Dialogflow will create a new agent with the name Weather, as shown in Figure 3-59.

Now, a new agent with the name Weather has been added to our account. To check, click the drop-down menu next to the current agent name (i.e., Travel) and click the “View all agents” link, as shown in Figure 3-60.

This will show a list of the available agents to our account where we can see the Weather agent listed, as shown in Figure 3-61.

You can also see a list of preconfigured intents listed under the Weather agent by clicking the Intents tab in the left menu bar, as shown in Figure 3-62.

We can also see training phrases, contexts, and entities for each of intents corresponding to the Weather agent. For example, click one of intents (i.e., weather) for the Weather agent. This will populate details such as training phrases, contexts, entities, etc., for the intent called weather, as shown in Figure 3-63.

The Weather agent populates some default parameters as part of the weather intent. But for now, we have removed all parameters as we just wanted to show the integration to our Travel agent. We will discuss the parameters for the weather intent in the next section.

Since we want our chatbot to be able to answer weather-related questions, we will copy the weather intent from the Weather agent to our Travel agent to answer weather-related queries for a city. Select the weather intent to open up a menu on top with the options such as COPY, MOVE, DELETE, and CANCEL. Click the COPY button, as shown in Figure 3-64.

Next, it will ask for the target agent. Select Travel [en] as your target agent and select the check box “Copy related entities.” Once done, click the Start button, as shown in Figure 3-65.

This will then add the weather intent along with its associated entities to our Travel agent. After a successful transfer, click the PROCEED TO AGENT button to see a list of intents in our Travel agent, as shown in Figure 3-66.

Now, the weather intent has been added to our Travel agent. So, users can ask questions related to the weather conditions. Go to the “Try it out” panel and enter the query shown in Figure 3-67.

In Figure 3-67, our Travel agent is able to categorize the query (i.e., Show me weather) into the weather intent.

In this section, we have copied the weather intent and entities from an existing Weather agent. Our Travel agent is now able to analyze weather-related questions. In the next section, we will discuss fulfillment and webhooks, which can be used to integrate with the Weather API to fetch weather information.

Fulfillment: Integration with the Weather API

Up until now, we have integrated the weather intent and related entities from the prebuilt Weather agent to our Travel agent. Next, we will discuss how our webhook service can be used to integrate Dialogflow and the Weather API.

Before that, we will deep dive into some basic concepts such as fulfillment, webhook services, and third-party APIs, as shown in Figure 3-68.

Open Source Weather API

Go to the following link to get an open source weather API:

https://home.openweathermap.org/

You will be redirected to the page shown in Figure 3-69. Click the sign-up button to create an account.

Fill in the required details to create an account, as shown in Figure 3-70.

A confirmation e-mail with your API key and technical instructions will be sent to your registered e-mail address. Please note that it takes up to two hours to activate the API key, as shown in Figure 3-71.

As the next step, go to the API tab to check the freely available weather API document.

Check this API URL through a browser as well. It will show the weather forecast details, as shown in Figure 3-72.

In this case, we are using the URL to call the API with the city name: http://api.openweathermap.org/data/2.5/weather?q=DELHI&units=metric&APPID=19e8588cb3d7d0623e3a5a8ec529232f.

In this section, we will showcase the integration of the Weather API to Google Dialogflow.

The complete scenario works as follows:

1. 1.

   Dialogflow calls the webhook service via fulfillment with the required input.

    
2. 2.

   The webhook service processes an input request from Dialogflow and sends it to the Weather API.

    
3. 3.

   The Weather API processes requests from the webhook and returns a response to the webhook service.

    
4. 4.

   The webhook service converts a response from the Weather API to a format that Dialogflow can understand and send it to Dialogflow via fulfillment for further processing.

    

You can download the agent that has the use case configured for weather.

Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_3 to download the Travel_weather.zip file.

Import the downloaded agent into Google Dialogflow, as explained in Chapter 2.

We will proceed with the weather intent that was configured in the previous section where we will integrate Dialogflow with the Weather API using fulfillment via a webhook service. For this book, we have created a webhook service in the Python programming language with a sample request and response JSON.

Here is the input JSON:

{

  "responseId": "12c5e0ed-7ec3-4ce6-938e-ce1a5fa5ceaf-7e4f1f27",

  "queryResult": {

    "queryText": "show me weather in London",

    "parameters": {

      "geo-city": "London"

    },

    "allRequiredParamsPresent": true,

    "intent": {

      "name": "projects/weather-hvucqi/agent/intents/9965f638-213e-4f1d-82a0-9928013f707c",

      "displayName": "Weather"

    },

    "intentDetectionConfidence": 1,

    "languageCode": "en"

  },

  "originalDetectIntentRequest": {

    "payload": {}

  },

  "session": "projects/weather-hvucqi/agent/sessions/67c6b15b-6cbc-a495-8bdd-50bd023f0eb7"

}

Here is the output JSON:

{

    "fulfillmentText": "Weather Report of London --> Current Temperature : 11.82, Pressure : 1003, Humidity : 93, temp_min : 10.56, temp_max : 13.33",

    "fulfillmentMessages": [

        {

            "text": {

                "text": [

                    "Weather Report of London --> Current Temperature : 11.82, Pressure : 1003, Humidity : 93, temp_min : 10.56, temp_max : 13.33"

                ]

            }

        }

    ],

    "source": "weather-webhook"

}

Next, we will add one mandatory parameter to the weather intent (i.e., geo-city), which will store entries of entity @sys.geo-city to value @geo-city if that is being recognized from the user input. Since this is a mandatory parameter, it will keep asking prompt questions to the user until this parameter has been provided (slot filling), as shown in Figure 3-73.

Now, go to the Fulfillment section under the weather intent and click the drop-down menu. Toggle the “Enable webhook call for this intent” option, as shown in Figure 3-74. This will enable webhook calling for this intent. Here, your web service will be called as soon as the weather intent has been recognized and all slots have been filled.

Go to the top-left menu and click Fulfillment. This will open a new page where either you can call your webhook service or use an inline editor to call the Weather API. For this book, we will call the webhook service. Click the toggle button on the right side of the Webhook label, as shown in Figure 3-75.

We have used ngrok to enable our webhook service to be accessible publicly. Once enabled, provide the URL, username, and password if required. For now, our webhook service doesn’t require any username and password, but all communication with the webhook service is encrypted. Click the Save button to configure fulfillment for this webhook service, as shown in Figure 3-76.

Now, go to the left menu and click the Integrations tab. Enable Web Demo integration and open the Web Demo URL in a browser of your choice. Next, the user can start a conversation with our Travel agent, as shown in Figure 3-77.

After classification of the user’s query to the weather intent, it asks for a location as this is a mandatory parameter that needs to be provided. Once the intent has been recognized and the slot has been filled, then it will call your webhook service with these arguments. Your webhook service calls the Weather API to fetch the weather information for the selected city and sends a response to the webhook service, which provides the response in a Dialogflow-specific format to Dialogflow, as shown in Figure 3-78.

Conclusion

In this chapter, we discussed how to design a complex use case using the concepts of input and output contexts, follow-up intents, etc. This chapter also explained various use cases with multiple intents, enabling fulfillment, and using webhooks for integration with external APIs to further enrich the quality of the conversation in our agent. We also explained how prebuilt agents can be used to reduce the time required to configure an agent for out-of-the-box use cases.

Chatbot Personality

A chatbot’s personality can be defined as the human-like qualities it exhibits in its conversations with end users. When building a chatbot for an organization, the chatbot’s personality represents an important element; otherwise, the chatbot will converse in a robot-like fashion with no emotions. For personality, we can use traits such as being friendly, formal, helpful, or polite in conversations with end users.

There are various ways to implement chatbot personality. Some organizations build their own organization-specific personality, and others implement common personality questions.

For this book, we have developed our own personality module in the Python language. For this exercise, you can go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_4 to download the code.

Please refer to the Python code file corpus.py inside the model package. This Python code file has a Python class that implements various functions to train and use a personality model for a user’s query, as discussed in this chapter.

In this function, we are loading training data from the MongoDB database and storing it in a pickle file that will be used to recommend a relevant answer for a user’s query.

These NLP tasks and related code snippets are available in the Python code file nlpTask.py inside the utils package. The definitions of all of these functions are as follows:

This code snippet compares the processed form of the user’s query with questions available as the part of training data and then returns an answer to a similar question as a response to the query.

So far, we have discussed how to train the personality module and use the trained model to find an answer to a question. Next, we will see how this system can be released as a REST API that can be consumed by other applications and software.

We will expose the personality module as a local URL. Since Google Dialogflow requires the public URL of your webhook service, we have used Ngrok to generate a public URL corresponding to our local URL.

Let’s look at the steps to set up a personality webhook service on both Windows and Linux Server.

Windows Server

For Windows Server, Python 3.6.x and PIP need to be installed in your system before continuing.

Step 1: Install Flask-RESTful

Flask-RESTful is an extension of the microframework Flask for building REST APIs.

To install it, run the following command at the Windows command prompt, as shown in Figure 4-1:

pip install flask-restful

This will install the package and its dependencies.

Step 2: Build the REST API

A RESTful API uses HTTP requests to get and post data.

Look for the PersonalityService.py file, which has code to release the personality module as a REST API, as shown here:

import json

from flask import request

from flask_classful import FlaskView, route

from models.corpus import PersonalityModel

class EndPointCall(FlaskView):

    @route("/personality", methods=['POST'])

    def endpoint(self):

        personalityModel = PersonalityModel('testData')

        json_data = request.get_json(force=True)

        question = json_data["queryResult"]["queryText"]

        answer = \ personalityModel.getRecommendation("testData", question)

        output = {

            "fulfillmentText": answer,

            "fulfillmentMessages": [{"text": {

                "text": [

                    answer

                ]

            }}],

            "source": "personality-webhook"

        }

        return json.dumps(output)

Step 3: Deploy the Flask REST API

To deploy the REST API service using Flask, run the following command at the Window command prompt, as shown in Figure 4-2:

python PersonalityService.py

Step 4: Get a Response from the REST API

Now the service is being hosted at the following URL:

http://127.0.0.1:5000/getResponse

We can check this URL in a REST client (e.g. Postman) with some sample input JSON.

Since Google Dialogflow requires a public URL of your webhook service, we will use Ngrok to generate a public URL corresponding to the local URL that we configured earlier.

Follow these steps to generate a public URL using Ngrok:

1. 1.

   To expose a local HTTPS server, download Ngrok from https://ngrok.com/download.

    
2. 2.

   The public URL is available only when the auth token is downloaded from https://dashboard.ngrok.com after signing up at https://ngrok.com/signup.

    
3. 3.

   The auth token must be specified to Ngrok so that the client is tied to this account. Ngrok saves the auth token at ~/.ngrok2/ngrok.yml so that there is no need to repeat the previous steps.

    
4. 4.

   Unzip the downloaded Ngrok folder and run the ngrok.exe application.

    
5. 5.

   Copy the auth token from the user account to use in the command and run this command at the Ngrok terminal prompt, as shown in Figure 4-3:

   ngrok authtoken <AUTHTOKEN>
    

1. 6.
   After the previous step, the auth token gets saved to the configuration file, as shown in Figure 4-4.

   

    
2. 7.
   Ngrok is a command-line application, so use ngrok http https://<IP>:<PORT> at the command prompt to expose the HTTPS URL. Here the IP and port correspond to the personality API host and port on which the API is hosted, as shown in Figure 4-5.

   

    
3. 8.
   A new terminal will open after the execution of the command that will display the public URL https://44e2f215.ngrok.io corresponding to the local server URL, as shown in Figure 4-6.

   

    

Now this REST API will be integrated with our webhook service, which integrates the chatbot personality module with Dialogflow.

This deployed API service can be used for a development environment.

For a production environment, the API should be hosted on Apache Server. Refer to the following URL to deploy a service on Apache Server in Windows:

https://medium.com/@madumalt/flask-app-deployment-in-windows-apache-server-mod-wsgi-82e1cfeeb2ed

Linux Server

For Linux, Python 3.6.x and PIP need to be installed in your system before continuing.

Step 1: Install Flask-RESTful

To install Flash-RESTful, run the following command in the Linux shell, as shown in Figure 4-7:

$ pip install flask-restful

This will install the package and its dependencies.

Step 2: Build the REST API

Create a PersonalityService.py file that will have the personality code that you downloaded from GitHub.

Step 3: Deploy the Flask REST API

To deploy the REST API service using Flask, run the following command in the Linux shell, as shown in Figure 4-8:

$ python PersonalityService.py

Step 4: Get a Response from the REST API

Now, the service has been hosted at the following URL:

http://127.0.0.1:5000/getResponse

We can check this URL on a REST client (e.g. Postman) with some sample input JSON that we are going to explain in this section.

Let’s see the steps to generate a public URL using Ngrok. The personality API will be hosted on a port on the server or a system via the Apache Server.

1. 1.

   To expose a local HTTPS server, download Ngrok from https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip for the Linux server.

    
2. 2.

   The public URL is available only when the auth token is downloaded from https://dashboard.ngrok.com after signing up at https://ngrok.com/signup.

    
3. 3.

   The auth token must be specified to Ngrok so that the client is tied to this account. Ngrok saves the auth token in ~/.ngrok2/ngrok.yml so that there is no need to repeat this step.

    
4. 4.

   Unzip the downloaded Ngrok from a terminal with the following command, as shown in Figure 4-9:

    

$ unzip /path/to/ngrok.zip

1. 5.

   Copy the auth token from the user account to the command and run this command at the Ngrok terminal prompt, as shown in Figure 4-10:

   ngrok authtoken <AUTHTOKEN>
    

1. 6.

   After the previous step, the auth token gets saved to the configuration file.

    
2. 7.
   Ngrok is a command-line application, so use ngrok http https://<IP>:<PORT> at the terminal prompt to expose the HTTPS URL. Here the IP and port correspond to the personality API host and port on which the API is hosted, as shown in Figure 4-11.

   

    
3. 8.
   After the execution of the command, the terminal will display the public URL https://44e2f215.ngrok.io corresponding to the local server URL, as shown in Figure 4-12.

   

    

For more details, please refer to the Ngrok documentation at https://ngrok.com/docs.

This deployed API service is feasible for a development environment.

For a production environment, the API should be hosted on Apache Server. Refer to the following URL to deploy a service on Apache Server in Linux:

https://www.codementor.io/abhishake/minimal-apache-configuration-for-deploying-a-flask-app-ubuntu-18-04-phu50a7ft

Personality Intent

Now, let’s see the steps to create a Personality intent.

1. 1.

   Create an intent named Personality in our Travel agent.

    
2. 2.

   Define these training phrases, as shown in Figure 4-13:

   What is your name?

   How are you?

   What are you doing?

   What is your favorite color?

    

Here, we have defined some personality questions in the Personality intent so that it can be passed on to our webhook service, which in turn makes a call to our chatbot personality service for an answer.

Click the toggle button for “Enable webhook call for this intent” for the Fulfillment field of the Personality intent page, as shown in Figure 4-14.

This enables a webhook call for the Personality intent. Now, go to the left sidebar, click Fulfillment, and then enable the toggle button next to Webhook. Enter the webhook service public URL for the chatbot personality that we configured earlier and click the Save button, as shown in Figure 4-15.

So, our chatbot personality service via the webhook service using fulfillment has been integrated with Google Dialogflow.

Now, go to the left sidebar menu and click Integrations. Enable the toggle button for Web Demo and open the URL in a browser of your choice.

In Figure 4-16, the user has asked the following personality questions:

• What are you doing?

• What is your favorite color?

The system responds with the relevant answer from our webhook service that is integrated with chatbot personality service.

Please note that we can integrate the personality webhook service into any personality service provided by other providers, but we need to ensure that the webhook service processes the input and output JSON in the same format as mentioned earlier.

Sentiment Analysis

Sentiment analysis can be defined as the process of identifying the user’s opinion from text. Sentiment analysis plays an important role in responding appropriately to the user. It classifies text into three categories (positive, negative, and neutral) based on the user’s opinion of their experience with the services offered. Once we have done that, we can tailor the responses of our chatbot to reflect the sentiment of the user. As an example, if the user is highly negative about something, we can begin by reassuring them or showing concern in our response.

Several NLP providers such as IBM Watson, LUIS, and Google have the capability to perform sentiment analysis on text. Google Dialogflow also has the ability to identify sentiments of the user, but this is applicable only for Enterprise Edition, as shown in Figure 4-17.

Here, we are going to show a sample sentiment analysis system that returns the sentiment of the user (i.e., positive, negative, or neutral). This has been exposed as a REST API. You can download the sample sentiment analysis code that has the use case configured for sentiment analysis.

Go to https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_4 to download the Sentiment_Analysis_code.txt file.

For this book, we have used the Deeppavlov library for the implementation of the sentiment analysis system. This library implements the Google BERT model to train the sentiment analysis system. The Deeppavlov library uses training data in CSV format, where each data point is a pair of sentences and their corresponding opinions (i.e., positive or negative).

For inference, the system processes the user’s query and provides the sentiment (i.e., positive or negative) as a response to a query.

We have used the following sample request JSON that is supplied on Dialogflow’s end and the response JSON that is defined inside the code for our webhook service (REST API) for sentiment analysis.

Now, we will explain the complete codebase present in the downloaded file named Sentiment_Analysis_code.txt.

The value of one argument of this function (i.e., download) is False. This is required if we don’t want the model to be downloaded every time we make a function call to get an instance of the sentiment model.

You can also use natural language services from any of the cloud vendors such as Google’s Cloud Natural Language API (https://cloud.google.com/natural-language/docs/analyzing-sentiment) for sentiment analysis. Since Google Dialogflow requires a public URL of our webhook service, we have used Ngrok to generate a public URL corresponding to our local URL, as explained in the previous chatbot personality use case.

The sentiment analysis webhook service integrates with this system to get an opinion and send it to Dialogflow using fulfillment.

Now, to configure the agent, you can either download the agent file (Sentiment_analysis.zip) that has the sentiment analysis use case by accessing the GitHub URL (https://github.com/dialogflow34/Chat-Bot/tree/master/Chapter_4) or follow the next steps to configure the use case from scratch.

Import the downloaded agent into Google Dialogflow, as explained in Chapter 2, if you are working with downloaded agent file.

Now, let’s see the steps to create a sentiment intent.

The Sentiment intent will be added to our Travel agent. It will be used during the feedback stage in our Travel agent. Enable the toggle button “Enable webhook call for this intent,” as explained in the earlier personality use case. This enables a webhook call to sentiment analysis. So, whenever the user provides feedback about services, Dialogflow first categorizes the feedback text to the Sentiment intent and then calls the sentiment analysis webhook service, which calls the API to the sentiment analysis system to get an opinion from the feedback text.

• In the left sidebar menu, click Fulfillment and then toggle the button next to Webhook.

• Enter the webhook service URL for sentiment analysis and click the Save button, as shown in Figure 4-18.

So, our sample sentiment analysis system via the webhook service using fulfillment has been integrated with Google Dialogflow. Now, go to the left sidebar menu from the sentiment intent page and click Integrations. Enable the toggle button for Web Demo and open the URL in a browser of your choice.

In Figure 4-19, the user has provided feedback (e.g., I really liked hotel options provided by you) for hotel options offered by the chatbot. Our travel agent classifies this text into sentiment intent and calls the corresponding sentiment analysis webhook service to get the response text.

Please note that we can integrate the sentiment analysis webhook service with any sentiment analysis service, but we need to ensure that the webhook service will process the input and output JSON in the same format as mentioned earlier.

Event Configuration: Chatbot Transfer

In the previous section, we integrated our Travel agent with sentiment analysis to identify the sentiments of a user from the feedback text. Sentiment identification has another use case as well. Specifically, output from sentiment analysis can be used to transfer a conversation to a human agent if the customer is not satisfied with the responses of our agent.

It also offers flexibility to change responses, to call any third-party APIs, etc. All this can be accomplished using events in Dialogflow.

Events in Google Dialogflow are used to trigger an intent not from the end-user expressions but based on the feedback response. We can leverage the same feature for this use case to trigger an opinion-specific intent.

Now, we need to create two intents: one for positive opinions (Issue_resolved) and one for negative opinions (Transfer_to_agent) in our Travel agent.

The user will give feedback to the Travel agent, which will trigger the sentiment analysis intent and call the sentiment analysis webhook service. Then the webhook service processes the input request and calls the sentiment analysis system to extract an opinion from the feedback text, as discussed in previous section and as shown in Figure 4-20.

Now, based on the response from the sentiment analysis system that is either positive or negative, the webhook service will set an event. For example, if the response is positive, then it will set an event to issue-resolved; and for a negative response, it will set a transfer-to-agent event. Once an event has been set, the webhook service generates the JSON response and sends it to Dialogflow via fulfillment.

Fulfillment triggers an intent, either Issue_resolved or Transfer_to_agent, based on the event received from the webhook service and displays a response as defined in the responses section of an intent, either Issue_resolved or Transfer_to_agent.

1. 1.

   Create an intent with the name Issue_resolved. Then, go to the Events section from the Issue_resolved intent page.

    
2. 2.

   Define the event name as issue-resolved, as shown in Figure 4-21.

    

Configure responses for the intent Issue_resolved, as shown in Figure 4-22.

Do the same steps for the negative opinion intent (i.e., Transfer_to_agent), as explained earlier. Define an event named transfer-to-agent, as shown in Figure 4-23.

Configure responses for the intent named Transfer_to_agent, as shown in Figure 4-24.

Here we will be using our existing sentiment analysis webhook service with a small change in the response JSON. Now, the webhook service will send information about events that will trigger an associated intent. The sample request and response JSON is as shown next. As you can see, the webhook service responds with the event name that will be used.

Now, go to the left sidebar menu from the agent page and click Integrations. Enable the toggle button for Web Demo and open the URL in a browser of your choice.

In Figure 4-25, an end user has provided feedback on his experience with our agent. As per the response, user is quite disappointed with the service, so he provided the feedback text Thanks for your service. But I didn’t like cab options. Our Travel agent classifies this response to the Sentiment intent and calls the sentiment analysis webhook service. The sentiment analysis webhook service processes the input request and calls the sentiment analysis system to extract the sentiment from the feedback text, as discussed in the previous section.

Integration with Facebook Messenger

In this section, we will go through the steps to integrate Google Dialogflow with third-party messengers such as Facebook and Slack. This example will help the end user to converse with our agent via Facebook Messenger.

Display Name: Dialog_flow

Google Dialogflow’s Knowledge Base Integration

In this section, we will implement a document search engine feature in Google Dialogflow.

It is not feasible to define intents and their training phrases for all types of use cases. It works better if we want to perform a specific action, say, fetching weather information for a particular city, finding available hotel accommodations, etc. But for other use cases, say, places to visit anywhere in the United States, it is not possible to define intents for such types of queries. These types of query generally fall into the category of long-tail queries where the answers are within relevant documents.

Google Dialogflow can be integrated with any document search engine service provided by various NLP providers using fulfillment via the webhook service.

Google Dialogflow also has a built-in connector to build a knowledge base where we can upload documents either manually or by specifying a location in cloud storage where the documents exist. The knowledge connector handles the user requests with answers extracted from the uploaded documents. In Google Dialogflow, this is a beta feature, which means it can’t be used in a production system. By the time you are reading this book, however, it is possible that the system will be out of beta and ready for use.

In the next sections, we will focus on Google Dialogflow’s built-in knowledge connector to build a search engine and integrate it with our Travel agent. Since Google Dialogflow’s knowledge base is in beta, we need to enable beta features for our agent.

Enable Beta Features

To enable beta features, please follow these steps:

1. 1.

   Go to the Dialogflow console and select the Travel agent.

    
2. 2.

   Click the settings button next to the agent name, i.e., Travel. Go to the General tab and click the toggle button for “Enable beta features and APIs.” Click the Save button to save the changes.

    

Now, the Google knowledge base plugin has been enabled for our Travel agent, as shown in Figure 4-44.

Create a Knowledge Base

To create a knowledge base, go to the left sidebar menu and click the Knowledge tab.

This will open the Knowledge Bases configuration page, as shown in Figure 4-45.

Follow these steps:

1. 1.

   Click the CREATE KNOWLEDGE BASE button.

    
2. 2.

   Fill in the knowledge base name (i.e., Travel-UseCase).

    
3. 3.

   Click the SAVE button, as shown in Figure 4-46.

    
4. 4.
   Click the “Create the first one” link to upload documents.

   

    
5. 5.

   Enter Places to visit in UK for Document Name, set Knowledge Type to Extractive Question Answering, set the MIME type to application / pdf.

    
6. 6.

   To upload the documents, choose any option: File on Cloud Storage, URL, or Upload file from your system. Here we will show how to upload a document from our system.

    
7. 7.

   Click the SELECT FILE button and select the files from the system that you want upload to the knowledge base.

    
8. 8.

   Click the CREATE button to finish the configuration, as shown in Figure 4-47.

    

The uploaded documents will be shown as part of your knowledge base, as shown in Figure 4-48.

Now, go to the Responses section of the knowledge base and define a response as $Knowledge.Answer[1] in the Text Response box, as shown in Figure 4-49.

Now, if the user asks a question for which no relevant intents exist in our agent, the chatbot will extract the relevant results from our knowledge base.

Next, go to the top-left menu and click the Integrations tab. Toggle the button for Web Demo and open the link in a browser of your choice. Next, enter the query What are the different places to visit in UK. You can see our agent has returned results from the knowledge base and not from the intents, as shown in Figure 4-50.

Spell Correction

A spell correction feature plays an important role in a chatbot system. This enables a chatbot system to be smart enough to capture spelling mistakes in user queries and responses.

Google Dialogflow provides a built-in spell correction feature. Please note that if there are any spelling mistakes in entity values, they should be corrected by the entity fuzzy matching feature first, before passing the values on to spell correction.

We can also integrate our chatbot with a spelling correction service provided by any other NLP provider using fulfillment via a webhook service that integrates with the spell-check service.

We will start by demonstrating our existing Travel agent without the spell correction feature enabled, where the agent is not able to recognize spelling mistakes.

Go to the “Try it now” panel and enter the query I want to trvl from Los Angeles to Sna Francisco. Now, the agent is not able to recognize the correct intent because of the misspelled data, so it goes to the fallback intent, as shown in Figure 4-51.

To enable spelling correction, click the settings button next to the agent name (i.e., Travel).

Go to the ML Settings tab and click the toggle button under AUTOMATIC SPELL CORRECTION. Then, click the SAVE button, as shown in Figure 4-52.

Now, enter the same misspelled query to see whether the spell correction feature is working or not, as shown in Figure 4-53.

Dialogflow will convert the misspelled query to the correct spelling during request processing and perform intent classification. Now, the correct intent is triggered, and the response comes with that intent, as shown in Figure 4-54.

Conclusion

In this chapter, we discussed the importance and implementation of chatbot personality. We looked at different scenarios for sentiment analysis such as transfer to a live agent. We also discussed the integration of Google Dialogflow with third-party messengers such as Facebook Messenger. Also, we implemented a document search feature using the Google Dialogflow knowledge base. Finally, we implementing spelling correction in our chatbot.

Current Research

Increasing demand for chatbot development in various domains, such as e-commerce, healthcare, online booking services, etc., has encouraged new research into some exciting features of chatbot and conversation systems.

These features include generating responses in natural forms, designing an end-to-end conversation flow from an existing conversation’s data, generating a response from a bot based on a user’s personality, etc.

Nowadays, chatbots with the capability to generate natural language responses are expected to make conversations more intuitive or human-like. Several patents and research papers exist on this feature, but this is difficult to implement in practice because of the different types of data in existence such as text, audio, images, video, etc. It is also difficult because of a lack of sufficient training data for various domains such as healthcare, insurance, etc. One of the patents uses the concept of how an individual communicates with the chatbot system and learns structure from those conversations to generate responses. Therefore, you may see different response text for the same query from different users, but all responses will have the same meaning.

The current implementations of cognitive virtual chatbots (CVAs) lack the capability to generate an end-to-end conversation flow from chat logs and to automatically generate dialogue. This is the most important research area, and most of the big industry players are actively working on the topic. One of the patents in this area implements this by analyzing conversation flows to understand the meaning of each line of text and by connecting each distinct response from both sides to finally generate conversation flows. The same approach can be used in spoken dialogue systems to automatically generate dialogue.

In current CVA systems, goal-oriented dialogue agents are generally trained to complete a specific goal. For any other new task, you may not find a dataset consisting of human-to-chatbot interactions. In such cases, a dialogue developer will need to develop this from scratch for all possible interactions. This is a time-consuming process and requires continuous support from developers. This problem has been well studied by Google, and its AI track has introduced a new framework known as machine-to-machine (M2M) that automates the generation of an end-to-end dialogue agent for goal-oriented systems. As an example, CVA developers now can use an existing dialogue agent to generate a new dialogue agent for tasks such as booking an online appointment.

Research Acquisitions

A lot of research in the AI field for chatbots is being led by various startups, and some of these good ideas have been acquired by industry majors. Deepmind, the creators of AlphaGo, was acquired by Google in January 2014, for $500 million. Some of its cutting-edge technologies have now been integrated into Google Assistant. Another example is that Siri was acquired by Apple in 2010 to enable virtual assistant capabilities in iPhones. Further examples include Wit.ai, which was acquired by Facebook, and Maluuba, which was acquired by Microsoft, to expand their AI ecosystems.

Research in the area of human conversations is an exciting and active field where every day new discoveries are being made and new technologies are being implemented to make cognitive virtual agents better. The day is not far that we will not be able to distinguish a CVA from a human being.