Getting started with LUIS
If I’ve learned anything over the last couple of years of working with developers creating chat bots it’s that there’s a lot of confusion about natural language processing (NLP). The confusion ranges from when to use it, how to implement a model, the best way to design the model, and finally improve the model.
I can certainly understand the confusion because, when we get right down to it, the problem space itself can be confusing. After all, NLP in general is the stuff of science fiction. This is Scotty picking up the mouse, saying “Hello, computer”, and having the computer actually respond. As developers, we typically don’t have a lot of experience in any form of machine learning or artificial intelligence, into which NLP squarely falls. It’s hard to get started when we don’t necessarily have a frame of reference for the underlying problem space.
Fortunately, services like Language Understanding Service (LUIS) can help us get started with setting up a model for our bots, and bring NLP to our users. Even still, there are some concepts that need to be learned, and that’s where I want to start with this blog post.
I want to introduce the concepts of intents and entities inside of LUIS, and talk about the different types of entities. I’m doing this without working through demonstrations or providing in depth samples, as I will be doing that over the next couple of blog posts. But before we can get there, there’s a few core concepts we need to cover, and some terminology to define, in order for us to venture into deeper topics.
With that, you’ll notice I’m not going to include screenshots or specific examples in this blog post. I suggest you open luis.ai and follow along from there. Or, you can skim this post, check out the later ones, and then return to this page to use as a reference.
Let’s start at the high level terminology wise with utterance. An utterance is a block of text sent to LUIS for parsing and classification. This might be a couple of sentences, a single sentence, or a phrase.
When you send an utterance to LUIS, it will break down the text, and identify the intent or intents it it believes the utterance matches, and also the entities it discovers for the top matching intent. LUIS will return a 0-1 based score for each intent it believes might match the utterance, with the highest number being the most likely correct match.
Training a LUIS model involves creating intents and giving LUIS examples of utterances that match those intents. Additionally, you identify the entities in each utterance under each intent. By training LUIS, it’s able to detect intents and entities on the text you send to it.
The word intent, according to dictionary.com, means purpose or what someone is trying to do. When creating intents in LUIS, that’s exactly what we’re trying to establish - what action is the user attempting to perform? Are they looking to book a flight? Send an email? File a trouble ticket? All of these are intents.
I like to think of intents as classification buckets when putting together my LUIS models. They allow me to group together a set of related actions. Let’s take filing a trouble ticket as an example. When creating an intent, that action of file a trouble ticket is the perfect grouping. I want to be careful about about getting too specific in designing the group, as I can pick up additional information via entities (which I’ll get to in a moment), via code, and additional follow-up questions to the user.
As we will see later, care must be taken to not create intents that are too narrow. Not only can this be tough to train for LUIS, and harder for LUIS to properly identify which intent to assign to an utterance, it can also make it more difficult to maintain. In addition, you will want to ensure the intents you add to your model are related to one domain (say travel, or help desk), rather than trying to answer every possible question a user might have. We’re not trying to solve the Turing test.
There is one special intent - the none intent. The none intent is designed to be the catch-all for anything that doesn’t fit. LUIS is designed to be a closed universe, meaning it will always match utterances to an intent. This can lead to unexpected results when users send unexpected utterances. None is designed to be the catch-all. If it doesn’t fit in any of the intents we’ve created, it’ll put it into none. In order for this to work, the none intent must also be trained. You’ll want to add utterances to the none intent that include either garbage text, or values that might make up a part of an intent (say the name of a city, or a type of product) but don’t actually match the full intent. If someone simply tells your bot San Diego, that doesn’t indicate they’re looking to book a flight, hotel, or anything. That type of value is perfect for none.
Going back to dictionary.com, an entity is a thing. I sort of like that definition when it comes to LUIS, as that is exactly what we’re looking for - things. We’re looking for anything which provides additional information about what the user is trying to do, such as context, options, or other supporting material. This might be the type of trouble ticket the user is looking to file, the name of the piece of hardware giving them issues, or a description of the issue.
As you can see, entities can cover a wide gamut of data types. Fortunately, LUIS provides several ways to manage and work with these various pieces of data via different types of entities. I want to introduce the options available to you when creating a LUIS model. There will be a couple (such as hierarchical and composite) which will require a bit more detail; my intent is to provide more information in future blog posts. (see what i did there - using the word intent. i crack myself up)
There are three entity types - simple, list and regex - which are the most commonly used varieties. They’re also the ones most people think of when they think of setting up a form of model for handling NLP.
This is typically the type of entity you’re looking for when designing a LUIS model. A simple entity is one that is machine learned, and based on context. The more examples of sentences sent from users to LUIS and training the model receives, the better LUIS will become at picking out the appropriate portions of a sentence as the entity type. Keeping with our trouble ticket example, a simple entity could be used to pick out the hardware type, such as in “My [printer] is giving me an error message”, or “My [camera] doesn’t work”.
Sometimes called a closed list, the list entity allows you to provide a list of possible options to LUIS for the entity. When creating a list entity, you specify the label, or “normalized value”, and then synonyms for that value. You might create a list entity for departments and normalize various abbreviations. For example, you might create an entry in your department list entity type called Human resources, and add HR as a possible value.
It’s extremely important to note two things about list entities:
- They do not perform any form of machine learning or other fuzzy matching. Only the values you entered into the list entity will match for that entity. New values must be entered manually (not machine learned), and abbreviations or other shortcuts must also be a part of the entity.
- They always match, regardless of the intent. What this will mean is as you are creating and configuring your entities, you will not have to specify your list entities matching values as that will happen automatically. It also means you aren’t able to tie the list entity to a value that doesn’t match because, as we mentioned above, your list entity is a closed list, and only values you specify will cause it to match.
Because list entities do not offer any type of machine learning, but are rather static closed lists, they’re best for small sets of keywords that you need to be on the lookout for when parsing user input. List entities are also not good for a large list of possible values (like cities or product names), as there will likely be regular updates that would need to take place, or numerous spelling mistakes; in these cases it’s best to combine LUIS with some form of search or full text indexing.
As you might suspect, a regex entity is an entity built from a regular expression. This allows you to add entities for pieces of information that have a particular pattern, such as employee numbers, record locators, or stock-keeping unit (SKUs). Similar to list entities, regex entities always match, and perform no machine learning.
The next two entity types build upon either the simple entity, or on other entities you add to your model. They allow you to add some additional meaning or metadata to the entities LUIS discovers in an utterance.
Hierarchical is built on top of the simple entity. It’s machine learned, and improves over time as LUIS receives more training and information about your domain. Unlike a normal simple entity, hierarchical allows you to add context or roles to the values it finds.
For example, if we were creating a model for booking travel, we might have an intent allowing people to book a flight. When booking a flight, we’d be looking for both a starting location and a destination location. At a high level, the expected entity values are the same - they would both be cities. But in order to parse it properly in our application I need to know which city is the starting point, and which is the destination. This is hierarchical entities come into play.
By using a hierarchical entity, I can create an entity named location, and then create two children of start and destination. LUIS knows at a high level the expected values would be logically the same, and that the context would provide the difference. It would then learn when someone types “I want to go from San Diego to San Francisco” that San Diego would be the starting point, and San Francisco the destination.
Composite entities are used to combine entities you have already added to your model into one encompassing entity. Composite entities are used when multiple entities used together are actually a related collection of values. If someone typed “I want to fly from San Diego to San Francisco, leaving on 1/5 and returning on 1/10”, all of that information is really indicating one collection of data. We have the source (San Diego), destination (San Francisco), departure date (1/5), and return date (1/10). A composite entity can roll those into one object for us, making it easier to work with inside of our application.
Composite entities are also useful when someone might send multiple instances of a collection of data. For a hardware ordering system, someone might tell our bot “I want to order one laptop and two monitors”. In that instance we want to ensure that one laptop is broken down into the quantity and value as one entity, and two monitors is broken down as well. Composite entities do exactly that for us.
Built-in entities come in two different varieties - prebuilt and prebuilt domain. While the names may be similar, the domains behave very differently.
Prebuilt entities are entities designed for common data types such as numbers or dates. The basic concept is many people will need to add entities to manage these types of information, so why not build them in for everyone to use. By creating these in-house (meaning, inside Microsoft), additional functionaly can be added, and values can be normalized. When you use the number prebuilt type, for example, LUIS will be able to pick up the word two, and normalize it to the value 2 automatically for us, making development simpler.
Like list entities, prebuilt entities are not machine learned. You’ll also notice prebuilt entities always match to appropriate values, and are not added to specific entities.
And unlike every other entity type, prebuilt entities can not be modified. You add them to your model, and that’s it.
Prebuilt domain entities can be thought of starting points or samples. Prebuilt domain entities are simple entities, just like ones we would create. The only difference is they already come with some training data. You are free to modify this training data as you see fit, adding and removing values to help fine tune it for your scenario.
(To be honest with you, I personally never use prebuilt domain entities)
I specifically buried this one at the bottom of the list as it’s a special entity used with patterns. In a nutshell, it’s designed to be a placeholder, or a blank space, where whatever is between certain keywords is returned as the entity value, regardless of anything else. It’s only used with patterns, which is a way to improve the accuracy of intents. We’ll take a look at patterns in a later post.
An utterance is a block of text (phrase, sentence or sentences) sent to LUIS for parsing. LUIS will identify the intent (or intents) the utterance matches, assigning a score for each matching intent. It will also parse the sentence looking for entities in the utterance. Entities come in many varieties, from those that are machine learned (simple), to those where you have no control but provide helpful services (prebuilt).
As I mentioned when we started, my goal in pecking out this blog post was to provide an overview of the moving parts in a typical LUIS model. I did this partly because I see a lot of confusion around the various entity types, and as a lead-in to future blog posts.