Managing versions and timeouts in Bot Framework
Creating bots in the real world brings additional challenges with versioning and user behavior. Obviously, when you update your bot, you want to make sure you seemlessly migrate users over to the new bot, often allowing them to complete the conversation they’re currently having before showing them the new bot. Also, because Bot Framework does not time out users, you can also run into a situation where a user has left for a sizeable amount of time, and either state has changed or you simply want to offer the user the chance to start over.
Let’s break this down by first talking about how things work behind the scenes inside of Bot Framework, introduce middleware, and then put forth a couple of patterns you can implement to handle these situations.
Well, one quick note before we get started - this is a 300 level concept, so I’m making the assumption you’re already familiar with Bot Framework, and have created a few bots. If not, you can check out the Bot Framework MVA.
A bot is, at the end of the day, simply a web service. Web services, of course, are stateless, meaning that after the message is sent from the user, and a reply returned, the connection is broken down, and all data is lost. When working with a bot, however, this poses a problem, because an operation will typically take place across multiple round trips from the user to the bot.
To handle this, Bot Framework “dehydrates” or serializes the current state of the bot for each user. When the user returns by sending a new message, the user state is “rehydrated” or reloaded, the bot determines where the user left off, and the next function is then run. You can see this happening behind the scenes when tracking your bot in the console, seeing the waterfall in action, and which step the bot is about to call. In a simple “Hello, world” bot, the user would send hello, the bot sends the prompt for the name, the user replies, and then the bot runs the next function - in this case, sending the hello message.
Because we’re dealing with humans, who may take some time to respond back to the bot, there is no timeout for a session. If a user leaves for a couple of hours, and then replies to the What is your name? question, the bot simply picks up where it left off, as if the user replied instantly.
The first challenge here is state may have modified - the price for an item may have changed, a reservation time may no longer be available, or the operation may simply no longer be valid. In addition, this might confuse the user in general - if you didn’t talk to a bot for a couple of days, and potentially had the chat history on your client cleared, you might not expect the bot to pick up, or even know, where you left off.
The second challenge is when updating the bot. If you modified a dialog, the bot might not be able to pick up where it left off seamlessly. You may have removed a step from a waterfall, meaning the bot will error, now knowing what to run. Or, if a step changed, it may ask the user a question that simply doesn’t make sense for the current flow.
Addressing these issues doesn’isn’t necessarily difficult, but it does take a little bit of forethought and planning. With a little bit of middleware, you can transition as needed.
Middleware allows you to intercept messages before they’re passed into the dialog flow, where the bot processes them as normal. This is useful for logging, rerouting messages (say for bot to human handoff), or, in our case, managing how a user is directed through the dialogs offered by the bot.
To implement middleware, you call
use, passing in an object that implements at least one of three methods:
receive, which fires when a event is received by the bot
send, which fires when an event is sent by the bot
botbuilder, which executes after
receiveand the message is bound to a session
The last part of
botbuilder is key in our case, because it’s going to allow us to open up where the user currently sits in the flow of the dialog(s), and determine the next step.
Let’s walk through a few strategies for managing timeouts and updating your bot. I’m not going to walk through every possibility, but rather show off a few representative solutions that will hopefully illustrate how to approach the problem, and provide enough information for you to implement what’s needed.
As mentioned above, a bot doesn’t timeout. As a result, if it’s been even a few days since the user last replied, the bot is going to attempt to pick up where it left off. This can either confuse the user, or operate on bad information, neither of which is a good thing.
The first potential solution, and the easiest to implement, is to simply reset the conversation. We’ll do this by adding a
lastAccess property to
userData. We’ll then query this to see how long it’s been since the user last talked to the bot, and, if necessary, reset the conversation.
We determine if the user is currently in a conversation by taking a look at the
dialogStack and determining if there is at least one dialog there. Our threshold is 10 seconds, which is obviously artificially small, but I want you to be able to easily test the implementation.
The next possibility, and slightly more advanced, is to ask the user what they want to do. We’ll use the same timestamp technique as above, but this time we’ll route the user to a confirmation dialog. If they say yes, we’ll end the dialog, which will reprompt the user. If they say no, we’ll reset the dialog like before. Again, the threshold is artificially small for testing purposes.
It’s really up to you to decide how to best manage timeouts and what works best for your bot. You might decide to time the user out after twenty minutes, or maybe a couple of hours. You might automatially reset, especially in simpler two or three turn dialogs, or to prompt the user for more complex bots. You might even go a couple steps further, detecting which dialog the user is in, and make the decision based on that.
Versioning a live bot is a little trickier. You want to get users onto the new bot as quickly as possible, but you probably don’t want to interrupt the current flow of a user already in the middle of a dialog. If nothing else, when you version your bot, you need to handle the fact that a user in the middle of a dialog might be returning to a dialog step that doesn’t exist, or a dialog that’s changed profoundly, meaning the message the user sent might not make sense in the flow you deployed.
If you do want to transition the user, you will need to pay a little bit of attention to how you update your bot. In theory, you could do this on a dialog by dialog basis, but the code becomes trickier as you need to figure out where the user is, and swap out the dialog. I find it much easier to simply swap out the entire bot. To help manage this, I’ll put the bot, and its associated dialogs, into a folder. When it comes time to update it, I’ll copy the entire folder structure, update the copy, and then update
app.js to call the updated (new) bot.
When you need to send a user to the old bot, you simply update the
library property of the
session, which sets the bot that will run for the incoming message. In the code snippet below, I have both an updated and original bot in the same file for simplicity’s sake. I hope you can see from here how you would implement this with multiple folders.
In the sample below, I’m going to transition the user once their current conversation completes. You could, if you so desire, update this code to always reset the conversation, with a message of course, which would cause the user to be rerouted to the new bot. Also, the code could be streamlined a little bit, but breaking it apart like I’ve done makes it (I think) a little easier to explain with comments.
If you wanted to test this, start by making the new bot version 1 and sending a message to get yourself into the dialog. Then update the version to 2, rerun the application while not restarting the conversation in Bot Emulator, and sending the second message. You should see the original bot message.
Versioning is tricky, especially if you’re in a CI/CD environment where you expect to make multiple updates to the bot. Bot Framework (unfortunately?) doesn’t automatically handle transitions for you, so you will need to decide what pattern will work best for you. This is also important when developing and testing your bot in a channel such as Skype or Slack, where how to reset the state from the client isn’t overly obvious. In dev you could certainly skip the transition, but in production you’ll likely want to implement a mechanism for doing so.
Once you introduce your bot to the “real world”, things change. Management of the experience becomes much more important, and dealing with timeouts and versioning allows you to ensure a good experience for your users. This is something you should thing about early on, as it’s easier to introduce this at the beginning than to an existing, complex bot.
There are numerous ways you could handle this. Hopefully I’ve provided enough of a foundation for you to create an implementation that’s right for your bot.
 I am referring to session in general, not to the
session object in Bot Framework.