Last week I attended a MeetUp about Conversational UI.  Ofer Ronen of Chatbase had some interesting slides on how to catalog errors for chatbots.   And I’d argue that this way of categorizing issues can make your team more efficient as they work to improve your chatbots.  When people do context switching it takes them time to transition from one area to another.  I certainly remember it during my days at GarageCity.  I learned real quick to group tasks that used the same technology into the same half day.  There were nuances between Perl, JavaScript, shell scripts and SQL.  If I bounced back and forth between them I lost time and made more mistakes.  This context switching make you less efficient and more prone to errors.  And it is something that can be applied when improving your chatbots.

If you aren’t aware, UMM is an acronym which stands for

• Unsupported requests
• Misunderstood requests
• Missed requests

I would suggest that understanding these, categorizing accordingly and breaking up either time or teams into work tasks based on the classifications will make the team more efficient.

Unsupported

Unsupported requests are those where the chat bot does not yet have the logic encoded in it to deal with certain requests.  Typically they end up mapping to a Fallback Intent..  There is a certain type of discipline that goes with resolving these issues.  There might be larger projects where this functionality is coming, or dependencies that need to be solved.  Unsupported requests need to be grouped and prioritized by how many inquiries they will help to answer or how much they will improve customer satisfaction.  It is also important to look at any parts of the conversation where they could lead to ambiguity or being Misunderstood.  There is a whole methodology that is behind adding support for new requests.

Misunderstood

On the other hand Misunderstood requests are ones where either the Intent was mislabeled or where some context was missed in the conversation.  So a user might appear be asking about returning a product.  But instead the bot incorrectly points them to the warranty page without context as to why.  It could have been as simple as a logic error pointing to the wrong intent.  Or there could have been context in the conversation that was missed in the logic.  Perhaps another question needs to be inserted into the logic.  Either way, this is a very different process than adding new responses for an Unsupported request.

Missed

The last M is for Missed requests.  Those happen to be where the response already exists, but the request wasn’t matched to it.  Instead it was routed to the Unsupported intent.  It could be a simple code fix, or could involve deeper logic work.  But either way it also is a different task to work than Unsupported or Misunderstood intents.

If one of your team members is tasked solely with a single type of error for the full day or even week they will have less context switching to deal with.  They will be far more efficient, see areas where they can reuse code or even get in the “flow” state while using a specific tool or methodology.   If you aren’t yet categorizing tasks like this and utilizing them for distributing tasks you need to reevaluate your method.

PS – on a final note,  the technology Chatbase has created was very exciting as far as saving time and finding these issues.  Take a look if you haven’t yet!