NarrativeDx Blog

Not all Natural Language Processing is Created Equal

- Tad Turpen

ironman1-kXj--621x414@LiveMint.jpg

This is the first post in a series detailing how Natural Language Processing (NLP) of patient comments can derive detailed, actionable insights to improve care experiences.   


What is and is not NLP

NLP is a way to take ambiguous textual data and interpret it in a structured and straightforward way. The term “NLP” is, however, used liberally.  It can describe a range of things from cutting-edge AI platforms like the one we have developed at NarrativeDx, to completely manual crowd-sourced analysis presented in a way that looks automated.

With such a broad meaning, it is clear that not all NLP platforms are created equal.  It is difficult to capture meaning from patient comments, especially when people can describe similar experiences using very dissimilar words, phrases and meaning.  

 

A Real World Example
A simple example will help distinguish between the NLP technology at NarrativeDx and other approaches to NLP. Consider the following comment:  

"Dr. Jones was really cool, but the shwr was filthy.  The food was too cool when it arrived at my room but the nurse was very apologetic about the delay"

Comments regarding care experiences are complex and full of detail.  It’s clear to a human reader that there are several important aspects to this care experience:

  1. "Dr. Jones was really cool” - The patient had a great experience with Dr Jones and enjoyed her bedside manner.

  2. "but the shwr was filthy” - The inpatient room, specifically the shower, was not clean and contributed to a bad experience.

  3. "The food was also too cool when it arrived at my room" - The food temperature was too cold when it arrived.  The reason it was cold, a delivery delay, isn’t contained in this part of the sentence.

  4. “but the nurse was very apologetic about the delay.” - The staff was apologetic, contributing to a positive experience, and explained the reason for the temperature was delivery delay.

In sum, Doctor Jones and the nurses are doing their part to contribute to a great experience.  The staff, however, seems to be struggling with meal delivery time and scheduled shower cleanings.  

While all of those details are apparent to a human, it’s not possible for a human to read hundreds of these comments a month and capture this level of information.  When done manually, this leads people to search for keywords to find relevant comments, such as “shower” or “bedside manner.”  Many NLP systems that do not rely on humans utilize this approach, known as keyword search.  There are several problems with this approach:

  1. The keyword you search for must be contained in the comment.  In this example, “shower” would not find this comment since it is misspelled “shwr.”  Similarly, unless you were searching for the word “cool”, which could describe food or the doctor, this would also not be helpful.

  2. You have to know what you are looking for ahead of time.  If you don’t know ahead of time to search for a doctor being “cool,” you’d never find it.

  3. This approach views the comment as a whole, and becomes confused when a single comment is about multiple aspects of the patient experience.  Instead of being able to identify aspects of the experience identified in parts A-D above, such a comment would simply be labeled “mixed,” which misses all important aspects of the experience.

NarrativeDx Can Help
We have built and patented the first automated NLP pipeline specifically for patient experience with human accuracy and automated speed.  Just as in the example above, it knows a comment about a “cool” doctor is positive and about bedside manner, and that “too cool” food is bad and about late meal delivery.  We do all of this automatically, at scale, and present the comments in an intuitive user interface.  It’s the best of both worlds.

Natural language processing is an incredibly powerful tool that, when wielded correctly, can empower those in a position to make meaningful changes to their organization, hospital, or beyond. We present meaningful insights at scale and present them through an intuitive dashboard so you can take action immediately.

Taylor (Tad) Turpen is the Chief Data Scientist at NarrativeDx.

  Test-drive NarrativeDx today    

Need an easy and smart solution for understanding patient and clinical staff comments?
Test-drive NarrativeDx today, it's like having a team of data scientists and patient experience experts at your fingertips.

Photo credit: livemint.com, a still from the movie, Iron Man. 

Get email updates