Not So Fast: Analyzing Patient Experience Sentiment is Trickier Than You Think


07.13.2017 • Senem Guney, PhD, CPXP

iStock-510790776.jpgWe’ve written several times about the linguistics and data science knowledge it takes to develop a cutting-edge AI platform like the NarrativeDx platform for patient experience.

What's different about NarrativeDx from other solutions that have been tried before? Our answer to that question continually leads us to exciting partnerships with solution providers in process improvement and clinical development, and I wanted to go over once again how we built our AI platform (the right way) to lead innovation in patient experience data analytics and why that matters for patient experience professionals.

Accuracy in Sentiment Analysis
In patient satisfaction surveys, open-ended questions tend to come right after check-box questions about a particular area of patient experience, as in the example below:
Screen Shot 2017-07-12 at 3.16.57 PM.pngAs you can see in this example, the open-ended question asks the patient to describe their good or bad experience about meals during their hospital stay. The check-box questions are usually about four different aspects of the food service: quality, quantity, temperature and the courtesy of the staff who served the food.
How likely is it for the patient to have a totally good or totally bad experience across these four aspects of food service? More often than not, patients write negatively about some aspects of their experience with the food service and positively about other aspects. For example:
The vegetables were always grey but the server was very kind and helpful in setting up the food tray on my bed.”
Legacy technologies would at best categorize a comment like the one above as having “mixed sentiment” if they provide basic analysis of comments from surveys - because the first sentence (marked in red) indicates a negative sentiment and the second sentence (marked in green) indicates a positive sentiment.
Note: I say “at best” because other providers of natural language processing (NLP) solutions generally don't have the analytical capability to recognize the sentiment in the first sentence correctly. Their analysis can capture negativity in a sentence like “Meals were terrible” but it fails to associate any sentiment to a sentence that has a more nuanced expression of negativity like “The vegetables were always grey.”
Accurate and reliable sentiment analysis matters to patient experience professionals, because there is actually nothing mixed about the sentiment in the comment above. The patient is clearly describing a bad experience with the quality of the food and a good experience with the food server.
Why would you want to limit the categorization of your patients’ verbatim feedback to either totally good or bad experiences in the same comment? Wouldn’t you like to know what specific aspects of meals your patients like and do not like, so you can fix exactly what’s not working? If you can see a negative trend in food quality and a positive trend in the courtesy of the food server, you wouldn’t waste time talking to the food servers about how to do their jobs better. Not to mention, if you have the ability to drill down into the comments behind the negative trend in food quality and identify the fundamental issue to be grey vegetables, you can take action by helping the hospital cook come up with a better way of preparing vegetables.

Keyword Searches vs. NLP-Based Comment Categorization
It often gets too overwhelming to quickly process qualitative data (I know this very well, because for a long time before I founded NarrativeDx, I conducted qualitative research in large and complex organizations). A quick solution is to do keyword searches and find out what people are mostly talking about in their accounts of their experiences. There is nothing fundamentally wrong with this approach to qualitative data - except that when you do keyword searches, the following happens:

  • You can only find what you knew to look for and nothing else ;-)
  • You can be misled by homonyms and waste a lot of time finding what you were not searching in the data. For example, comment categorization based on keyword searches would put these two comments under the same category:

“I would really prefer a private room.”
I thought my private insurance would cover everything.”

If you were doing a keyword search to categorize your patients’ comments about how the staff respected their privacy, a comment about insurance would not belong to that category. While keyword searches cannot help you differentiate between multiple meanings of the same word, an AI platform built on robust NLP expertise about how language works does.
Also, wouldn’t you like to know the specific context in which your patients raise issues about privacy? What if the visualization of trends from your patients’ comments came with filter options to show whether the patient was talking about doctor communication or nurse communication or any other aspect of their experience when they mentioned privacy?
Would that be helpful in your work with your staff to improve your patients’ privacy needs? If the answer is yes, then you need to stop relying on keyword searches and bring in the analytical capability built on “AI done right” that gives actionable insights into your patient’s care experiences.
Also, remember that patients talk about their experiences in venues beyond patient satisfaction surveys. Patient experience professionals need to understand the full “care experience” picture by learning from their patients’ comments from multiple sources along the patient’s journey - from rounding to surveys to social media.
At NarrativeDx, we’ve built a powerful AI platform for patient experience to extract insights from everything patients say whenever and wherever they talk about the care they received. We work hard to ensure that patient experience professionals no longer have to deal with mixed sentiments.
The future of patient experience data analytics is here.
Want to better understand comments from your patients, and have a more accurate way to identify sentiment?
I'd love to help - please feel free to connect with me on LinkedIn, or email me: Senem (at) narrativedx.com

Senem Guney, PhD, CPXP is the Founder and Chief Experience Officer at
NarrativeDx. Follow Senem on Twitter.


Senem Guney, PhD, CPXP

Senem Guney, PhD, CPXP

Founder and Chief Patient Experience Officer, NarrativeDx

Get ”Insights into Patient Experience“ delivered to your inbox