Why a Keyword “wasn’t” Enough for a Nation-Leading Pediatric Health System

09.24.2019 • Senem Guney, PhD, CPXP


I’ve always been fascinated by the complexity of language. There are many diverse ways, even among highly similar cohorts of people, of describing experiences. Even a simple concept such as room temperature could be described as “I was freezing” or “it’s colder in this room than Minnesota in December,” or “the temperature in the exam room was 66 degrees.”  

This is why approaches such as natural language processing (NLP) have become so popular in understanding feedback from patients and providers. But as we discussed previously, not all NLP is created equal. We identify use cases for the needs of high quality NLP in our original research study in the recent special issue of the Patient Experience Journal (PXJ) on the Technology and Innovation in Patient Experience

In this study, we used NLP to understand the patient experience comments of pediatric patients’ family members describing care provided by their physician at one of the largest pediatric health systems in the US. As we discuss in the article (p. 56), we observed a peculiar phenomenon in the descriptions of positive experiences: Use of words like “wasn’t” that are traditionally thought of as negative.

On the surface, this doesn’t make sense. You would search for words like “wasn’t” if you wanted to find language describing negative experiences and you would likely find some negative comments like “the doctor wasn’t nice,” “the nurse wasn’t listening to me,” and so on.

You would, however, also miss most of what you are trying to find. As we indicate in our study, family members frequently say: “My doctor wasn’t rushed” or “The doctor wasn’t late” when describing aspects of positive care experiences. In other words, family members described their positive experiences with physicians in terms of what the experience was not like or what did not happen.

This is why keyword models and NLP without expertise can be misleading. It’s incredibly difficult to accurately and reliably reveal what patients are actually saying due to the diversity in their descriptions. This is where our patented technology and industry expertise lead the way for our clients. 

If you are interested in learning more about the real world applications of this study, here is a case study of Cedars-Sinai applying this approach in the real world to improve their Likelihood to Recommend scores by 28%. 

Senem Guney, PhD, CPXP

Senem Guney, PhD, CPXP

Founder and Chief Patient Experience Officer, NarrativeDx

Get ”Insights into Patient Experience“ delivered to your inbox