The Relationship Between Physician Ranking and Patients’ Comments

08.11.2017 • Tad Turpen

Screen Shot 2017-08-11 at 11.24.27 AM.pngWe built the NarrativeDx AI platform for patient experience with the utmost attention to balancing explanatory power with predictive power in our analytical model.

We’ve written before why it's so valuable for our customers to gain crucial insights from patient feedback and improve their patients’ experiences.

At NarrativeDx, we regularly process hundreds of thousands of comments and through an intuitive dashboard, show our customers the trends that emerge from these comments in specific areas of patient experience - such as nurse communication. When our customers get a result from patient satisfaction surveys, such as “70% of the time patients said your nurses were always courteous,” we make it possible for our customers to understand what that score means in the words of their patients.

Patients talk about a wide range of things and in different ways in their comments. For example, when we analyze comments for partners whose aggregate “Top Box” score (responses of “Always” on a scale from Always to Never) for nurse communication is 60%, the themes that emerge from those comments are significantly different as compared to our analysis of comments for partners whose aggregate Top Box score for nurse communication is, say, 90%.

This observation led us to develop the following hypothesis: Patients use different language in their comments when they describe positive experiences as opposed to when they describe negative experiences of the care they receive.

In a recent research project, our Founder and CXO, Senem Guney, PhD, CPXP¹ and I teamed up with Lea Matthews, a fourth-year medical student at Sidney Kimmel Medical College at Thomas Jefferson University. During this project, I led the application of automated natural language processing (NLP) approaches on the analysis of qualitative patient experience data. We applied the fundamental principles of NLP on the analysis of comments from family members about their experiences with physicians who took care of pediatric patients. We designed our project to test our hypothesis that patients use specific language when describing their experience with top performing physicians as opposed to their experience with bottom performing physicians.² Our experiment showed a statistical correlation between the language that patients use in their comments and the physician’s ranking based on the results from patient satisfaction surveys.

“Our study turned out to be a much bigger contribution than I originally hoped to bring to the patient experience in pediatric care,” shares Matthews. “The first interesting part was how words and phrases emerged from our analysis as valuable tools to help physicians adjust their communication skills and empathy levels. We also uncovered physician behaviors that the patients believed to be the most important. It was very illuminating to see how our methodology for language analysis allowed us to tap into the family members’ perspectives. We analyzed patient experience data in a new way and gained insights that we wouldn’t be able to otherwise.”

According to Guney, “It is great to see the scientific and pragmatic impact of our work in the industry. The results from this research project validate that the analytical model behind our platform is as robust as it gets for comment analysis. It also shows that when you have done the hard work, you can shed significant analytical light on the relationship between qualitative and quantitative data in patient experience.”

It was also a joy to work with Ms. Matthews, who was so enthusiastic about our analysis and findings on provider-patient communication. The next-generation physicians are coming out of the medical school with a sincere interest in patient experience. We are thrilled to provide the analytical tools for them to know how to perform their best in their interactions with patients.

Stay tuned to read our full study in a peer-reviewed publication!

I’d love to hear about the ways you’ve applied NLP in the healthcare setting, please feel free to get in touch: tad (at)

Tad Turpen is the Chief Data Scientist at NarrativeDx.

¹We would like to thank Paul Rosen, MD of Nemours Children’s Health System for his support and guidance in this project.
Top performance and bottom performance were defined on the percentile rankings of physicians based on the results from patient satisfaction surveys. 

Tad Turpen

Tad Turpen

Chief Data Scientist, NarrativeDx

Get ”Insights into Patient Experience“ delivered to your inbox