Fox News posted an article recently that said many more people are using online ratings in order to choose a doctor. Apparently 59% now say these ratings are either somewhat important or important when choosing a new doctor. Similarly, The Guardian recently had an article about how we should be evaluating the quality of education at the University level.
The core problem with rating systems and key information statistics and the like is that often, the rating systems are biased both due to the people who choose to participate in the rating systems, and that as we gather more and more data, we’re not always sure of the markers of quality, assuming we can define quality well.
Let’s take the physician example. Patients who leave reviews online are either thrilled with their doctor or disgruntled, or they have been specifically asked to leave a review. So you should expect, on average, to see wildly great recommendations or really awful ones, but not a lot of middle ground, which probably is more indicative of patient experience. On top of that, we know that no matter how great a doctor is technically and no matter how well educated, other things like office wait time and “front of house” staff also help bias a patient’s view of the experience in the office. If you come to see a doctor and you are nervous and worried, and that’s compounded by seemingly indifferent office staff, loud daytime TV and antiquated magazines, it’s natural to think the doctor is not spending enough time managing the customer service portion of the business, so how well is he managing the patient care aspect? However, in many offices, especially in hospital based practices, the doctor may have very little to do with the hiring or management of the front office, so these markers are entirely irrelevant to the quality of care delivered.
Likewise, there are many doctors I know who are wonderful and terrific people, but I hear stories from other physicians and nurses about their level of attention to detail or general practice styles that would make you very wary of sending a friend or neighbor to see them. In other words, medicine’s core skill set- observation, diagnosis, and treatment decisions- are less likely to be observed or adequately evaluated from the patient’s perspective, although you will get a good sense of bed side manner perhaps or office efficiency.
Now let’s look at school. Just because we all have been to school, and are armed with both positive and negative experiences from our years in education, gives us very little experience with understanding different pedagogies, ways to teach, or curriculum design. Very few parents even understand basic child development and psychology, let alone the best way to teach a subject to a specific child. While I think students should have a voice in teacher evaluations, since they are in the best position to see what it’s like to sit in a teacher’s class every day, this information has to be filtered as well, as students will be as likely as the rest of us to judge a teacher’s effectiveness based on their treatment of us as individuals. For example, a few years ago, my son had a run in with a teacher that was unusual in its scope and difficulty. But that teacher has been teaching for years, and for most students, there isn’t much of a problem. Should her career be determined by whatever her conflict with my son was about, or should it be looked at as a whole, and all the kids she has helped?
The bottom line here is that we can only expect so much out of online rating systems, and even recommendations by friends. The best we can hope for is an opinion from someone who knows the profession well enough (peer to peer) to give a more educated and nuanced evaluation of quality, and data to consider regarding trends of interactions and experiences. This is not to say that there aren’t external markers of quality that can be created and benchmarked- things like c-section rate, or common assessment test scores- but these are just one among a group of factors that make up what we can call quality or competency at a job, particularly one that is so dependent on personal service and interaction.
As much as I love the internet, feedback, reviews and rating systems, I also know some of them are biased and colored by extremely good and extremely bad experiences, some of which may not even reflect on the business itself. (Let’s not even talk about people gaming Yelp reviews.) And I become more and more concerned when people are evaluating services that have long term, life altering consequences like education and medical care. If I have a bad experience with a plumber, I can try a new one. If I have a bad experience with a cardiologist, I could die.
Take those reviews, especially when everything important to you is on the line, with a grain of salt.