I recently read an article noting that the expected feedback value for Uber riders and drivers was 5 (out of 5) unless there were major reasons to offer something lower. It seemed that a 4 was for unreasonable behaviour and a 3 meant disaster. Who knows how bad you'd have to be in order to rate a 1 or 2!!

There is something inherently wrong with this kind of assessment – the expectation that maximum marks will be given unless there was an exceptional reason not to (this smells a lot like modern primary schools' "everyone gets a prize" mentality that we hear of so often). Surely, there is almost always room for improvement, no matter how exceptional the service delivery.

Until recently, this seemed to be relatively "academic", I'd not experienced it personally (not being an Uber user – am I the only one left?). However, I came to have my car serviced at a local dealership that represented the vehicle's very well-known international brand.

At the time I paid the account and collected my keys, I was warned by the assistant that I may receive an email requesting a response to the brand's standard customer survey. As part of the conversation, the assistant warned me that any rating lower than a '9' (out of 10) would reflect poorly on the dealership. I elected to stay schtum and not reveal a surprised/aghast expression on my face. My surprise extended in two directions – that the survey might actually expect this, and secondly that the person felt the necessity to warn me of this likelihood.

Sure enough, a few days later I did receive the email and proceeded to open the survey. Equally sure enough, the opening salvo looked like this:

 

I felt compelled to write to the team managing the survey to express my total dissatisfaction with their understanding of suitable data-gathering. The following is a slightly edited version of that email.

Whoever added the annotation that a 7 or 8 was "neutral" and a 9 or 10 was "extremely likely" has a remarkably limited understanding of maths and statistics.

On any reasonable 0-10 scale, "neutral" is a 4, 5 or 6, while 7 and 8 would be rated as "positive" and 9 or 10, as you suggest "extremely likely". Labelling a 6 as "not at all likely" is both condescending and highly inaccurate, not to mention mathematically misleading.

Any customer interaction will contain room for improvement, therefore insisting (as your annotations do) that the only acceptable rating is a 9 or 10 is the worst kind of virtue-signalling I've seen in a long time. Even worse that the on-side attendant felt the need to set his expectation in advance.

Worse, you are NOT collecting any useful data — all you will get is a collection of 9s and 10s — everyone in your organisation is smiling and happy, safe in the knowledge that the entire customer body is satisfied. The marketing team achieves their KPIs and everyone's pay-packet is suitably enriched.

Unfortunately, if there is a level of dissatisfaction, you won't see it. Having been told that the dealership will be "penalised" for low ratings, I would reasonably assume that people will probably not respond (as I am doing) or they will grit their teeth and respond according to your expectations.

I can only assume there is no-one in your team with even rudimentary statistical training, otherwise this would surely never have been foisted upon an unsuspecting public.

By the way, I stepped through the remainder of the pages (with random responses and without submitting) and it became remarkably clear that EVERY QUESTION was directly linked to a specific measure of the performance of the workshop. Questions like, "did they greet me by name," or "was the work to be performed fully explained, along with a break-down of costs."

As they say, "what gets measured gets done," or to quote Goodhart's Law, "When a measure becomes a target, it ceases to be a good measure".

I wrote to the company in the hope that some statistical "moral fibre" might be instilled, but I'm not holding my breath. I write this on iTWire in the hope that the message might be seen more broadly; again, my breath is not bated.

As I noted in the opening, just because you know how to use a tool, doesn't mean you're an expert in the process for which that tool is used. Knowing how to use Microsoft Word does NOT make you a writer and knowing how to create a survey in SurveyMonkey (or any of the other tools available) doesn't mean you're going to get meaningful, actionable data.

Perhaps the company in question doesn't want to learn the truth – if so they've done an excellent job of ensuring that specific outcome, as absolutely nothing useful will be learned from the survey in its current form.

Addendum: Prior to publishing this comment, I caught up with a family member who worked for a multi-brand car dealership in a major Australian country town. His own observations were very similar, noting that they were very close to losing one of their brands due to "low" feedback ratings. He also mentioned that his own mother had told him (I paraphrase both statements), "I thought the service was really good – I gave you 7s and 8s for everything." He replied, "but Mum, but we need 9s and 10s to stay in the car company's good books."