Author's Opinion

The views in this column are those of the author and do not necessarily reflect the views of iTWire.

Have your say and comment below.

Tuesday, 09 October 2018 16:18

Marketing people and useful statistics – n'er the twain shall meet

Marketing people and useful statistics – n'er the twain shall meet Image: Max Pixel (used with permission)

Word doesn't make you an author, Photoshop doesn't make you an artist, and SurveyMonkey most certainly doesn't make you a researcher.

I recently read an article noting that the expected feedback value for Uber riders and drivers was 5 (out of 5) unless there were major reasons to offer something lower. It seemed that a 4 was for unreasonable behaviour and a 3 meant disaster. Who knows how bad you'd have to be in order to rate a 1 or 2!!

There is something inherently wrong with this kind of assessment – the expectation that maximum marks will be given unless there was an exceptional reason not to (this smells a lot like modern primary schools' "everyone gets a prize" mentality that we hear of so often). Surely, there is almost always room for improvement, no matter how exceptional the service delivery.

Until recently, this seemed to be relatively "academic", I'd not experienced it personally (not being an Uber user – am I the only one left?). However, I came to have my car serviced at a local dealership that represented the vehicle's very well-known international brand.

At the time I paid the account and collected my keys, I was warned by the assistant that I may receive an email requesting a response to the brand's standard customer survey. As part of the conversation, the assistant warned me that any rating lower than a '9' (out of 10) would reflect poorly on the dealership. I elected to stay schtum and not reveal a surprised/aghast expression on my face. My surprise extended in two directions – that the survey might actually expect this, and secondly that the person felt the necessity to warn me of this likelihood.

Sure enough, a few days later I did receive the email and proceeded to open the survey. Equally sure enough, the opening salvo looked like this:

 Redacted Rating

I felt compelled to write to the team managing the survey to express my total dissatisfaction with their understanding of suitable data-gathering. The following is a slightly edited version of that email.

Whoever added the annotation that a 7 or 8 was "neutral" and a 9 or 10 was "extremely likely" has a remarkably limited understanding of maths and statistics.

On any reasonable 0-10 scale, "neutral" is a 4, 5 or 6, while 7 and 8 would be rated as "positive" and 9 or 10, as you suggest "extremely likely". Labelling a 6 as "not at all likely" is both condescending and highly inaccurate, not to mention mathematically misleading.

Any customer interaction will contain room for improvement, therefore insisting (as your annotations do) that the only acceptable rating is a 9 or 10 is the worst kind of virtue-signalling I've seen in a long time. Even worse that the on-side attendant felt the need to set his expectation in advance.

Worse, you are NOT collecting any useful data — all you will get is a collection of 9s and 10s everyone in your organisation is smiling and happy, safe in the knowledge that the entire customer body is satisfied. The marketing team achieves their KPIs and everyone's pay-packet is suitably enriched.

Unfortunately, if there is a level of dissatisfaction, you won't see it. Having been told that the dealership will be "penalised" for low ratings, I would reasonably assume that people will probably not respond (as I am doing) or they will grit their teeth and respond according to your expectations.

I can only assume there is no-one in your team with even rudimentary statistical training, otherwise this would surely never have been foisted upon an unsuspecting public.

By the way, I stepped through the remainder of the pages (with random responses and without submitting) and it became remarkably clear that EVERY QUESTION was directly linked to a specific measure of the performance of the workshop. Questions like, "did they greet me by name," or "was the work to be performed fully explained, along with a break-down of costs."

As they say, "what gets measured gets done," or to quote Goodhart's Law, "When a measure becomes a target, it ceases to be a good measure".

I wrote to the company in the hope that some statistical "moral fibre" might be instilled, but I'm not holding my breath. I write this on iTWire in the hope that the message might be seen more broadly; again, my breath is not bated.

As I noted in the opening, just because you know how to use a tool, doesn't mean you're an expert in the process for which that tool is used. Knowing how to use Microsoft Word does NOT make you a writer and knowing how to create a survey in SurveyMonkey (or any of the other tools available) doesn't mean you're going to get meaningful, actionable data.

Perhaps the company in question doesn't want to learn the truth – if so they've done an excellent job of ensuring that specific outcome, as absolutely nothing useful will be learned from the survey in its current form.

Addendum: Prior to publishing this comment, I caught up with a family member who worked for a multi-brand car dealership in a major Australian country town. His own observations were very similar, noting that they were very close to losing one of their brands due to "low" feedback ratings. He also mentioned that his own mother had told him (I paraphrase both statements), "I thought the service was really good – I gave you 7s and 8s for everything." He replied, "but Mum, but we need 9s and 10s to stay in the car company's good books."



Recently iTWire remodelled and relaunched how we approach "Sponsored Content" and this is now referred to as "Promotional News and Content”.

This repositioning of our promotional stories has come about due to customer focus groups and their feedback from PR firms, bloggers and advertising firms.

Your Promotional story will be prominently displayed on the Home Page.

We will also provide you with a second post that will be displayed on every page on the right hand side for at least 6 weeks and also it will appear for 4 weeks in the newsletter every day that goes to 75,000 readers twice daily.




Denodo, the leader in data virtualisation, has announced a debate-style three-part Experts Roundtable Series, with the first event to be hosted in the APAC region.

The round table will feature high-level executives and thought leaders from some of the region’s most influential organisations.

They will debate the latest trends in cloud adoption and technologies altering the data management industry.

The debate will centre on the recently-published Denodo 2020 Global Cloud Survey.

To discover more and register for the event, please click the button below.


David Heath

David Heath has had a long and varied career in the IT industry having worked as a Pre-sales Network Engineer (remember Novell NetWare?), General Manager of IT&T for the TV Shopping Network, as a Technical manager in the Biometrics industry, and as a Technical Trainer and Instructional Designer in the industrial control sector. In all aspects, security has been a driving focus. Throughout his career, David has sought to inform and educate people and has done that through his writings and in more formal educational environments.


Webinars & Events