Tuesday, 09 October 2018 16:18

Marketing people and useful statistics – n'er the twain shall meet

By
Marketing people and useful statistics – n'er the twain shall meet Image: Max Pixel (used with permission)

Word doesn't make you an author, Photoshop doesn't make you an artist, and SurveyMonkey most certainly doesn't make you a researcher.

I recently read an article noting that the expected feedback value for Uber riders and drivers was 5 (out of 5) unless there were major reasons to offer something lower. It seemed that a 4 was for unreasonable behaviour and a 3 meant disaster. Who knows how bad you'd have to be in order to rate a 1 or 2!!

There is something inherently wrong with this kind of assessment – the expectation that maximum marks will be given unless there was an exceptional reason not to (this smells a lot like modern primary schools' "everyone gets a prize" mentality that we hear of so often). Surely, there is almost always room for improvement, no matter how exceptional the service delivery.

Until recently, this seemed to be relatively "academic", I'd not experienced it personally (not being an Uber user – am I the only one left?). However, I came to have my car serviced at a local dealership that represented the vehicle's very well-known international brand.

At the time I paid the account and collected my keys, I was warned by the assistant that I may receive an email requesting a response to the brand's standard customer survey. As part of the conversation, the assistant warned me that any rating lower than a '9' (out of 10) would reflect poorly on the dealership. I elected to stay schtum and not reveal a surprised/aghast expression on my face. My surprise extended in two directions – that the survey might actually expect this, and secondly that the person felt the necessity to warn me of this likelihood.

Sure enough, a few days later I did receive the email and proceeded to open the survey. Equally sure enough, the opening salvo looked like this:

 Redacted Rating

I felt compelled to write to the team managing the survey to express my total dissatisfaction with their understanding of suitable data-gathering. The following is a slightly edited version of that email.

Whoever added the annotation that a 7 or 8 was "neutral" and a 9 or 10 was "extremely likely" has a remarkably limited understanding of maths and statistics.

On any reasonable 0-10 scale, "neutral" is a 4, 5 or 6, while 7 and 8 would be rated as "positive" and 9 or 10, as you suggest "extremely likely". Labelling a 6 as "not at all likely" is both condescending and highly inaccurate, not to mention mathematically misleading.

Any customer interaction will contain room for improvement, therefore insisting (as your annotations do) that the only acceptable rating is a 9 or 10 is the worst kind of virtue-signalling I've seen in a long time. Even worse that the on-side attendant felt the need to set his expectation in advance.

Worse, you are NOT collecting any useful data — all you will get is a collection of 9s and 10s everyone in your organisation is smiling and happy, safe in the knowledge that the entire customer body is satisfied. The marketing team achieves their KPIs and everyone's pay-packet is suitably enriched.

Unfortunately, if there is a level of dissatisfaction, you won't see it. Having been told that the dealership will be "penalised" for low ratings, I would reasonably assume that people will probably not respond (as I am doing) or they will grit their teeth and respond according to your expectations.

I can only assume there is no-one in your team with even rudimentary statistical training, otherwise this would surely never have been foisted upon an unsuspecting public.

By the way, I stepped through the remainder of the pages (with random responses and without submitting) and it became remarkably clear that EVERY QUESTION was directly linked to a specific measure of the performance of the workshop. Questions like, "did they greet me by name," or "was the work to be performed fully explained, along with a break-down of costs."

As they say, "what gets measured gets done," or to quote Goodhart's Law, "When a measure becomes a target, it ceases to be a good measure".

I wrote to the company in the hope that some statistical "moral fibre" might be instilled, but I'm not holding my breath. I write this on iTWire in the hope that the message might be seen more broadly; again, my breath is not bated.

As I noted in the opening, just because you know how to use a tool, doesn't mean you're an expert in the process for which that tool is used. Knowing how to use Microsoft Word does NOT make you a writer and knowing how to create a survey in SurveyMonkey (or any of the other tools available) doesn't mean you're going to get meaningful, actionable data.

Perhaps the company in question doesn't want to learn the truth – if so they've done an excellent job of ensuring that specific outcome, as absolutely nothing useful will be learned from the survey in its current form.

Addendum: Prior to publishing this comment, I caught up with a family member who worked for a multi-brand car dealership in a major Australian country town. His own observations were very similar, noting that they were very close to losing one of their brands due to "low" feedback ratings. He also mentioned that his own mother had told him (I paraphrase both statements), "I thought the service was really good – I gave you 7s and 8s for everything." He replied, "but Mum, but we need 9s and 10s to stay in the car company's good books."

Read 5469 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here

GET READY FOR XCONF AUSTRALIA 2022

Thoughtworks presents XConf Australia, back in-person in three cities, bringing together people who care deeply about software and its impact on the world.

In its fifth year, XConf is our annual technology event created by technologists for technologists.

Participate in a robust agenda of talks as local thought leaders and Thoughtworks technologists share first-hand experiences and exchange new ways to empower teams, deliver quality software and drive innovation for responsible tech.

Explore how at Thoughtworks, we are making tech better, together.

Tickets are now available and all proceeds will be donated to Indigitek, a not-for-profit organisation that aims to create technology employment pathways for First Nations Peoples.


Click the button below to register and get your ticket for the Melbourne, Sydney or Brisbane event

GET YOUR TICKET!

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

BACK TO HOME PAGE
David Heath

David Heath has had a long and varied career in the IT industry having worked as a Pre-sales Network Engineer (remember Novell NetWare?), General Manager of IT&T for the TV Shopping Network, as a Technical manager in the Biometrics industry, and as a Technical Trainer and Instructional Designer in the industrial control sector. In all aspects, security has been a driving focus. Throughout his career, David has sought to inform and educate people and has done that through his writings and in more formal educational environments.

Share News tips for the iTWire Journalists? Your tip will be anonymous

VENDOR NEWS