The article ‘Lack of standards = more marketing BS’ led me to do some more investigation.
A series of tweets from @andreif7 suggested that Samsung was manipulating benchmarks – to gain boasting rights of course – by detecting commonly used benchmark software and putting its products into overdrive by overclocking all processor cores, ignoring thermal limits and power management. When the benchmark software was ‘camouflaged’, the same device ran slower.
|
AnandTech yesterday ran an article ‘They're (Almost) All Dirty: The State of Cheating in Android Benchmarks’ hmarks which is a long technical read - very enlightening.
Authors Anand Lal Shimpi, and Brian Klug discovered that Samsung had ‘tweaked’ the Galaxy S4, Note 3, and Tab 3 devices to cheat in commonly used tests including 3DM, AnTuTu, AndEBench, Basemark X, Geekbench, GFXB, and Vellamo.
Samsung impressively stood out for their proclivity to ‘optimise’ but were not alone – other manufacturers had indulged in less intensive tweaking. AnandTech did infer that Apple and chipmakers including nVIDIA, Qualcomm and Intel did not appear to condone this.
Interestingly they compared Nexus devices (vanilla Android marketed by Google) and their branded counterparts and noticed differences. Sure, a manufacturer is entitled to optimise – to use a euphemistic term – a device’s performance under differing conditions but said conditions must reflect real world use.
Opinion
Some would say this is just a storm in a teacup. I think it reflects the ethics of companies driven by management dictums to win at any cost.
The Japanese have a wonderful saying “The fish rots from the head.” In essence it means that employees live under the shadow of a leader – their likes, dislikes, treatment of subordinates, language and idioms, personal preferences, beliefs and values tends to shape the characteristics, culture and ways of doing business in the organization. To observe that Samsung wants to win in all markets would be a reasonable.
Unfortunately, for Samsung it has a history of getting things wrong – here is recent a selection obtained from searching for Samsung Class Action.
- Overstating energy efficiency on some fridges - known and unresolved design faults
- Overstating water efficiency of washing machines - known and unresolved design faults
- Avoiding repair obligations in the US between 2010-12 on more than 7.5 million faulty TV’s now out of warranty – later settled via Class Action
- Knowingly installing spyware (Carrier iQ) without user permission on Samsung handsets to collect usage metrics. HTC was also involved.
- Samsung D6000 series TVs claiming 3D in full HD 1080p but only being only 540p
- The interminable Apple/Samsung patent wars - Apple usually has the moral high ground
- A US class action held Samsung to count for overstating the effective storage capacity of its S4 handsets.
- Another US class action over allegedly promised Android updates that never came
- Samsung and Hynix colluding to keep memory prices high
Sure, you could say that for such a mega company these instances are minor – I accept that and give Samsung the benefit of the doubt. The irony is that today – here and now - Samsung makes some damned good products and the ‘little white lies’ and ‘crawling over the bones of competitors’ to win at all costs should no longer be necessary. If this behaviour continues, it would then reflect poor top-level management and consequently overzealous minions getting away with this behaviour.
One thing is certain - careless disregard for the truth will be all too easily caught out via the ubiquitous internet.