Friday, 04 October 2013 11:38

More BS benchmarks

By

iTWire yesterday noted the formation of the MobileBench industry consortium to ‘standardize evaluation tools for the mobile space.’ It appears this may be just a front to help its members manipulate benchmarks even further.

The article ‘Lack of standards = more marketing BS’ led me to do some more investigation.

A series of tweets from @andreif7 suggested that Samsung was manipulating benchmarks – to gain boasting rights of course – by detecting commonly used benchmark software and putting its products into overdrive by overclocking all processor cores, ignoring thermal limits and power management. When the benchmark software was ‘camouflaged’, the same device ran slower.

AnandTech yesterday ran an article ‘They're (Almost) All Dirty: The State of Cheating in Android Benchmarks’ hmarks which is a long technical read - very enlightening.

Authors Anand Lal Shimpi, and Brian Klug discovered that Samsung had ‘tweaked’ the Galaxy S4, Note 3, and Tab 3 devices to cheat in commonly used tests including 3DM, AnTuTu, AndEBench, Basemark X, Geekbench, GFXB, and Vellamo.

Samsung impressively stood out for their proclivity to ‘optimise’ but were not alone – other manufacturers had indulged in less intensive tweaking. AnandTech did infer that Apple and chipmakers including nVIDIA, Qualcomm and Intel did not appear to condone this.

Interestingly they compared Nexus devices (vanilla Android marketed by Google) and their branded counterparts and noticed differences. Sure, a manufacturer is entitled to optimise – to use a euphemistic term – a device’s performance under differing conditions but said conditions must reflect real world use.

Opinion

Some would say this is just a storm in a teacup. I think it reflects the ethics of companies driven by management dictums to win at any cost.

The Japanese have a wonderful saying “The fish rots from the head.” In essence it means that employees live under the shadow of a leader – their likes, dislikes, treatment of subordinates, language and idioms, personal preferences, beliefs and values tends to shape the characteristics, culture and ways of doing business in the organization. To observe that Samsung wants to win in all markets would be a reasonable.

Unfortunately, for Samsung it has a history of getting things wrong – here is recent a selection obtained from searching for Samsung Class Action.

  • Overstating energy efficiency on some fridges - known and unresolved design faults
  • Overstating water efficiency of washing machines - known and unresolved design faults
  • Avoiding repair obligations in the US between 2010-12 on more than 7.5 million faulty TV’s now out of warranty – later settled via Class Action
  • Knowingly installing spyware (Carrier iQ) without user permission on Samsung handsets to collect usage metrics. HTC was also involved.
  • Samsung D6000 series TVs claiming 3D in full HD 1080p but only being only 540p
  • The interminable Apple/Samsung patent wars - Apple usually has the moral high ground
  • A US class action held Samsung to count for overstating the effective storage capacity of its S4 handsets.
  • Another US class action over allegedly promised Android updates that never came
  • Samsung and Hynix colluding to keep memory prices high

Sure, you could say that for such a mega company these instances are minor – I accept that and give Samsung the benefit of the doubt. The irony is that today – here and now - Samsung makes some damned good products and the ‘little white lies’ and ‘crawling over the bones of competitors’ to win at all costs should no longer be necessary. If this behaviour continues, it would then reflect poor top-level management and consequently overzealous minions getting away with this behaviour.

One thing is certain - careless disregard for the truth will be all too easily caught out via the ubiquitous internet.

Read 2076 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here




EXL AI IN ACTION VIRTUAL EVENT 20 MARCH 2025

Industry leaders are looking to transform their businesses and achieve measurable outcomes with AI.

As organisations across APAC navigate the complexities of AI adoption, this must-attend event brings together industry leaders, real-world demonstrations, and visionary panel discussions to bridge the gap between proof-of-concepts and enterprise-wide AI implementation.

Learn how to overcome common challenges in deploying AI at scale.​

Unlock cost savings, efficiency, and better customer experiences with AI.

Discover how industry expertise and data intelligence enable practical AI deployment.

Register for the event now!

REGISTER!

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

BACK TO HOME PAGE
Ray Shaw

joomla stats

Ray Shaw ray@im.com.au  has a passion for IT ever since building his first computer in 1980. He is a qualified journalist, hosted a consumer IT based radio program on ABC radio for 10 years, has developed world leading software for the events industry and is smart enough to no longer own a retail computer store!

Share News tips for the iTWire Journalists? Your tip will be anonymous

Subscribe to Newsletter

*  Enter the security code shown: img0