Under Stephen Rue, this function has been outsourced to a company named AlphaBeta which, in its latest effort, is fast approaching what can only be termed peak BS. Some of the claims in the report released — not in Australia, but in the Netherlands on Thursday — have no foundation.
But AlphaBeta's mission — for which it was paid out of taxpayer funds by the NBN Co — was apparently to "prove" that the speeds achievable by the NBN are far better than 59th in the world (which ranking was put down to the Ookla speed test in May) and actually much better.
Before I go any further, let me point out that I have largely stopped writing about the NBN because it is a failed project, just one more example of the politicisation of something that should benefit the public. It is a stark example of the fact that the government is reluctant to spend money on projects that benefit the public and would rather keep taxpayer funds for things that benefit the ruler class.
It is somewhat shocking that statements from the AlphaBeta report should be quoted verbatim in news reports without any questioning. Some have done a cursory examination of the report and called it for what it is, a report that draws conclusions based on extraordinary suppositions.
But without any further ado, let's get into some of the claims made by the good folk at AlphaBeta, who, no doubt, will be laughing all the way to the bank. For one, the report claims that since NBN speeds are unreliable — which nobody would contest — the speed that needs to be counted when computing Australia's rank in the world is the speed that your Internet service provider sells to you.
Yes, if you have bought a 100/40Mbps plan, then that is the speed which AlphaBeta says should be counted. Not the actual speed you experience when you go online. I have had a 100/40 connection for about two years and have measured speeds of as low as 2Mbps and as high as 94Mbps (both download speeds), never that perfect 100Mbps.
But yet, AlphaBeta says the number to be taken into account is 100Mbps. Based on what? They don't say.
The report claims that the NBN is connecting eight million homes and businesses; the figure cited in all financial reports that I have seen is 12 million.
In order to cast doubt on the use of Ookla speed tests, AlphaBeta says: "International speed comparisons are challenging and many global ‘speed test’ rankings present a misleading comparison of international broadband speeds. These rankings are problematic as speed test samples are typically small, unrepresentative, and highly volatile. Rankings also do not account for cross-country differences in broadband availability, population and geography."
Ookla, which provides rankings for both mobile speeds and broadband, only lists countries that have at least 670 unique user results for mobile and at least 3333 for fixed broadband. It compares oranges with oranges. The last time I wrote a report based on Ookla statistics, Australia ranked 60th; that was in March and the figures were from January.
But according to AlphaBeta, this ranking is wrong. In its reckoning, "Australia’s average broadband speed is comparable to those of peer countries. This report finds that Australia ranks 17th among comparable economies based on data representative of all broadband users and accounting for significant cross-country differences in the share of households with access to fixed line broadband. Projecting forward to the full rollout of the NBN, Australia’s rank would rise to 13th among comparable countries."
Before I go any further, let me point out that AlphaBeta does not claim to have any knowledge of technology. In its own words, "AlphaBeta is a strategy, economics and data analytics firm... AlphaBeta specialises in combining advanced analytical techniques and innovative data to generate new insights and fresh perspectives on the challenges facing business and government."
AlphaBeta claims the speed tests that are employed by Ookla, Akamai and M-Lab, all of which tell the truth about Australia — that its broadband sucks when it comes to speed — do not tell the truth.
"Speed test samples are not representative of the general population, as they account only for fixed line broadband users who use speed tests," is a claim AlphaBeta makes, adding, "These samples are often small, highly volatile and unreliable." And what rationale is behind these conclusions? Only the Almighty and the AlphaBeta people know.
Again, it is claimed, "Speed tests do not represent a country’s true average broadband speed as they neglect the share of households without access to broadband." This is plain silly. To get the average of something you calculate the total and then divide it by the number of samples. Why would you count premises without broadband when computing numbers about broadband? Would someone carry out a survey about iPhone users and also include those who have no phones?
"Speed tests do not account for population and geographic factors that affect the cost and complexity of providing high-speed broadband," AlphaBeta says. Huh? What exactly does that mean? A speed test measures how many bits are passing down a line at a given time. Period.
Every company that puts out data on global broadband speeds has computed what would be a sample that is statistically significant; Akamai, for example, is not the type of firm that would publish data here and there and try to ruin its reputation by having that data questioned. Akamai is a very sophisticated operator that knows and understands the broadband business very well. Of course, it may not be as good as producing "alternative facts" as AlphaBeta is.
You do not have to measure the broadband speeds of all users to arrive at an average for a country; you just include a representative sample. If one goes by the AlphaBeta dictum, "the data does not represent all broadband users", then no survey would have any veracity at all. In 40 years of journalism across three countries, I have yet to see a survey that counts all possible samples before drawing conclusions; a statistically significant sample is all that is needed. AlphaBeta offers no empirical evidence to show that the data used by Ookla, Akamai or M-Lab are not statistically significant.
One AlphaBeta statement illustrates the type of fallacious reasoning that it has employed: "Many highly ranked countries like Macao have small groups of speed test users, who are likely to be better connected than the average household. Such comparisons disadvantage countries like Australia in which speed testing is more prevalent." Once again, this is an evidence-free assertion; something like Holy Writ which you can choose to believe. Or not. Hallelujah!
Another statement that is also quite silly is this: "Larger samples are less biased by individual tests and more likely to produce lower average speeds." Really? Since when did that happen?
AlphaBeta appears to be unable to grasp one thing: the various speed tests that it tries to discredit are measuring speed. Just speed. Not geography. Not climate. Not skin colour. Not diet. Not soft-drink consumption. Is that too difficult to grasp?
The alternative that AlphaBeta proposes is to measure speeds by subscription data. And in very fine print, it says: "Subscription speeds are equivalent to the advertised speed of a plan, different to speed test data as they do not measure actual speeds experienced by end users."
But I forget; what AlphaBeta has been tasked with was decided in advance by the NBN Co: "We have an image problem due to these speed tests, so just obfuscate the whole thing and make life easier for us." Rather than tell the truth and refuse to try and paint the metetricious as the magnificent, AlphaBeta went for it. After all, this means future commissions, better translated as Australian dollars, will come its way.
The NBN has been a disaster in the making from the time the technology mix was changed. But even at the start, a grievous mistake was made: neoliberal thinking ensured that it would be not put on the budget, but instead designed as a project which would reclaim its own costs. It was not thought of as a public utility — like water, gas and electricity — but rather as a means to make a buck.
The Coalition Government has US$35 billion to spend on F-35 jet fighters. It has $90 billion to spend on submarines. It has $150 million to toss out to make US President Donald Trump happy, as a contribution to some space program. But there is no money for decent broadband that would enable people in all corners of the country to think of businesses that could be run remotely.
And so we have shameless exercises like this, indulged in by the NBN Co. And it won't be the last.