Security Market Segment LS
Wednesday, 17 June 2020 09:31

Failed network configuration changes by T-Mobile caused outage: Cloudflare chief Featured

Matthew Prince: "...don’t worry, this is one thing that does not need to get added to the list of craziness that has been 2020." Matthew Prince: "...don’t worry, this is one thing that does not need to get added to the list of craziness that has been 2020." Supplied

The outages experienced by many companies in the US on Tuesday (Australian time) were apparently due to T-Mobile making changes to its network configuration resulting in a series of cascading failures for its users which affected both voice and data networks, the chief executive and co-founder of web-infrastructure and website-security company, Cloudflare claims.

Matthew Prince said in a series of tweets that this resulted in many users complaining on Twitter and other forums that they could not reach popular forums.

"So now people are looking around for an explanation and they stumble across sites like the Arbor Networks attack map," Prince said. "It looks terrifying today! Thing is, it always looks terrifying. It’s a marketing gimmick put up to sell DDoS mitigation services so that’s not surprising."

T-Mobile chief executive Mike Siewert made no mention of the configuration issues in a short post he made on the company's website at 8.45pm PST on 15 June (1.45pm AEDT on 16 June): "Starting just after 12 pm ET (3am AEDT on 16 June) and continuing throughout the day, T-Mobile has been experiencing a voice and text issue that has intermittently impacted customers in markets across the US," he wrote.

"We are recovering from this now but it may still take several more hours before customer calling and texting is fully recovered. Neville Ray has shared updates throughout the day but I wanted to share the latest on what we know and what we’re doing to address it.

"This is an IP traffic-related issue that has created significant capacity issues in the network core throughout the day. Data services have been working throughout the day and customers have been using services like FaceTime, iMessage, Google Meet, Google Duo, Zoom, Skype and others to connect."

Prince's explanation appears to be on the mark as there was not a single post about this so-called DDoS attack on the North American Network Operators Group mailing list – and any time there is a major outage due to a DDoS or other reason, like a BGP error, this group is full of chatter.

Prince said that from Cloudflare's point of view, there were many things that showed there was no big DDoS attack. "First, traffic from WARP to supposedly impacted services is normal and has no increase in errors.

"Second, there is no spike in traffic to any of the major Internet Exchanges, which you do see during actual DDoS attacks and definitely would during one allegedly this disruptive.

"Finally, our team know the network operators at nearly all the other major Internet services and platforms and none of them are reporting anything anomalous."

He ended with a jab at T-Mobile, saying that nobody was having a bad day, "Except T-Mobile, which is having a bad day almost certainly entirely of their own team’s making. So, please, #hugops. And don’t worry, this is one thing that does not need to get added to the list of craziness that has been 2020."

Former NSA hacker Jake Williams told iTWire that it was just T-Mobile and then Down Detector showing false positives "because people were talking about the T-Mobile outage on other platforms and it became a self licking ice cream cone".

Williams, who now runs his own outfit, Rendition Infosec, said: "I haven't looked at the root cause inside of T-Mobile – other than some Duo issues, it didn't impact me or clients". Duo is enterprise multi-factor authentication that primarily uses push notifications but which could fall back to SMS depending on the configuration, he added.

Quotes in the last two paragraphs of this article were mistakenly attributed to British security expert Kevin Beaumont at the time of publication. The error is regretted and has been corrected. 

Subscribe to ITWIRE UPDATE Newsletter here

Now’s the Time for 400G Migration

The optical fibre community is anxiously awaiting the benefits that 400G capacity per wavelength will bring to existing and future fibre optic networks.

Nearly every business wants to leverage the latest in digital offerings to remain competitive in their respective markets and to provide support for fast and ever-increasing demands for data capacity. 400G is the answer.

Initial challenges are associated with supporting such project and upgrades to fulfil the promise of higher-capacity transport.

The foundation of optical networking infrastructure includes coherent optical transceivers and digital signal processing (DSP), mux/demux, ROADM, and optical amplifiers, all of which must be able to support 400G capacity.

With today’s proprietary power-hungry and high cost transceivers and DSP, how is migration to 400G networks going to be a viable option?

PacketLight's next-generation standardised solutions may be the answer. Click below to read the full article.


WEBINAR PROMOTION ON ITWIRE: It's all about webinars

These days our customers Advertising & Marketing campaigns are mainly focussed on webinars.

If you wish to promote a Webinar we recommend at least a 2 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site and prominent Newsletter promotion and Promotional News & Editorial.

This coupled with the new capabilities 5G brings opens up huge opportunities for both network operators and enterprise organisations.

We have a Webinar Business Booster Pack and other supportive programs.

We look forward to discussing your campaign goals with you.


Sam Varghese

website statistics

Sam Varghese has been writing for iTWire since 2006, a year after the site came into existence. For nearly a decade thereafter, he wrote mostly about free and open source software, based on his own use of this genre of software. Since May 2016, he has been writing across many areas of technology. He has been a journalist for nearly 40 years in India (Indian Express and Deccan Herald), the UAE (Khaleej Times) and Australia (Daily Commercial News (now defunct) and The Age). His personal blog is titled Irregular Expression.

Share News tips for the iTWire Journalists? Your tip will be anonymous




Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News