Security Market Segment LS
Tuesday, 05 October 2021 09:58

Facebook starts to come back, but Cloudflare tells the tale Featured

By
Facebook starts to come back, but Cloudflare tells the tale Pixabay

Web infrastructure and website security company Cloudflare has beaten Facebook to the punch in explaining why the latter suffered a major outage earlier today, with the site disappearing from the Web at about 2.30am AEDT.

Facebook's blog and its media statements site both have nothing to say about the outage. The behemoth's Twitter accounts has two tweets, one about two hours ago, and the other six hours ago.

Essentially, nothing seems to have happened at the company's headquarters despite reports of a border gateway protocol snafu appearing on every site belonging to world+dog and an outage that lasted more than six hours.

facebook twitter

Stuck in the past... the Facebook Twitter account at the time of writing.

But Cloudflare's Tom Strickx and Celso Martinho issued a blog post at 21.28 UTC (10.28am AEDT), explaining the ins and outs of how Facebook and its fellow travellers Instagram, WhatsApp and Oculus Web went into a black hole, and how they were slowly returning to the Web.

The two said initially, when they opened an internal incident headlined "Facebook DNS lookup returning SERVFAIL" they were afraid that something was wrong with their DNS resolver 1.1.1.1.

But they said they quickly realised that something much bigger was going on when "social media quickly burst into flames, reporting what our engineers rapidly confirmed too. Facebook and its affiliated services WhatsApp and Instagram were, in fact, all down.

facebook media releases

Nothing to see here... the Facebook media statement page where the last post was on 9 September.

"Their DNS names stopped resolving, and their infrastructure IPs were unreachable. It was as if someone had 'pulled the cables' from their data centres all at once and disconnected them from the Internet."

Strickx and Martinho said at 1658 UTC, they noticed that Facebook was no longer announcing its routes to its DNS prefixes. "That meant that, at least, Facebook’s DNS servers were unavailable. Because of this Cloudflare’s 1.1.1.1 DNS resolver could no longer respond to queries asking for the IP address of facebook.com or instagram.com," they wrote.

On checking their database of BGP updates, the duo found a number of routing changes made by Facebook at about 15.40 UTC.

"Routes were withdrawn, Facebook’s DNS servers went offline, and one minute after the problem occurred, Cloudflare engineers were in a room wondering why 1.1.1.1 couldn’t resolve facebook.com and worrying that it was somehow a fault with our systems," they said.

facebook media

The Facebook media blog... again no information about the outage.

Due to this, resolvers across the globe stopped resolving their domain names, they said, providing a detailed explanation of how the DNS system works.

When Facebook stopped announcing its DNS prefix routes through BGP, Cloudflare's and everyone else's DNS resolvers had no way to connect to their nameservers.

"Consequently, 1.1.1.1, 8.8.8.8, and other major public DNS resolvers started issuing (and caching) SERVFAIL responses," Strickx and Martinho said.

"But that's not all. Now human behaviour and application logic kicks in and causes another exponential effect. A tsunami of additional DNS traffic follows."

They said at about 21.20 UTC, the availability of the DNS name facebook.com on Cloudflare's DNS resolver 1.1.1.1 returned.

"Undoubtedly Facebook, WhatsApp and Instagram services will take further time to come online but as of 21:28 UTC Facebook appears to be reconnected to the global Internet and DNS working again," the pair said.

"Today's events are a gentle reminder that the Internet is a very complex and interdependent system of millions of systems and protocols working together. That trust, standardisation, and co-operation between entities are at the centre of making it work for almost five billion active users worldwide," they concluded.

Read 2031 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here

SONICWALL 2022 CYBER THREAT REPORT

The past year has seen a meteoric rise in ransomware incidents worldwide.

Over the past 12 months, SonicWall Capture Labs threat researchers have diligently tracked the meteoric rise in cyberattacks, as well as trends and activity across all threat vectors, including:

Ransomware
Cryptojacking
Encrypted threats
IoT malware
Zero-day attacks and more

These exclusive findings are now available via the 2022 SonicWall Cyber Threat Report, which ensures SMBs, government agencies, enterprises and other organizations have the actionable threat intelligence needed to combat the rising tide of cybercrime.

Click the button below to get the report.

GET REPORT!

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

BACK TO HOME PAGE
Sam Varghese

Sam Varghese has been writing for iTWire since 2006, a year after the site came into existence. For nearly a decade thereafter, he wrote mostly about free and open source software, based on his own use of this genre of software. Since May 2016, he has been writing across many areas of technology. He has been a journalist for nearly 40 years in India (Indian Express and Deccan Herald), the UAE (Khaleej Times) and Australia (Daily Commercial News (now defunct) and The Age). His personal blog is titled Irregular Expression.

Share News tips for the iTWire Journalists? Your tip will be anonymous

WEBINARS ONLINE & ON-DEMAND

GUEST ARTICLES

VENDOR NEWS

Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News

Comments