I mean… not only is it not very concerning, I barely noticed. If not for news about it here on fediverse, I might not have known. I guess I dont visit the corpo internet all that much.
it’s very concerning, but what are the alternatives?
A lot of people are getting this wrong. Cloudflare’s system is distributed the way it is to make DDoS attacks against individuals hosts and routes much more difficult. The goal is to block the traffic closer to the source to protect downstream routes. This morning it arguably worked as intended. The outage was around 30 minutes, and there was still intermittent connectivity during that time. Cloudflare didn’t collapse, it was struggling to separate legitimate traffic from DOS traffic, and throwing circui breakers and isolating nodes, because it was such a massive attack.
I remember experts saying 5 or 10 years ago that the increased standardization and centralization of the internet would lead to more frequent and widespread internet blackouts.
First AWS, and now this. It looks like they’re right.
Two things happen when we centralize. Doesn’t matter if it’s big business or infrastructure.
-
Profits go up for the controlling few
-
consumers get fucked.
We get fucked when things go wrong, the system fails, our data gets hacked, our power goes out, our rents go up, insurance rates go up… etc etc. MegaCorps all say sorry, give us 50¢ off our next purchase and a free credit check, and carry on while we eat the losses and increasing costs.
-
Don’t forget the Azure/Intune outage not one week after AWS, too.
The outages are almost beginning to feel deliberate at this point.
Obviously it is concerning. We have just been given a window of a glimpse as to what would happen if one service in which so many things rely on, gets messed up. Like today I was having trouble logging into my bank because guess what, they rely on CloudFlare.
I’ve read individuals relying on services provided by CloudFlare, their processes were interrupted.
I know that CloudFlare has a purpose and its purpose is being served, but there’s a reason why people love and should embrace the idea of multiple alternatives and hate monopolies.
It would be like, if Comcast as an ISP has a blackout, do you know how many subscribers they have? Some people in certain areas are all that they have so the blackout would knock them offline for however long. That’s why alternatives are important.
Amazon the other week, then this. Really does show how vulnerable much of the net is
The snark of the following comment is not directed towards you, OP, but at the tech industry at large.
What I don’t understand is why people are still surprised when this shit happens. Today, cloudflare takes down half the internet, last month it was AWS. Crowdstrike did it last year even more severely. Akamai has also caused major issues like this before, as has Google. M365/azure outages barely get reported on because they are so frequent. Yet, they are all still being used to hold up most of our infrastructure. Every single company I’ve done IT for has used at least one of these companies for critical infrastructure. There just aren’t any other realistic options due to the refusal of non IT people to learn about IT.
If you try to use something other than one of the big companies, you’re hit with one or more roadblocks.
-
You “don’t have the budget” to selfhost. Bean counters would rather pay $100 a month indefinitely than $5k to buy new hardware that will save $1000 a month for years.
-
No approval for non giant corpo option, because using AWS is cheaper and has brand recognition. This is due to the same economics and myopia that caused Walmart to be one of the only places you can get groceries.
-
There is no other option. Every year that goes by, more small companies get gobbled up by big tech M&A. Unless your company opts to create its own implementation of a service/software, you’re stuck with one of only a few options, even if you could get the approval to use something not run on big tech.
-
Even if you manage to jump all of the previous hurdles, the Internet connected software you’re using probably relies on big tech infrastructure too. Every company has to navigate all of these hurdles for every saas/infrastructure implementation, and the only ones that successfully do it have to have leadership that not only understands why the decisions have to be made, but also need to be willing to accept the extra cost. Anyone that has dealt with upper management knows that this is exceptionally rare.
So what we are left with is a system that every professional knows is deeply broken and monopolized. The people that actually make the final decisions are largely ignorant and unwilling to invest money in fixing it, instead choosing short term savings and lack of commitment over long term security and continuity.
-
It just means the internet is built on a very flimsy stack of technologies and any of them failing causes huge downstream issues. We saw that with AWS, and now with Cloudflare.
It’s only concerning if there are no alternatives, but as it stands there are other companies that all of these websites could have done a failover to when both AWS or Cloudflare went down. But they decided that their websites having a single point of failure was worth the risk over paying for having a proper backup system ready to go.
I now imagine all the websites to fail over to the same backup services, effectively ddosing them and creating a chain reaction :D
Relevant XKCD, as always:
XKCD//2347
(Joke stolen from another post that’s since been deleted, so reproduced here.)
And azure also went down too
The fact that Cloudflare controls half the web is concerning both for unintentional crashes like this, and for something even more insidious; what if they’re coerced to cause an intentional outage should cyber war ever break out? An intentional outage for half the web in a cyber war would be devastating to put it nicely.
Being a good CDN is an expensive exercise that requires the ability to run POPs in many countries around the world.
Cloudflare captured the market by basically being simultaneously much cheaper, better distributed and ultimately better performing than the incumbents at the time (Akamai and Limelight IIRC)
The rest of the story is capitalism doing capitalist things
People have other things to worry about. It’s concerning but there is a barrage of shit going on that this barely registers. And companies will always choose what’s cheap in the short term. They believe the risk of something going wrong is small enough to warrant the possibly large impact. It’s like that everywhere: in the car industry, chocolate industry, clothing industry, and so on. There’s always one seemingly small decision that could fuck up the entire company but isn’t worth investing in in the short term.
I wish cloudfare (whatever it is and whatever it does) had more reach and went down for longer. For so long that competitors would be considered. But alas…
I didn’t even notice that it went down.
It took down a fifth of the Internet, not half.
I found two websites that didn’t work, that’s it.
A third of the “top 100” were in that 1/5th total. Most websites I personally wanted were down, including lemmy for me.
You must not have been paying attention when that one Java programmer quit and took his code with him.
I find it at least concerning for CloudFlare’s change control process. Apparently some new traffic analysis config took half the web? Maybe test things a little more?
It’s good that it goes down once in a while, so that people notice.
