Root Causes 259: What Went Wrong with the Twitter Blue Check Marks

Hosted by

Tim Callan

Chief Compliance Officer

Jason Soroko

Fellow

Original broadcast date

November 30, 2022

The Twitter authenticated identity blue check marks made a big splash and then quickly went away. In this episode we explore the intent of these check marks and why they failed. In particular, we detail the challenges involved in authenticating and vouching for the identity of an individual or organization.

Podcast Transcript

Lightly edited for flow and brevity.

Tim CallanLet’s talk about something that’s been very big in the news and, obviously, we are gonna have our particular angle on it. They came and they went but we are talking, of course, about the Twitter authenticated blue checkmark. So, this didn’t turn out very well, did it?

It’s strange. Originally, the blue checkmark was something that mysteriously appeared after I guess Twitter would reach out to certain kinds of people, celebrities, companies, etc. I don’t even know if they had a set of rules in place. Nobody really knew what they were. But it seemed to be a bit of a place of pride to have one. It kind of meant you were somebody as decided by somebody at Twitter, and I guess rules changed once Elon took over. So, I think the point of it was supposed to be that it was supposed to vouch for the authentic identity of somebody famous. And yes, what’s your definition of famous. But, you can see where this makes perfect sense. If I’m some kind of a celebrity or a famous politician or a very powerful person in industry that somebody going on Twitter pretending to be me, putting up a picture of me – which are trivially easy to get if you are that famous – and then saying things that people are gonna think that I’m saying would be bad. And you can see a lot of reasons why it would be bad and so this blue checkmark was supposed to be, authenticate, that this was authenticated and that this was really this person or this company or something along those lines.

And then so, of course, in the early days of the new era of Twitter, the company announced that everybody could just get a blue checkmark for whatever identity they want and certainly not surprisingly to me and I think we’ll return to this, it turned out that this thing was very widely abused and so we had an account that purported to be Coca-Cola saying that Pepsi was better. We had an account purportedly to be I believe it was Pfizer saying that they were gonna make certain products like insulin for free and various other things. And the consequences of this were really bad. Some of them were reputational damages. Some of them actually cost companies money because things happened like share prices dropped. And so, the stakes were really high and then after a couple days, I think recognizing that the whole thing was ill-formed and unconsidered and not put together in a reasonable way, Twitter dropped it. Dropped the whole thing.

And the reason I say not surprising and what I’d like to focus on, Jason, if you don’t mind, is because authenticating the identity of an individual or an organization or an individual in alignment with an organization, is not at all a trivial task. And in fact, there is a whole industry – I happen to be part of it. It’s public CAs. – that have been committed to this for decades. And if you go back to the very beginning – why did we create SSL certs at all back in 1995? It was to vouch for the identity of an organization on the worldwide web. So it was in order to say, this is really this bank. Or this online retailer. Or this online brokerage. And to do that, there was an exercise that CAs would through that was codified where they had specific rules scheduled and they would go through their rules and they would ensure, they believed to their best of their ability or to a sufficiently high degree of accuracy they believed at the end, that they could really say this really is – name your favorite bank. And all of that, from what I can tell, was just missing. Completely missing from the Twitter blue checkmark.

Jason SorokoProvisioning and really understanding and verifying who an identity is is hard work and when your bank provisions you to open a bank account, that’s hard work and when your cell phone company provisions you to be a customer, that’s hard work and that is why those kinds of provision databases of real provisioned identities that are tied to a number of important attributes, everything from potentially government-issued IDs to credit card numbers to things that really are tied to you, that hard work is worth a lot. It’s worth a lot because now it allows people to do business with you. It’s worth a lot because of the cost of doing the provisioning and therefore, federating into those kinds of accounts that have been provisioned is really important and in the world of CAs, the entire basis of certain types of higher-level certificates are based on it. And it's hard work. Twitter didn’t do the hard work and now we get to see what it’s like when a social media company – and all of them do this to some degree, Tim. All of them are usable for federation into all kinds of logins and I think some of them ended up with the fantasy that this very light touch form of provisioning that they do for their social media accounts is actually people who pretend that they can vouch for people and now we see how hard that actually is.

Tim CallanIt’s hard under even the best of circumstances. So, when we sit around in the CA world, we talk about sort of fundamental hard problems that are unavoidable, like there are multiple individuals in the world with the same name or there can be multiple companies with the same name. There can be multiple companies with the same name in the same city. If I create Tim’s Carwash in San Jose, California, that doesn’t mean that there isn’t another Tim’ Carwash in Santa Monica, California. And so, those sorts of things are fundamentally hard. Or if we go to the example of Donald Trump and they are putting a blue checkmark saying this is really Donald Trump, well there are other people in the world named Trump as a last name and I’ll bet you that at least one of them has the first name Donald. So why can’t that person go to Twitter and get a blue checkmark that says Donald Trump? And in the world of a true ubiquitous authentication platform kind of scenario, the answer is they can, and they have to be able to.

And so now if you start to say well this is a problem because you’ve got somebody out there with this name and it’s also the name of the famous person, ok, that’s a problem that’s bigger than Twitter or a social media platform. That’s a problem that’s bigger than a certain authentication scheme. That’s a problem that has to do with the fact that the way we name ourselves is fundamentally not unique and that’s a fact that you have to live with and until you live with that fact, any scheme like sticking blue checkmarks on accounts is gonna be fundamentally subject to flaw. But, of course, this didn’t even get close to that. Like there is a lot of ways that this particular strategy could have been much stronger and much better, and it would have been using the benefit of nearly 30 years of public CAs as a starting point.

But in this case, it was just hurried to market, extremely hurried to market and nobody did that and you could imagine a Twitter or another major social media platform, a Facebook, etc., sitting and doing the homework and doing it right, possibly attaching themselves to something like certificates. They could require an EV certificate or they could do the equivalent of an EV authentication. They could require an eIDAS certificate. There’s a number of ways they could do it which at least would vouch to the degree that a human in society can be shown to have a certain name that that human has that name. Now is it the human with that name who is the same individual with the same DNA as the person you think of with that name who is famous? Not necessarily. But that’s even another level of difficulty that needs to be dealt with. But again, the blue checkmarks didn’t even get there. They didn’t even get close.

Jason SorokoTim, I think you are gonna actually start to see more and more of this because when you take a look at decentralized identities and you take a look at various schemes of holding a wallet of attributes about yourself in order to be able to engage in activities with government and companies and even other people, even things like decentralized commerce. Peer-to-peer lending. All these kinds of things that our generation, Tim, we didn’t grow up with this but there’s a generation right now that never even knew a world without it and so, therefore, provisioning identities in the old-fashioned brick and mortar way, which is what we are all used to, is not gonna be the way it’s done in the future and what you just saw with Twitter is a large social media company flexing its muscles and ultimately failing horribly. This is a lesson to anybody who is gonna be getting into the decentralized identity business about how not to provision people.

Tim CallanAnd the good news is, from a computer science perspective, it’s actually very well solved. You can give people a cryptographically secure, unique identity that they can have that is only theirs such that if they come and they act online they are unambiguously that digital actor. And that part of things is very solid, and it’s well understood and there’s lot of ways to implement these things and, serious people who do their homework can do this correctly and get it right and it can be rock solid.

Jason SorokoThose of who live in the enterprise authentication world know exactly what you are talking about, Tim. We live and breathe that stuff every single day.

Tim CallanThis happens all the time. When I come into my systems at work, it is uniquely me. The challenge comes when we start to think that that very secure, unique computer science-based authentication can translate to saying I am certain that this is this individual or this organization or this company and when you do that you are suddenly getting out of the realm of clean mathematical cryptography and you are getting into the realm of the murky systems that operate our society and that’s where you have these problems. And again, this is the bridge that CAs have been crossing dealing with all of the difficulty and the nuance and the mud and the hairiness for decades. Because there was an understanding that there was value in being able to cross those two things and that’s why CAs really were created so that they could do that. But you get in trouble when people who don’t understand the nuances of these things start thinking this is pure and absolute.

Jason SorokoExactly. And I think that there has been a generation of people who really do believe that the sun kind of rises and sets by these large powerful social media companies and the enormous personalities that are behind them and we are seeing exactly what happens, Tim, when you start playing in hard things. And it’s not necessarily about the technicalities. It’s about setting up the correct policies and doing things correctly. I tell you, I spent a large chunk of my adult life in and around the verification process with CAs and I tell you, still to this day, we have sass about it and kudos to the people who do it daily for a living because it’s hard work.

Tim CallanIt is. And not something you can just whip out in two days. I think we saw that in a big way.

Jason SorokoTim, my favorite, favorite fake Twitter account was, of course, the banana, the fruit company, Chiquita Bananas, somebody had set up a fake Chiquita Banana corporate account and had gotten that $8.00 badge and had said, we have now taken over your country or some ominous message like that and the real Chiquita Banana Corporation said, no, listen, we absolutely guarantee we haven’t taken over any countries. The last time we did that was 1954, which was true with the country of Ecuador. That was my favorite of all time.

Tim CallanI like that. That’s good. Good job. Good on you, Chiquita, having a sense of humor about these things.

So, right. But what if someone takes that seriously. Probably a good place to leave this one for now. I for one wanted to point out the difference between being able to computer science-wise set up a blue badge that appears on a certain account and being able to make claims about who really controls that account. And I think we saw that fall apart in a big way, but this isn’t the last time this is gonna come up because, you know, how do you map an online entity or an online brand to something in the physical world. It’s not a problem that’s going away.

Stay informed with expert insights

Subscribe to Root Causes for engaging discussions on PKI, digital security, and best practices for protecting your organization's critical assets. Don’t miss an episode!