Podcast
Root Causes 448: The Privilege of Being a Public CA


Hosted by
Tim Callan
Chief Compliance Officer
Jason Soroko
Fellow
Original broadcast date
December 17, 2024
We go over Tim's September 2024 keynote speech at ENISA CA Day, "The Privilege of Being a Public CA."
Podcast Transcript
Lightly edited for flow and brevity.
And this, of course, had slides, so you're not going to get to see the slides, but I'll try to articulate this in a way that it comes across fine, and it’ll make sense. So with that, the privilege of being a public CA. I started actually talking about delayed revocation. So let's start there. So let's talk about delayed revocation. Jason.
And there's a word we use in the web PKI community. It's called delrev – d-e-l-r-e-v. That just means a delayed revocation bug. A Bugzilla bug that is occurred when a mandatory revocation doesn't get done on time and we've done a number of episodes on revocation. So if you want more details on that, go back and read those. And I want to focus in on what I call deliberate delayed revocation and that's kind of my coinage, but I think it's pretty obvious what that is. That's a delrev, where the CA has the technical and procedural ability to do the revocation and is aware that the revocation must be done, and yet, by choice, doesn't do the revocation. I want to distinguish deliberate delrev from accidental delrev.
The accidental delrev would be where the delay in revocation occurs unwittingly. Maybe there's a software error, or kind of an undetected procedural error, or possibly the CA doesn't realize that there's a bug and doesn't get the revocation done because somehow they don't know that that's supposed to happen. And so, accidental delrevs are their own problem, but they're a different problem from deliberate delrevs because an accidental delrev, if anything, is a competence problem. A deliberate delrev is a choice. It is a decision making problem, if you will. So the snapshot in September 2024 when I took this, as of 9/20/24, which is when I sat and did this research, not too long before the presentation occurred, there were 70 open bugs in Bugzilla. Of those, 22 were delayed revocation bugs and of those, all but two were instances of a deliberate delrev. So 29% of the open Bugzilla bugs, nearly 1/3 were deliberate, delayed revocation. Of the other two, one of them was an accidental delrev. It was what I talked about before. In that case, it was a procedural error, not a software error and then there was one meta bug, which was just kind of gathering together all the delayed revocations, because there was so much going on in that space. So 20 deliberate delrevs, which is pretty incredible, if you think about it because - -
And the point here, though, is that this wasn't a speech, and this today isn't an episode about delayed revocation. Rather, it's an episode about doing the right thing. And so delrev is just an example.
So in addition to delrev, there's other things. There's refrev, which is refused revocation, which is where the CA just decides not to do the revocation. There's failure to lint. There's failure to report problems when they do occur. And then another one that's rampant is failure to answer corrections or answer questions, correct previous errors or learn from other CAs.
I get questions about my behavior and why the bug answered and or why the bug occurred, and I just don't answer them, or don't answer them on time, or give vague, elusive answers or correct previous errors. People have an error. There's an error. They know it's an error. They're aware of the fact, and they don't go and do the work to make it not happen again. They knowingly just don't do it. Or a big one is failure to learn from other CAs. CA A has a problem, and then a year later, CA B has the exact, precise same problem, even though the whole community already watched the CA A walk through this, have a root cause analysis, identify action items and apply those action items. And yet, other CAs didn't know of it, didn't act on it, didn't apply the same learnings and didn't themselves from having the same problem.
So the question I posed is, is this what we want? Is this what we want for our community? Is this what we want for our industry? Is this what we want for the group of organizations that are supposed to be the gatekeepers of identity and trust for the entire world? And if you think about it, it is an incredibly rare thing that has been given. I don't even know the answer to this, Jason. So you and I are both guessing. But how many technology companies do you think are in the world?
These are rules we have that are hard rules in the CA/Browser Forum. There is a rule requiring that CAs do pre-certificate linting, pre-issuance linting. That's crazy. Linting is such a wonderful best practice. It is a silver bullet that just knocks out so many sources of misissuance. It's so powerful and free linters are available that people put in open source that they do just out of the goodness of their hearts. Why on earth would any CA not do pre-issuance linting? Why on earth does there need to be a rule that forces them to do it. There is a rule that requires that CAs support automation. Automation, Jay. Are you a little bit of a fan of automation maybe.
And that was what I just wanted to leave that group with. There were a lot of people. There were, I think, 300 people in the room. There were 1000 people online. It was the biggest audience I was ever going to have. And I said, for this audience, what is the one thing I want to say to this audience? And that's what I said to them. Come on. Come on people. Have some pride, have some dignity, have some self-awareness. Recognize that you have been granted a privilege that almost nobody on the planet has been granted, and at least try a little bit to live up to it.
If we're a complete mess, as an industry, the way, the way that things were, say, 2012 and before going into post-quantum era, we're finished. Like we're gonna have no trust on the internet won't exist.
So like right now, I'm really glad that we're in a place where I don't hear any of the CAs yet, complaining about MPIC. As an example.
The industry kind of came together and said BGP attacks are real. None of us wants to be misissuing due to a BGP attack. We'll do what's necessary and put in the rules around MPIC, and then we're going to implement it. That's the process where all the CAs are in right now, and I think that that's a huge difference from where we were in the past, where I bet you, I'll bet at least $1 and a half, there would have been a CA or two who would have been all whining about that and not implemented. And it would have been bad.
One of the analogies that I give a lot, and I may have done this on the podcast already, but I think it's appropriate, is if you think of the WebPKI as a wall around a fortress, every CA is a gate in that wall. It's how things go in and out. Every CA is a gate. And to prevent invaders from coming through your wall, you have to prevent invaders from coming through your gate. Now, all of the gates are equally effective for an invader, so you don't have to hit the big, popular gate that everybody has heard of, like Sectigo or, Let's Encrypt, you can go hit the tiny little CA located somewhere in Eastern Europe or Asia that is weak and sloppy and has poor security and still get through the wall. And so that's one of the problems. You've got folks who aren't signed up to be pursuing rigor and security and exactitude at the level that is required to be one of the gates through the wall that is the WebPKI, and yet they still are one and that's where we have our trouble.
And that's the castle you're talking about. And so what Moxie was talking about this way back in 2011 saying, maybe we should give users the ability to make it easier to switch in and out who they trust. And I think that with enough time having passed, I think that we should be pleased that we're far from where we were in the 2011 time frame in terms of just how CAs have evolved, how the CA/Browser Forum has evolved. Things have gotten so much better. So we're not in that dire circumstance that the world was in back then.
But on the other hand, I think that that idea of trust agility never went anywhere for the same reasons that the green bar has gone out of the browsers. It's just any of these things that you have to have a propeller on your head to understand what it is you're actually doing doesn't really add security. And so I guess what I'm really trying to say, Tim, is the onus really is on the CAs to do the right thing.

