Podcast
Root Causes 94: Revocation Checking Through OCSP and CRL
Hosted by
Jason Soroko
Fellow
Original broadcast date
May 26, 2020
One essential portion of the certificate lifecycle is the ability to revoke certificates. Public SSL certificates use a pair of mechanisms to communicate this revocation status to client machines, CRL and OCSP. In this episode we explain how these mechanisms work and some of their strengths and challenges.
Podcast transcript
Lightly edited for flow and brevity.
TCTim CallanToday, we are going to talk about revocation checking mechanisms in the world of public SSL, which is to say CRL and OCSP.
TCTim CallanYeah.
TCTim CallanSo, let's go back—revocation right. There are good reasons why we need to have revocation. Those can be owner-initiated revocation, which is to say, I had this cert, but I'm not sure, I think maybe the private key was compromised so I just want to revoke it and get a new one, or I had the cert, but I know I'm not going to use it anymore and just in order to keep things clean, I want to revoke it. There's good reasons why the private certificate owner needs to be able to do that. There are also reasons why the CA needs to be able to do that. We think the certificate is being used incorrectly, the certificate was issued in error, the certificate was etc. and then you need to be able to go revoke. So, the mechanism, therefore, is that there are a list of certificates that are known to be revoked by the CA, right? The CA is this single hierarchical authority that sits at the top; thus, the name CA and the CA says, here is the list of certificates that are revoked. Now you need a way that the client machine that's out there in the world can compare to that list to say, is this certificate revoked, or is this certificate not revoked, and the first thing that the world had was CRL, which stands for certificate revocation list and CRL is just basically a big list of revoked certificates. And there are things that CAs can do to make that system a little more efficient like once the certificates would have expired, you take them off the list, like they can break the list up into shards. So. instead of one list, there's multiple lists and there's a way to get the right list so those are things that they can do. But at the end of the day, as you might imagine, it's a heavy system. Right? It's a heavy process and it's an especially heavy process, as the number of certificates in the world continues to increase by leaps and bounds. And so, a long time ago, well, more than a decade ago, the industry had created what we call OCSP, which stands for online certificate security protocol. And what OCSP does - - status protocol. Online certificate status protocol, excuse me. And what OCSP does, is it's a one-time check of the one cert in question. So, there's a SaaS system that lives, that takes a query on a cert and returns a status. This is revoked. This is not revoked. And OCSP is the more modern and the more current version. CRL is still available, but I think - - I don't think CRL is - - we couldn't use CRL as a cornerstone for our security. Let's put it that way. So go ahead, Jay.
TCTim CallanRight and some of that comes to some of the baggage with OCSP. So, you might say, okay, great well, that sounds lovely. Right? That sounds perfect. I think that's a great system and in principle on paper, it looks like it would be a great system. Unfortunately, OCSP is not perfect, either. The problem with OCSP has to do with what happens if that individual query doesn't return. Right? So, you've got lots and lots of computers all over the world, hitting OCSP responders that themselves might be all over the world and they're asking for the status of a cert and they want an answer back right darn away because if you don't get your answer back right away what are you going to do? Are you going to stop loading the page or you're going to keep loading the page, right? And if you stop loading the page then that's a problem because people have slow internet usage and it sucks or if you stop that process whatever it is, and if you keep going with that process or keep loading that page, then you potentially have the problem that a certificate validity - - the certificate validity status might not be obtained and a potentially revoked cert might be not known to revoked. If that makes sense.
TCTim CallanRight.
TCTim CallanYeah, so let's walk through that because it's not necessarily intuitively obvious until someone explains it. But I think it's a very, very important point. So, you go, okay, we will have a fail-soft system. So, what will happen is if the certificate is revoked, then what we will do is - - I'm sorry. If we don't get an answer, then what we will do is we will fail soft. So, the problem with that goes as follows.
Let's look at the scenario where I do steal a key. So, I say I'm a sophisticated attacker. I steal a key. I hijack your DNS. I send you off to my fake website, that looks like the real website, including the URL. I stick your certificate on there. Right? Now, the problem with that at this point is that if that certificate - - so, the owner of the certificate finds out that this has happened. The owner of certificate revokes the cert. Now, if my captive audience that I've stolen their DNS comes to my site and they go off and they check for certificate revocation once I own your machine anyway, I can arrange for that OCSP response not to come back. And under those circumstances, if the software fails soft, then it says, okay, let's proceed, which means that in the exact scenario that the system is built to defend against, that is the scenario where the system ops not to work. So, it’s equivalent to a seatbelt that breaks when you get in an accident. Why bother? I wear my seatbelt and my seatbelts there across my chest and everything's fine all the time while I'm driving around but the instant that I get into an accident, the seatbelt breaks and offers me no protection, on the one occasion that I need it. And that is - - that's the scenario we just described with OCSP.
Let's look at the scenario where I do steal a key. So, I say I'm a sophisticated attacker. I steal a key. I hijack your DNS. I send you off to my fake website, that looks like the real website, including the URL. I stick your certificate on there. Right? Now, the problem with that at this point is that if that certificate - - so, the owner of the certificate finds out that this has happened. The owner of certificate revokes the cert. Now, if my captive audience that I've stolen their DNS comes to my site and they go off and they check for certificate revocation once I own your machine anyway, I can arrange for that OCSP response not to come back. And under those circumstances, if the software fails soft, then it says, okay, let's proceed, which means that in the exact scenario that the system is built to defend against, that is the scenario where the system ops not to work. So, it’s equivalent to a seatbelt that breaks when you get in an accident. Why bother? I wear my seatbelt and my seatbelts there across my chest and everything's fine all the time while I'm driving around but the instant that I get into an accident, the seatbelt breaks and offers me no protection, on the one occasion that I need it. And that is - - that's the scenario we just described with OCSP.
TCTim CallanYeah.
TCTim CallanIt's a scaling problem. It was fine in 1996. But not anymore. We're just so many orders of magnitude bigger and how we use these things.
TCTim CallanWho's going where? Absolutely. Right.
TCTim CallanKnowing us, we probably will. But not today.
TCTim CallanAnd even if you were like, even if we were, let's say we could snap our fingers and know that all of the software was working correctly and in the same way everywhere in the world, that wouldn't solve all of the problems with OCSP stapling, because one of the big concerns that people have about OCSP stapling is crypto agility. So, I have something, I have an emergency situation and I have to do something to my certs right now.
TCTim CallanWell, guess what, right, because, you know, turns out someone stands up at a hacker conference and says, guess what, I blew all this crypto up and everybody goes, oh my God, I got to stop using RSA today, right? Or somebody says, I don’t trust the CA anymore, right? We've seen this with CA's DigiNotar, Certinomis, Symantec, right, stopped being trusted CAs and people add those certs had to move off those certs. So, these situations happen in the real world. And by the way, you know, if you've stapled - - now if you've used OCSP stapling and systems, you don't necessarily - - you can't just swap those certs out, they're not all interchangeable anymore. To further complicate things, what happens if I write something and it sits around and it's a legacy system and I've got a certain cert and I keep renewing that certain cert everything's fine and then one day, I decided to switch certificate vendors because the other guys got better tools, or he’s given me a better price, or the first one went out of business and I don't even realize that that decision was made until my systems are failing and I don't know why my systems are failing, because I installed the cert, it's right there. And so, there are a lot of concerns about OCSP stapling, that it's really not agile, that it's really not crypto agile, and then it's really not robust.
TCTim CallanYeah.
TCTim CallanYes.
TCTim CallanIt is. It is. And I think that's definitely a net safety improvement. I'm not sure that that gets us all the way to home plate, though, right? Because, okay, you say, if I take what used to be six-year certs and I turn them into one-year certs, I've reduced the attack time to 1/6 of what it was. Okay, that's good. If I take one-year certs, and I turn them into 90-day certs, I reduce the attack time again. But gee, man, if I steal your private key, I don't need very much time to do something bad. 90 days is plenty. So, a 90-day cert doesn't get us there. Maybe a two-hour DevOps cert gets us there. Maybe. But that doesn't work in the world of static web servers. That doesn't work for my cert that's sitting on my public facing web page, or even on my cert that's inside my own owned hardware clients - - you know, in-house systems, that that doesn't solve it for them. So, we still aren’t at the point where we can live without revocation. Even with the kind of shorter lifespans that we're talking about, right? If there were a truly automated system - - if I had a truly automated system, where all of my systems internally could get a one-day certificate updated every 12 hours, continually, right, 365 days a year, then, okay, now we're getting there. But I think that's the kind of thing we need to be talking about.
TCTim CallanYeah, this has been one of the thornier and trickier issues, like all the way back in the very beginning, when we first started to say, we're going to use, you know, PKI-based certificates in a public set of circumstances, like the internet, people all the way back then thought about this problem. This revocation problem. And there aren’t a lot of sort of fundamental ideas of computer science, that somebody posed in the early 1990s, that still in the year 2020, aren’t really worked through. That's kind a rare animal. Right? And, and it's a thorny one, it's a tough one, and the amount of intellectual horsepower that has gone into this is very high. So, it's not unresolved through lack of attention, or lack of focus, or lack of trying, it's unresolved because it's genuinely really hard.
TCTim CallanYes.
TCTim CallanRight. Right. And there's that right, like you were saying, and we talk about this a lot, right. One of the things about our standard PKI systems, our x.509, and our certificate types that we all use and we all know starting with TLS, one of the big, big strengths of it is this amazing, ubiquitous support. That I can reliably go buy any hardware or software application from any vendor in the world, or subscribe to any SaaS service from any vendor in the world and have damn near 100% confidence that my off-the-shelf SSL certificate is going to work, right? That is hugely important. And but what's the downside? The downside is agility, right? The downside is we have a situation like this and it's, it becomes very difficult to change these basic foundational elements of the system.
TCTim CallanRight and you can imagine something like that makes more sense, right? Where especially the intermediate, because the number of intermediates you have in the world is I don't know how many orders of magnitude lower than the number of end leaf certs you have in the world, but a lot. And that's where if you could see that somebody malicious was doing something, you know, in an organized systematic fashion and this is not unlike what we talked about with Kazakhstan last year, and, you know, you start to imagine state actors and high value criminal organizations and things like that, at that point, you can say, okay, maintaining a rigorously checked CRL for those intermediates, is scalable, can be fast, and can really matter.
TCTim CallanYeah, it works. I think it would be fair to say it works most of the time. But, not the damn near 100% of the time that I said earlier on. Right? And that's, of course, what we all want to be better. Like most of the time in the world of security, most of the time isn't all that good. Like you want to do better than that. So, this is definitely an area where I think everybody feels some pain.
TCTim CallanAll right. Well, thank you for listening listeners. Thank you for talking to me, Jay. I always enjoy it.
TCTim CallanAnd this has been Root Causes.