Root Causes 437: Don't Blame the Linter

Episode Transcript

Lightly edited for flow and brevity.

• 

  Tim Callan

  I want to talk about blaming the linter. We've done episodes on this in the past. Basically, what a linter is, at a high level, is it's a very objective check that you put on a piece of software, or, in the case of public CA, a certificate that looks for certain things that must be correct and lets you know if they're not.

• 

  Jason Soroko

  Tim, I came up with an analogy. For those of you who don't live 24 hours a day in the CA world, you might have brought in your car at some point because there was a check engine light, and the technician may have put in the OBD2 port, a diagnostic and the computer is looking at all the configurations of your car and saying, yep, your check engine light is on because you left the gas cap off. There's the problem. Go fix it, Mr. Technician, Mrs. Technician. So to me, it's a diagnostic for certificates.

• 

  Tim Callan

  I think that's a great way of putting it. There's a bunch of things it's looking at, and it's trying to make sure these things are right. These are all objective things. That it's real simple. It's right or it's wrong. There's no real ambiguity. It's not a matter of opinion. Linters are a best practice. They're pretty new. Five years ago, most people weren't really linting, or if they were, they were homegrown linters. One thing that's happened in recent years is that there have been a bunch of community, kind of open source, public-based linters that have been created. A popular one is called ZLint. Another one that's emerging now pretty big is Pkilint. There's others being worked on. These linters are basically there to check to see if a certificate is correct. Now, these linters could be applied in a variety of ways. They could be applied as post certificate linters. You go and you run a mass of certificates, and you try to see if any of them have errors, or they could be run as pre-issuance linters, where you take a potential cert, and before you push it through, you throw it against the linter and see if it comes out the other side. If it doesn't, something's wrong. It gets blocked. You fix it. If it does, then it goes and it releases. Sounds like a best practice.

• 

  Jason Soroko

  Why not run a diagnostic on something that needs to be right?

• 

  Tim Callan

  Agreed. But now what happens? So here's the thing. Linters are made by people. Linters have the scope that they have. Linters have the currency that they have. There is no linter I know of that covers everything.

• 

  Jason Soroko

  I was going to ask you that because I haven't been covering it as deeply as you have so therefore there are no linters that cover every diagnostic, every rule in EV set of rules, so the scope are always limited, and there is no definitive linter.

• 

  Tim Callan

  Now what we're seeing, because there are linters and they’re publicly available and they're good, we're seeing a set of CA behaviors where they basically only use the linter. They’ve got the linter. Throw it through the linter. If it comes out the other end, they say it's okay. I could give you a checklist of a number of things that need to happen or your day is going to be bad. Got to have your wallet. Got to have your shoes on. Got to make sure you're wearing pants. But your day could be bad even without those things, because something else could be bad about it. That list is not comprehensive. It's necessary, but it's not complete. The same thing happens with these linters. They're necessary, but they're not complete. Now we see this set of errors that have come up in the Bugzilla bloodbath - and these are some of the original errors - where, when the CA gets around to talking about the root cause, the root cause is well, the linter didn't catch it. I think this is problematic.

  I think it's problematic for a couple reasons. One of which is, the linters aren't going to catch everything. They're just not. If you start to act like you think that the linter is the only line of defense you need, then you are destined for trouble, because they can never be comprehensive. Another problem with it is that linters are open source projects made by volunteers, and sometimes those volunteers are working on their day jobs, and those volunteers might not put the new thing into the linter the same day that the new thing goes into effect. So there's a new requirement and now we need a things-a-bobem. Nobody gets around to adding things-a-bobems into the linter for two months because they have jobs. In the meantime, the linter didn’t tell me there was no things-a-bobem. So there is this tendency we're seeing for CAs to give up their own ability to understand and be responsible for what they need to do, because they feel like they can outsource it entirely to a linter, and that is proving not to work.

• 

  Jason Soroko

  So Tim, what that means is that until we have a definitive check, a definitive diagnostic, whether it's a linter concept or not, we're (a) this could be a shock to a lot of people, but misissuance could be, and probably should be considered inevitable. But also, let's think about alternatives. Like a very modern example of an alternative to this would be put the BR rules and the EV rules into an AI, train the AI on what all the rules are, and have the AI check the certificates. But what I would argue is this, a lot of AIs are non-deterministic.

• 

  Tim Callan

  I agree. I don't think this is the rule for an AI. Where AIs work well, is things that, for want of a better word, and this is me anthropomorphizing a little, things that demand judgment. This doesn't demand judgment. This demands absolute fidelity to codified rules. It’s kind of the opposite action.

• 

  Jason Soroko

  Exactly right. We need something that is truly a deterministic diagnostic against the entire scope of the rules.

• 

  Tim Callan

  Which the linters strive to be. Knowing that they can't be perfect, they try to be as good as they can, and they're definitely better than not having them. The people who are working on linters are doing really good things for the whole community. They're a net improvement. I think what I'm seeing is that there is this tendency, and it's possibly because the linters are so good for certain CAs to check their brains at the door now. And just blindly obey the linter and that's getting them in trouble.

• 

  Jason Soroko

  There you go. There is the problem right there is you’ve got something that works to a certain degree, but you cannot hang your hat on any linter right now to avoid misissuance.

• 

  Tim Callan

  So, if you're a CA, the linter didn't catch it, is not a root cause for misissuance. It's not the root cause. I mean, it's a thing that would have stopped it. But to blame the linter is not identifying the root cause, and CAs need to identify the root cause. That's point number one.

  Point number two is all CAs need to recognize that linters will inevitably, by their nature, remain incomplete, and there will always be a race to keep them current. So linters can have gaps, and linters can have time gaps, and both of those things are built into the system and the process. So you as a CA need to recognize that.

  Then number three, of course, is if you did have one of these problems, and your root cause problem was to blame the linter, then when we get to the what are you going to do about it, one of the things that also I can't stand is when the CAs don't say we'll make the linter better. Like these are open source projects. If you have a flaw because the linter didn't catch it, go put it in. Be part of the solution. Contribute to the community. So those are all things that I think we shouldn't really be tolerant of. If those are being used as excuses for a CA having a failure.

• 

  Jason Soroko

  There you go, Tim. Interesting thought. A lot of people don't think about linters as just part of a process, but I think what you're saying is it should be just part of a process, not the entire process.

• 

  Tim Callan

  Correct. Yes. That's exactly right. And we get people who are over reliant on it, who view it as being everything, the be all, end all. It's not the be all, end all. It is a safeguard that can frequently keep you from getting in trouble, but it is not a guarantee that you will not get in trouble.