Root Causes 473: Does Security Software Lack Creativity?

Hosted by

Tim Callan

Chief Compliance Officer

Jason Soroko

Fellow

Original broadcast date

March 28, 2025

Jason reports on a 2024 Black Hat keynote about how modern software development practices inhibit innovation and invention.

Podcast Transcript

Lightly edited for flow and brevity.

Tim CallanSo not too long ago, you and I had a conversation about a provocative and thought provoking talk that occurred at the 2024 Black Hat, and it was something that had stuck with you, and you had kept thinking about and you felt that you wanted to bring it to the audience. Coming out of that episode, that got us talking and I think there was another one at Black Hat, Jason, that you would say, I would say that's also a fair description. That you really thought touched you. We were going to talk about that today, too.

Jason SorokoExactly, Tim. Jason Healy had done a great talk, and it was about asking the question, are the defenders winning? Are the adversaries winning? I think that there was also a keynote that was done with Moxie Marlinspike. Obviously whenever he's speaking, I'm gonna listen, especially when it's a keynote. I think it got under reported, this keynote, because of the fact that so many software vendors, especially cybersecurity software vendors, are in attendance and he said was, the software development has kind of lost its magic, and how do we restore that? Part of how Moxie Marlinspeak was describing this loss of magic was really about we've become really, really good, especially with things like agile development framework.

We've gotten really, really, really good at adding layers of abstraction between developer teams and subject matter experts. There being this just a lot of abstraction for sometimes very good reasons. It allows developers to think and focus, but it also creates a lack of transparency about what's in the minds of developers, and when you're trying to tackle really tough problems, developing newer or innovative techniques becomes kind of difficult. I think the analogy, one of the analogies that was said very quickly was it’s almost like developers are trying to learn their craft in a library, without being in a classroom, without being in normal, other human settings where there's discussions about how to tackle a problem. It's almost like they're learning how to tackle the problem with just the books that are in front of them, and nothing else. They're off completely on their own and this ability to innovate and solve tough problems, and of course, obviously cyber security software has a lot of these problems to have to solve over the next 5 to 10 years. I just thought, geez, what a really good point. Agile has been amazing for productivity. I don't think it's been amazing for innovation.

Tim CallanSo, I want to examine this a little. I want to make sure I'm understanding this correctly. Is this because Agile, by its nature, is a small bites kind of process, and it interferes with the big ideas, or the big, I hate to say it, but the kind of the moon shots? Or is it more that putting so much structure and codification around the process distances that mind from the work? Or is it both?

Jason SorokoIf we go all the way back, Tim, to a book that I read a million years ago, and I bet you a lot of the audience has read it as well. Frederick Brooke’s book, The Mythical Man Month. This was the whole idea that you can't just shove a whole pile of developers, and develop software quicker. It just doesn't scale that way.

Tim CallanYou can’t turn the oven up to 500 and have a lasagna in an hour.

Jason SorokoThat is a very good analogy. The idea is, software development, really, over many years, started to learn, geez, if we isolate developer teams from one another, and we only have them communicate when there's, you know, hey, what's the definition of the API? As an example. If that's the only time that various teams talk to each other, and there's an expression between each other, and essentially they're working on black boxes that are only expressed from an API and nobody ever talks within these black box groups that kind of organization of software development teams, and it's gone by a lot of different names, and today we have Agile, but essentially what it comes down to is this real ability to focus on what needs to be built and then build it very efficiently within what typically would be called a sprint, and then you move on to the next problem and the next problem, and the next problem.

Tim CallanWell, that’s the small bites thing.

Jason SorokoThere's very little visibility from the outside. So that's the thing. Frederick Brooks' book, I don't think is as read anymore, because people have kind of realized that really, the way to solve the problem of software development is to break the problem down into small pieces and then isolate and get the job done and this very, very, very loose coupling between software development groups has yielded just amazing productivity. It has not yielded amazing innovation.

Tim CallanDoes Moxie offer a way forward? Is there a suggestion for how to be different and therefore be more innovative?

Jason SorokoI think the way that he was talking about it was to basically say, hey, the subject matter experts in cybersecurity - and he's not talking about software development in general - he's talking about, very specifically about cybersecurity software. In that case, he's saying, hey, cybersecurity professionals and administrators of cybersecurity within organizations have a really, really deep understanding of what the products need to be and the protocols that are needed and those people are not talking to the developers. My gut feel tells me that that's probably the heart of the problem and really the way to solve the problem, or at least begin to solve the problem, is perhaps to start a non-rigorous communication process between people who are not the developers, but who are really deep SMEs on topics in cybersecurity, and have them talk more in a normal, natural human setting with developers, so that developers can start to take the ideas that they're synthesizing along with the SMEs, and then turn it into more innovative products, Tim. I think that at the heart is how you start to solve the problem.

Tim CallanAnd presumably we need developers in the broad sense. We include product management teams; we include project management teams; we include architects; we include a broad set of people who are involved in the product creation process.

Jason SorokoAbsolutely product managers, when they're doing the job really well, are talking to customers. They know what's going on in the field. Then you start to get into the architects who are driving what is being built by the developers and onwards. Then by the time you get to the developers, those developers, have typically had very little communications out to everybody else. In good organizations, sometimes smaller organizations, you will have developers who are having good chats with product managers and SMEs. It does happen. I'm just saying it probably needs to happen more and the larger the organization gets, the more that this probably needs to happen.

Tim CallanI can see that.

Jason SorokoBut that's it, Tim. I think it was a very focused keynote. I thought, geez, I haven't heard anybody else talk like this. Here we are. Most companies are just completely headlong into Agile development, and nobody's thinking of the consequences.

Tim CallanWell, I mean, Agile development is basically, is certainly the norm and certainly it's done a lot of good. Like, let's be clear. It's done a lot, The massive waterfall releases that sit kind of in the back room for two years and never see the light of day, and wind up being a year late on delivery, and all of those things that happened in the old software world. Windows 95 very famously, was supposed to be done in 1993. So that sort of thing is much better now in an Agile world, but I think to have a conversation about, okay, but are there other consequences and negatives, and then what do we do about that is, it’s an interesting viewpoint.

Stay informed with expert insights

Subscribe to Root Causes for engaging discussions on PKI, digital security, and best practices for protecting your organization's critical assets. Don’t miss an episode!