Root Causes 205: Anatomy of an Encrypted Peer-to-Peer Mesh Network

Hosted by

Tim Callan

Chief Compliance Officer

Jason Soroko

Fellow

Original broadcast date

February 9, 2022

Secure online collaboration poses logistical and technical challenges under the best of circumstances. Now imagine you have no designated IT staff, no designated hardware, a small budget, and remote participants who are not deeply technical. In this episode Jason Soroko explains how he was able to quickly and easily create an encrypted communications mesh for use by him and his collaboration team.

Podcast Transcript

Lightly edited for flow and brevity.

Tim CallanSo, Jay, you were telling me a personal anecdote recently, and we thought it might be good to discuss it here. You had a content project you were working on in your own personal life, and you’re not an enterprise, you’re just you, but you had multiple individuals that all needed to be able to collaborate, and you needed to securely manage and transfer information across this group of people in their own locations with their own home systems. I think you had an interesting approach to that problem. Did I summarize that correctly?

Jason SorokoI think you did summarize it very well, Tim. So, let me describe the issue. It’s actually pretty basic, but it’s tougher to solve than most people think. So, basically, I wanted to run a media server, not something that a lot of people do. A lot of people will choose Linux to that, simply because there’s a lot of good media server software packages that are out there for it. As you said, I had some collaborators, and we wanted to be able to very quickly put stuff up to the cloud. Some of my collaborators were not 20-something year PKI veterans, and some of them just wanted to use their iPad because they were generating their content from there and they didn’t want to leave that. They didn’t want a Windows laptop. They didn’t want a Mac laptop. They just wanted to run their iPad. I had one user who wanted to run an Android phone and do something very similar. So, great, what are we going to do? I was administrating this whole thing from a Mac laptop. I also had a Linux VM running Fusion. So, think about it. This is a microcosm of hybrid/cloud for an enterprise because we wanted to be able to secure access a central location, securely access each other sometimes. Let me give you a real good example, a real-world example of what was going on here. I don’t know if you’ve ever used Apple’s iOS Operating System, Tim, but they’ve got a feature called AirDrop. And whenever my collaborators and I were in a room together and I had my trusty MacBook and some of them had the one that didn’t have Android wasn’t in the room, but the folks who did have iPhones and iPads, we were all able to AirDrop files to each other. Just so darn easy. The problem is though you got to be within Bluetooth distance for that. And so, anytime we weren’t in Bluetooth distance, well, we would use a multitude of other technologies, then I had to be the central gatekeeper of collecting files and moving it up to the whatever location I needed it to be. Well, I had a thought to myself. Wouldn’t it be nice if I could have AirDrop-type functionality for everyone? Anybody could drop files to any of the noted collaborators.

Tim CallanAnd presumably they could do it without being in the same physical location?

Jason Soroko100%. In other words, even if they were NAT’d behind some network, it didn’t matter. I want them to be able to collaborate, and basically, through an encrypted tunnel. So, this was the challenge in my head. I looked into it, and what did I come up with? I came up with, hey, you know what, there might be other vendors out there, there might be other technologies out there, open source or otherwise, but the one I came up with was called Tailscale. I’ve noticed a number of people talking about it because essentially, Tim, what you’re able to create is an encrypted mesh network between peers. So, think about the concept of VPN, where you use a client and you authenticate via a VPN into a network and your node is terminated at that point and then if you’re talking about a node-termination in an enterprise, well, you’re able to access that enterprise network just as if you were in the building. That’s typically the way VPN works. But what I’ve accomplished here with Tailscale is, I have the ability for people to be anywhere. This is not about VPN node-termination where I’m controlling a computer elsewhere. I’m just issuing very simple commands. Sometimes just, as I say, using like an AirDrop control, or I just want to do those of you who are Linux administrators, I just want it to do the equivalent of an SCP, a secure copy command, to another location, but I didn’t want people to have to learn SSH. I didn’t want anybody to have to learn anything other than using an incredibly simple client. So, as of right now, Tim, I keep using the term AirDrop, but it really is secure coping of a file to a cloud location which happens to be an Ubuntu Linux server, and I have people who are very, very unsophisticated computer users but very sophisticated contact creators who can do the same thing with a simple click of button from their iPads and Android devices. Hallelujah.

And this is peer-to-peer mesh network encryption underlying all of this. I hope that’s a good enough explanation for what the use case is and what I’ve accomplished with the software. I’d like to talk about why this is interesting on this podcast. So, what’s going on here? Yes, it’s crypto keys. This is what’s underlying all of this. Obviously, because we’re talking about an encrypted mesh, each of the encrypted tunnels between peer-to-peers are being created by basically a key pair that was generated at the point of provisioning the users. So, the user themselves possesses the private key on their device. All the public keys are managed in a central server, and my instance basically it’s Tailscale who actually hosts to those public keys for me.

Tim CallanSo a central server from the service. Not a central server for your instance?

Jason SorokoIf you think about this, you and I were going to have a podcast at some point in the future about using PGP keys to send each other encrypted messages.

Tim CallanWhen we get around to that one.

Jason SorokoAnd remember, one of the key challenges is how are the public keys managed? Because there is no infrastructure. Well, the way that it might be done is I have to advertise my PGP public key on my Twitter handle or maybe I put in on a website or maybe I’ll just bloody-well send it to you. And in this case, nobody has to do anything because the announcement through the Tailscale client is, hey, I want to be able to connect securely to my Linux server and copy a file up there. What the client does is say, I hear you. I know who you are. I know the destination you want to connect to. What I’m going to do is I’m basically going to retrieve the public keys from the central repository where they’re kept, and then I’m going to make all that authentication happen. And the way it’s happening, it is using WireGuard VPN, and as I say, the public keys of, hey, how do I connect over to that location? I know that computer exists, it’s part of my family of the list of public keys that Tailscale is hosting for me. I just want to connect to it, send the file, I don’t want to do anything technical. And Tailscale is doing all that work for me by hosting the public keys and using WireGuard as the underlying VPN. To my, it is just no-brainer, drop-dead simple. The most completely unbelievably, unsophisticated computer users possible use this seamlessly, and I have honest to goodness very secure mesh network going on that I don’t even have to think about. I’m kind of enjoying it right now.

Tim CallanThat’s great. Now that’s possible for them because it’s all inside of their world. It’s their walled garden. So, they can do that, they just have to think through the user experience and technology and make sure that everything’s connecting, correct? That’s a big advantage for them.

Jason SorokoCorrect. I can see how a big enterprise might say, hey, we need to be able to host those public keys ourselves. I’m not an expert in what Tailscale is going for big enterprises versus what they’re not.

All I know is that the free-tier option of what they’re doing what they call the friends and family tier, I’ve got essentially one administrator with a handful of different nodes that are connecting, and it’s working beautifully for me.

Tim CallanAnd if that’s what you're after, if it’s a small group SMB/friends and family kind of use case model that you're going for, I think it’s easy to make some of these assumptions. Like you said, look, we’re going to manage the keys. If you don’t like that, sorry, our service isn’t for you. And I think they will find that the overwhelming majority of the people they’re targeting will say, , I can live with that very easily.

Jason SorokoYou got it, Tim. So, this to me comes under the category of podcasts of hey, just for those of you who do this stuff and might have a need for it, it’s just a heads up. We did a podcast a while ago on I believe the folks at Trail of Bits created an Ansible script to be able to create VPNs at will on public clouds of your choice. And to me that was a really good exercise in, hey, check out how easily you can create a rock-solid VPN for yourself with a node termination point that you might trust because it’s your own public cloud instance as an alternative to purchasing services from a typical VPN vendor. That was a previous podcast we did that to me is along the same lines as this, and anybody who’s interested in this podcast, I fully invite you to even try that out still because learning how to do automated Ansible scripts, how to run them, how to set up accounts in public clouds – these are all skills that I think modern people in this, in our field should just have at your fingertips. And I think Tailscale is another one of these where even if your CSO says, well, we’re not using that because we don’t have complete control over the public keys, blah, blah, blah, or we don’t want to pay for that, we have some other service doing that for us – that’s all great, but for the average person, it’s killer. It’s just killer. And to me, it’s just like, if you want to get your hands dirty with this kind of stuff and see how the world is moving on, this is another one that I flag to anybody listening to this podcast and say, get your hands dirty. Try it out. It’s pretty darn cool.

Stay informed with expert insights

Subscribe to Root Causes for engaging discussions on PKI, digital security, and best practices for protecting your organization's critical assets. Don’t miss an episode!