Transcription performed by LeahTranscribes[FULL TRANSCRIPT]
JACK: [MUSIC] Okay, so this one time at Defcon – see, Defcon is in Las Vegas and Vegas never sleeps. Well, neither does Defcon. After the conference ends for the night, the place morphs into a night party, so after me and some friends spent the whole day at Defcon, we went and ate dinner, freshened up, and headed back to Defcon to check out the scene. We were told there was this rocking party in this one conference room so we all pop in and check it out. It was loud, like really loud. The room was actually quite small, about the size of a small classroom, and at one end of the room was a DJ spinning tunes. He looked bored as he was doing it. The room had bright red lights everywhere with intense blacklights shining in your face. I looked around; there were like me, my three friends, the DJ, and two other guys in this room.
The two other guys were bumping their heads to the DJ but their eyes looked like they were lost in deep thought. That was it. The place was dead. Pretty much as soon as I came into the room, I knew this; the music sucked, the lights were blinding. I wanted to leave right away. I scanned the room to look around. There’s an ice chest over there. Let’s go check it out; it’s empty. There’s a photo booth in the corner. No, no thank you. I told the boys let’s go, this sucks. We head for the door. I take one last look over my shoulder and I see four girls and two guys come out of the photo booth. This was a regular sized photo booth, way too small for six people to fit into it. The room was so disorienting but I didn’t put that together so we walked out and looked for another party. We ended up going down to the pool and hanging out there.
The next day my friend told me about this banging party at Defcon last night. I was like where was it? He’s like oh, it was in this one conference room. I’m like, I was in that exact conference room and that party was not banging. He’s like well, did you go through the photo booth? Yeah, the photo booth was the doorway into the actual party. They staged an entirely fake party just outside the real party to fool me and I was properly fooled. What a smokescreen. Why didn’t I register that six people coming out of a photo booth was weird? I don’t know but I feel like this story kind of sums up what Defcon is like. There’s crazy stuff happening all over, right in front of your nose. You kind of need the right set of eyes to see exactly what’s happening or you’ll miss it.
JACK (INTRO): [INTRO MUSIC] These are true stories from the dark side of the internet. I’m Jack Rhysider. This is Darknet Diaries. [INTRO MUSIC ENDS]
JACK: It’s summertime. You know what that means? Summer hacker camp. [MUSIC] There’s so many stories that come out of summer hacker camp and I want to talk about a few. What’s summer hacker camp, you say? This is what we call the week around Defcon in Las Vegas. See, Defcon is the largest hacking conference in the world but there are like, four or five or six or twelve security conferences all happening at the same time that week. It’s just crazy. First, the week starts out with Black Hat. Black Hat is a security conference but it’s more geared towards professionals. You’ll see people here wearing the typical business-casual attire. There are a lot of vendors all over the place, trying to sell you solutions on keeping your network more secure.
Of course, there’s talks and workshops at Black Hat, too. But Black Hat is super expensive so another conference started up at the same time that Black Hat is going on but it’s more community-ran. It’s called B-Sides. While Black Hat is happening, B-Sides, the security conference, is happening just a few blocks away and it’s completely free. It’s a great place to meet people and socialize with other security-minded folks. Here, the dress is more casual; cargo pants, t-shirts, that’s more common. At B-Sides you’ll see a lot of amazing talks too from fantastic security professionals. A lot of these talks are rejected from Black Hat so some are really great, and only a handful of vendors are here so you’re not overwhelmed with [00:05:00] people selling you stuff. I should also mention that there are security B-Sides conferences all over the world and they’re all community-ran so you might want to check to see if there’s a B-Sides in your town and go to that because it’s great. Now, as the weekend comes, so does Defcon.
Defcon is Friday, Saturday, Sunday. Defcon at its core is a hacker conference. The people you see here are sometimes wearing mohawks. They often dress in all black and have a bunch of electronics dangling out of their backpacks. The crowd is younger compared to Black Hat, too. I’ve ran into many high school kids at Defcon but I’ve never seen a high schooler at Black Hat. Defcon has talks, a lot of talks. There are tracks all over the place on so many subjects. There are speakers there who will show you how they’ve hacked into so many things. But Defcon is also big on being hands-on. There are a ton of villages with all kinds of things to try hacking into. There’s a car-hacking village, a picklock village, a voting machine village, a bio-hacking village, a social engineering village, and so many more. In each of these places you get to learn hands-on how to hack stuff. It’s a fantastic way to learn and you can spend your whole time at Defcon never going to see a single talk because there’s so much to do.
Defcon has vendors but these vendors are different. They aren’t selling you solutions to keep your network secure. They’re selling you hacks and exploits to break into networks; things like antennas, lockpicks, electronics, rubber duckies, key loggers, Pwn Plugs, and so much more. It’s so much fun to wander the vendor hall and see all the latest tech that you can just pick up for a few bucks and start practicing hacking in just a few minutes. Oh, and while at Defcon, there are many other conferences happening within and around Defcon. It’s weird but there’s Queercon, Hushcon, Vetcon, Gothcon, and even Deafcon, as in D-E-A-F for those hard-of-hearing. There’s Roots which is a conference just for kids. There’s also Diana Initiative which is a conference that focuses on women in security careers. Yeah, there’s a dozen cons going on all over town. The week of August 5th in Las Vegas, Nevada is the place to be for security professionals around the world.
I could go on and on about all there is to do at Defcon but what I want to talk about for the rest of this episode are the contests. There are so many contests at Defcon too, and here is where I learned the most. I love joining contests with one goal; the goal is to not get last place. If I can beat anyone else, I feel like it’s a victory for me. But let me tell you, it’s not so easy to do that. There are contests on cracking passwords like who can crack the most amount of passwords in a weekend. There’s writing contests, and beard contests, and scavenger hunts, and a bunch of trivia contests, and so many more. But one year there was a contest I just couldn’t ignore. The thing is, I didn’t even know it was a contest. Here’s what happened.
[MUSIC] This was Defcon 19. The year was 2011. Upon registering into Defcon, you’re given a little badge. Now this badge serves one purpose; it’s your pass into Defcon. Without it, security will stop you and throw you out. But being a hacker con, a paper badge is kind of easy to counterfeit, right? The organizers started making electronic badges, ones that had little blinking lights at first, and then LCD screens. Then one year a badge had a microphone built into it, and eventually these badges became pretty elaborate little electronic devices. People loved it but it was kind of a pain to design a new cool electronic gizmo every year so the organizers decided to do an electronic badge one year, and a non-electronic badge the next. At Defcon 19, the badges that were given were simply a solid metal, non-electronic. Some say it was even made out of titanium. I paid for my ticket into Defcon and was given one of these metal badges.
It was a dark grey, metallic-looking thing. It was round and had an eye of Horus cut out in the middle of it and it simply said H3 on it. No mention of Defcon on the badge itself which was kind of weird. The H stood for human which is what the standard admission is to Defcon. Some said V and that’s for vendors, and some said S; that’s for speakers. Some said G for goon. Those are the security guards. Yeah, even the security guards have badges. But connected to the badge was a lanyard, and on the lanyard, it said Defcon 19. That’s the only English it had. But what it also had was a lot of strings of ones and zeroes. These strings were thirteen characters long and there were fifteen strings of these. These ones and zeroes weren’t printed here by accident. I knew this was some kind of puzzle so I started poking at it. Nothing in my mind is thirteen characters long, though. It’s not IPv4 or IPv6, not ASCII, not hex.
Hm. When you register at Defcon, you’re given a schedule, too, a little black book. I was looking at the book and on Page 4, something stood out. It said HACKUPONXYLEM. For some reason, the way it looked, it had similar symbols as the lanyard so I copied all the ones and zeroes off the lanyard and put them in a row. I tried to put the clues together somehow. Strangely enough, HACKUPONXYLEM also had thirteen characters in it. By arranging it all in the right way, HACKUPONXYLEM became the key to unlock what the lanyard was saying. By doing this I discovered the hidden message was LAUNCHKEYNOPMYX. Now I have a launch key, but what’s this for? [00:10:00] There was a strange URL in the book with X’s in it. I typed the URL with NOPMYX in where the X’s are and boom, it gave me a secret web page. The secret website said something like ‘You have discovered us. We are the Brotherhood of Horus. We have accepted your launch code and the sleeper agents are now active.’
It went on to say that there are sleeper agents at Defcon that are infiltrating Project Xylem and that I must find them and expose them. The website went on to show me ten pictures of these agents, and each picture looked like a spy took them. This was getting serious now. There was a note saying that I was now part of the Brotherhood of Horus. I think I just got recruited to help out. I think this Brotherhood of Horus group was trying to send a message out to get someone to help but didn’t want it to be too obvious or the sleeper agents would know. It was on. I was ready for this. Forget about the talks I wanted to go to. I wanted to play this game. Time for the next clue. The website told me I had to get an Ace of spades and hand it to one of the sleeper agents but I have to write the password on the card. When I give it to the agent, they will look at the Ace of spades. If it’s the right password, they’ll then give me the inside information I need. I was told to do this as discrete as possible or else the agents will not do it.
Okay, this is getting good but I need a password. What’s the password? NOPMYX? No, that’s just the launch code. I don’t want to blow this and try NOPMYX and it not work. I think I better look for another password. The bottom of the pages in Defcon had a little puzzle. It took me a long time but I solved what it said and it said ‘Find code word ghost.’ I looked all over the conference for a ghost. I didn’t see one but there were huge pieces of artwork stuck to the floor of the conference, giant circles with the words Defcon on them, but with lots of strange symbols, too. One had Japanese writing on it so I stood there and profiled people, looking for anyone who I might know they speak Japanese. I asked people and eventually found someone who could read Japanese. He told me the Japanese symbol on the floor said ‘ghost.’ Hah, I found the code word ghost. This led me to a logic puzzle which I had to solve to find another clue, but I still didn’t have the password.
At this point, running all over the conference, looking for clues and standing on top of them for thirty minutes at a time, writing things down and asking people for help, I started finding other people who were solving the same puzzle as me. [MUSIC] We started trading information. I told them how I solved one thing and then they would tell me what that password was. The password was ‘little sister.’ Excellent. Me and a friend found an Ace of spades and we wrote ‘little sister’ on it and started looking for sleeper agents. But this was hard. I was looking for one of ten people in a crowd of ten thousand people. All I had was their picture, too, not like I can ask for names or anything. I stood in the hallway staring at every person walking by, trying to recognize if any of them matched the faces in the photos. Nothing. Nobody. People were a little weirded out by me too, staring at everyone. Then I met another team solving the puzzle and they told me they just saw one of the agents in the vendor area.
Quickly, I ran down and spotted them. He had a Z on his badge which was really strange. At this point I realized I’m playing an ARG, an Alternate Reality Game, a game that combines the real world with fantasy and I was having a blast. I very casually walked up to him, handed him the card. I said nothing. He looked at the card, looked around for a moment, and told me the code. ‘Candy.’ Candy? That’s it? I typed it into the website, that mysterious URL we saw earlier, and ‘candy’ worked. I was supposed to send a message to something, but the something was garbled. I had the message but I didn’t know where to send it. I had to solve this little puzzle to figure out what it was, and it turned out to be an e-mail address. I sent the message to the e-mail address and somebody responded on the e-mail which gave me the next clue. That clue made me believe that I needed to gather all the different badges at the conference, stack them up on top of each other, and then that will give me the key to unlock the code.
[MUSIC] Me and a few friends started going to every person at the conference, looking at their badge to see if it was one I hadn’t seen before, or a new one, or a different one. I wanted to see the vendor one, the human one, the contestant ones, the black badges, the goon badges, and I took a photo of every one and I traced it on a piece of paper. I documented it as best I could, every single badge. This took hours and hours and hours. Finally, I felt like I got them all traced on a piece of paper and when I did that, I noticed there were certain notches in some positions. I’m starting to think these notches mean something. The notches are the key or the code or something but this is madness; I mean, what kind of key is a bunch of notches in thirty different badges? How do you use that to decipher a string of numbers? I couldn’t do it, my friends couldn’t do it, and at this point it’s Sunday now. The conference is almost over.
I gave up. I wouldn’t be able to solve this. I’ve got to go home soon. I start asking around. None of the teams have actually solved it. We were all stuck on that same exact last step. We grabbed a corner [00:15:00] of a conference room at Defcon and all the teams came together to try to figure it out. We brought a big screen TV in and started putting all the clues on it and discussing all the possibilities. We went over everything we tried with each other, and this gave us new ideas to try but those ideas weren’t working, either. The conference was pretty much done now; there were no more talks happening. The place was starting to clear out. It was closing. We were at the end of our time but we needed to solve this so we kept at it, plugging away at this last puzzle. At this point there were about twenty of us from eight different teams all in the same room, sweating over this puzzle. At some point one guy squeaked.
We all looked and he was furiously writing something out on a piece of paper and he said hang on, hang on, this might be it. He wrote out a string and just like that, the puzzle was solved. We quickly e-mailed the clue in and got a response. Infiltration successful. Congratulations on completing the badge puzzle. Yes! We all roared with excitement and there were high-fives. We solved it. The puzzle was created by a guy named Lost and he said he’ll come to us and give us the reward. We told him we’re in this conference room and he shows up and gives us the prize, a black badge. It was also made of titanium and looked like The Punisher skull. Holy cow, the black badge is the most coveted prize at Defcon. Only a few contests have the black badge as a reward. It’s like a gold medal for hackers. It actually has real value; your black badge gets you free entry to Defcon for life. Lost handed it to us. We were all smiling and loving this moment, but then he said ‘but we can only have one.’
We were like, but there’s twenty of us. How are we supposed to split one amongst us all? He’s like well, you’ll have to figure that out. Sorry, guys. We were all pretty mad at this because this is not a puzzle that one person can solve, but only one person gets a prize? It’s just not fair. But that was that, one badge for our group of twenty people. Well, we agreed which team should get the badge based on their performance and they’d figure out something special to do with it. We all exchanged e-mail addresses, went home. The next year, lo and behold, that winning team came through. [MUSIC] They spent the year making replicas of the black skull badge. It looked the same in every way except it was about three-quarter size and they printed on the top of it, Brotherhood of Horus, which is what we called ourselves during this challenge. This was a cool little trophy to keep and I still have it right here in front of me, on my desk, and I look at it all the time.
You tell me, did I win the black badge? I don’t think so. I don’t get free passes to Defcon but did I help one-twentieth of the way to get the black badge for the team? Hell yes, I did. I spent my entire Defcon weekend on that one puzzle and I don’t remember a single other thing from Defcon 19. But that’s what it takes to win these contests. If you’re going to compete to win in a contest, it’ll be one of the hardest, craziest weekends you’ll ever experience. Today the badges at Defcon have gone nutso. Because people love those electronic badges so much, people just started making their own badges. Some designate what hacker group you’re with and some designate what city you’re from. Others show what skills you have.
Most people wear them because they’re just fun; blinking lights, little video games, swappable parts, Wi-Fi strength meters. These things add up quick. Many people will just wear ten or more badges around their neck at Defcon, kind of like collecting them. This is what’s called ‘badge life.’ That’s the story about how I kinda, sorta, almost won a black badge. But after the break we’ll talk to a guy who won four of them. Carnegie Mellon is a university in Pittsburg, Pennsylvania and one of the schools they have there is the School of Computer Science. Here is where they teach IT.
TYLER: Basically, there was a computer research group at Carnegie Mellon and it was David Brumley’s research group.
JACK: David Brumley is a professor at Carnegie Mellon who teaches courses on computer security and has a research group there who does analysis on security threats.
TYLER: There are a bunch of people there doing all sorts of interesting computer security research. One of the researchers was like you know what’d be fun? Is if I was playing in CTF or something. He’s like oh, you know, maybe I’ll look.
JACK: He wanted to look around for students to join another CTF team. CTF stands [00:20:00] for Capture the Flag. It’s a hacker competition. Basically, whoever is running the competition hides a flag somewhere in a computer and you have to find it. It’s not usually an actual flag; it’s like a secret word or something and it’s just to prove that if you know it, then you’ll get the points. But here’s the thing; in a hacker CTF, they often tell you exactly where the flag is on the computer but you just don’t have permission to see it. You need to hack into the machine somehow to see it. This is great fun because it teaches you how to hack with real hands-on experience. One student really wanted to play these and asked his professor, David Brumley. David looked around for some online CTF teams for this student to join but came up with a different idea.
TYLER: He was like well, actually, we’ve got all these security researchers here. Maybe we should just form a team ourselves and see how that goes. Maybe that’d be more fun.
JACK: The CTF team at Carnegie Mellon was formed. The first big CTF that was coming up they all wanted to compete in was called Seesaw. This is sort of an entry-level CTF and it only allows students who are undergraduates to compete. It was at NYU just a few states over but the team needed more people to help and compete.
TYLER: They were like oh, we should find a couple other people that we know who are in security. Then myself and one other person joined up.
JACK: Oh, I should mention this is Tyler we’re talking to and in 2009 he was an undergrad student at Carnegie Mellon. He just joined this CTF group.
TYLER: At that time, the captain was a guy called Brian Pak and then there were a few other, just a handful of other people so myself and then two other undergrads, and then a few grad students who were all interested in computer security stuff.
JACK: This team wasn’t even studying computer security at this university.
TYLER: I think we’re all studying computer science in general but happened to be interested in security things so that’s how it started, yeah.
JACK: They started studying for their first CTF, Seesaw. When they went to register to compete, they had to come up with a name for themselves so they named it PPP which stands for…
TYLER: Plaid Parliament of Pwning. The school color for Carnegie Mellon is plaid which, you know, one could argue isn’t a real color. Instead of doing PPP it used to be PPoP, Plaid Parliament of Pwning, and we eventually shortened that.
JACK: The Seesaw competition, it was a ways away so in the meantime they began practicing, well, hacking.
TYLER: I remember one of my friends, Andrew, who is one of the first people on the team, he’d basically go and we’d be walking around campus or something similar and he’d just list off some assembly to me and be like, ‘What does that do?’ I’d have to sit there and think or he’d ask me, ‘What’s a function prolog look like in an X86 assembly?’ Or just kind of weird things like that.
JACK: [MUSIC] For the team to practice, they would sit down and solve some Jeopardy-style CTFs. This is where you’re given a challenge to solve and you solve it; maybe something like read the contents of this file that you don’t have permission to read or find the hidden message in this file, or decrypt these files that you don’t have the key to, or find a way into this web server. The team would try these things and learn all about hacking, and they got better as they went. Their confidence was building. They were getting better and better. Then they headed out to Seesaw to compete in New York for this CTF. There were a bunch of other teams competing but there were only three questions. You had eight hours to complete it and there was no scoreboard so it was hard to know if anyone was doing right or wrong the whole time. They did their best, submitted some answers. Time was up.
TYLER: I remember sitting there at the award ceremony and you know how they always go third, second, first? We were like okay, I hope we did well, I hope we did well. They announced third and we were like ah, shoot. Okay, well we didn’t get third. Then they announced second and we were like oh, that’s the team I would have expected to win. That’s weird. Then they announced first and it was our team and we were like oh, man. We were so excited ‘cause we were very much not expecting that.
JACK: Whoa, nice, their first big win. They actually won money for this competition.
TYLER: Something really small, I don’t remember. It was probably like $500 and a plaque. It was a pretty short-lived celebration ‘cause we had to go back to classes after the weekend or whatever, so we didn’t get to spend much time doing anything fun but we were all super excited because it was a success for us, which we weren’t even expecting.
JACK: But this win wasn’t small. It proved that this small team they formed had big potential so they immediately started looking for more CTFs to join and play. Winning is addictive and they were going to as many of these hacker competitions as they could.
TYLER: One of the things that was kind of funny for our team that we always joked about is we played in all these really obscure Korean CTFs. I guess South Korea has always had more of this kind of CTF scene, or at least in the early days they did. Brian Pak, [00:25:00] who is the founder of PPP, speaks Korean so he’d find us these weird CTFs and we’d start playing them and we’d get all these weird things. Throughout the year we were playing all these weird obscure CTFs where we’d have to have him translate stuff ‘cause we don’t understand what’s going on. We started doing pretty well at most of those during the year. For some of them we were getting at least Top 3 or so. There were a few other competitions; there was ICTF which is run by the University of California Santa Barbara which is a super popular one for universities. We played in that and did – I think we did okay our first year.
JACK: This became their obsession for everyone on the team. CTF, CTF, CTF all the time, everywhere. They were qualifying through online challenges and then when they’d get accepted, they’d fly to that place to compete. New York, California. Many of these CTFs would pay for their flight, and room, and entry if you qualified. The team wasn’t winning all that much money but they were still doing really well and they were starting to get to know some of the other teams they were competing against, too. PPP was doing so many more CTFs than anyone else.
TYLER: If you looked around at the time, most of the other teams would play in maybe three competitions a year whereas PPP was playing in twenty or thirty competitions a year. It was mostly just a trial by fire where we were just jumping in and doing all the CTF problems that we could, and it turns out that that’s a really good way to get very good at CTFs.
JACK: This student-ran team at Carnegie Mellon was picking up some new students, too. The only requirement to be in PPP was just to be a student and be interested in security. People were fascinated with what this team was doing and they wanted to learn hacking too, so they’d come by to a practice session and join up. The team started growing a little bit, and because the sheer number of competitions they competed in, they learned a lot of tricks on hacking and really refined their skills. They were writing their own exploits, learning classic cryptography, and solving cyphers, and learning how to reverse-engineer software like pros. Let’s talk about reverse-engineering a little bit. This is an extremely important skill for these hackers.
See, what a typical penetration tester does is scan a computer for known vulnerabilities. Then when they find that vulnerability, they exploit it using a tool that someone else already made. But see, here’s where these CTFs are different than penetration tests; some of these advanced CTFs, you’re told to exploit some software to get a flag, but the thing is, that software was just created specifically for this challenge, meaning it was just created last week and there’s no known vulnerability for it. Your scanners won’t work here. Your off-the-shelf hacking tools don’t work here and you have to somehow find vulnerabilities yourself. That’s where reverse-engineering comes in. Since you can’t look at the source code to see what the software is doing, your only option is to look at the machine code in assembly language.
This is very low-level commands, like you’re almost looking at the ones and zeroes going across the wire. Not quite, but almost, right. You’re looking at where the data is stored and moved and how it’s changed in the memory, then piecing all this together to get an idea of what the program does. This is what’s called reverse-engineering and you use a disassembler like IDA Pro to do that with, completely taking the software apart, looking inside of it, and looking for its flaws. The team has to do that on some of these challenges. It’s crazy hard, super technical, and intense to try to get this done within the time allotted. But they kept at it, getting better at it and better at it, doing CTF after CTF for a whole year, doing as many challenges that they could. One of the big competitions they wanted to compete in was called Codegate which is a really big competition in Korea.
TYLER: We managed to qualify for the competition and managed to convince a bunch of our teachers to give us some time off to travel to Korea to play in a hacking competition. We went there and you know, all the teams there were the big names in CTF at the time. But especially as university students who were going there, these were more like people who were in industry or professionals, or things like that.
JACK: They get all set up and begin the competition.
TYLER: I know we were doing pretty well through most of it and actually at some parts of the game, we ended up getting – we were in first for a little bit but then in the last thirty minutes of the twenty-four hour competition, the Swedish team managed to solve something else and they got up to first place. The end of the competition, it was the Swedish team, and then us, and then a Spanish team. But again, for us, the fact that we even got Top 3 was mind-blowing and shocking. It was super exciting. Also, in contrast to Seesaw where it’s a bunch of American universities which [00:30:00] is good, it’s tough competition, but this felt like the real deal. Then in the end, I don’t remember exactly, but I think the prize was something like $5,000 or $10,000 in addition to having all our flights and hotels paid for to South Korea, which is a lot. Kinda sexier than going to New York from Pittsburg.
JACK: Now things are heating up. For the members of PPP to travel to a prestigious hacker conference, see a lot of the other top CTF teams competing there too, and to get second place among them? Whoa. That means his team really does have a lot of potential and they’re just getting started. Soon as they got back home, they immediately started looking for more CTFs to do.
TYLER: [MUSIC] We basically, almost every weekend, we’d hole up in some building on the corner of campus and we’d work on these competitions for twenty-four, forty-eight hours straight. Then we’d go back to class on Monday but again, if you think about it, running forty hours a week for a year, that’s like, two thousand hours. Spending two thousand hours or something a year on this after a few years, you start to accrue skills pretty quickly.
JACK: That brings us to Defcon. Remember Defcon, right? The largest hacker conference in the world with tons of competitions all over the place? Well, the most prestigious competition at Defcon is the CTF. It’s the main event. The Defcon CTF is like the world series of CTFs. It’s the most challenging, most competitive, and it earns you the most bragging rights of any other CTF. The team at PPP decided to give it a try. Now, months before Defcon is a qualification for CTF. They only accept a certain amount of teams. This is played online from anywhere in the world. You have a limited amount of time to solve the problems and hack as much stuff as you can. PPP gave it a shot to qualify.
TYLER: I think it was a seventy-two-hour competition. We played it and we did, I mean, I’d say we did pretty respectable, but we ended up in something like 11th or 12th place which was kind of shocking to us ‘cause up to that point we had been doing Top 5 or something for most of the competitions we played. But one of the things we didn’t realize is there was this second group of people who play CTFs but only play in Defcon and a couple other competitions. A lot of the people who were in industry didn’t bother playing these smaller contests. They’d only play Defcon. I wouldn’t say we got our butts kicked, but we didn’t do very well.
JACK: Whoa. See what I mean? Even though PPP was hot stuff, winning competitions all over the world, they didn’t even qualify for the Defcon CTF. Defcon only accepted the Top 10 teams that year and they didn’t make it. The teams here are just that high caliber, best in the world at hacking. PPP had to go back to practicing. [MUSIC] Around this time the school year started back up at Carnegie Mellon which brought some new students interested in hacking to help. With the summer being over, the PPP team was excited to get back into CTFs.
TYLER: Everyone’s always excited after summer. They’re like this year I’m gonna play CTFs even harder than I did last year, things like that.
JACK: They decided to hit up all the same competitions they did the previous year. They went back to Seesaw.
TYLER: We ended up getting first place for the second time in a row.
JACK: They did a bunch of smaller online CTFs but then Codegate came up again and this is the biggest CTF in Korea. Remember last year they got beat out by that Swedish team? This year they qualified for it and flew to Korea again to compete.
TYLER: We managed to get first place which was I think, $20,000 prize money. We were quite ecstatic with that. Kind of early on, our team has had a lot of Korean influence. We played on the early Korean CTFs, a lot of the grad students we had in our early team were Korean, so we always had traditions about going out for Korean food and stuff to celebrate things. We went out, grabbed a whole bunch of food. All the other teams, after the conference is done and they break everything down, they throw a huge party because the conference is a little bit smaller. All of the CTF teams that played in the conference go out to a bar and drink and talk about the competition. Everyone’s still kind of dreary from not sleeping but excited from the award ceremony and everything that just happened. It was just really cool to do that and just a lot of fun to hang out and drink and get to know all the other teams, and eat delicious food and things like that.
JACK: This was the biggest victory yet. Codegate is a very competitive competition and they walked home with the [00:35:00] grand prize of $20,000 which by the way, they saved all this money to use to travel to more CTF competitions. PPP was definitely making a name for itself in the hacker competitions but they still wanted a shot at competing at Defcon. Winning that would be a dream come true. They kept practicing, CTF after CTF, doing as many competitions as they could.
TYLER: [MUSIC] At this point it was like, maybe not a CTF every single weekend, but it was getting close to it. We had done so many CTFs that when the last time that you’ve played in a competition is like, two weeks ago, or the week before, you’re not very rusty. You’re like, I’m in pretty good shape to play this.
JACK: [MUSIC] Many of these CTF competitions lasted a full twenty-four hours or forty-eight hours, or even seventy-two hours. The team had to learn how to manage their time effectively to perform best during this time. For instance, they’d have their food all picked out and ready to go just so they can grab it and keep going. They had to figure out other interesting logistics to keep them going the whole time. Defcon qualifications came around again and PPP gave it a shot. This time they did much better and did qualify. Yes! Now they’re on their way to the most prestigious hacking conference in the world.
TYLER: We were finally ready to actually make it to Defcon.
JACK: Unlike the Seesaw and Codegate hacker challenges which paid for the PPP’s flights and hotels, Defcon didn’t pay for anything. But it’s okay because they had a decent amount saved from all the winnings of their other competitions. But another thing that’s different about the Defcon CTF is that this one isn’t Jeopardy-based where you have to find the clues. This style is called Attack/Defend which was not something PPP had much experience at.
TYLER: Defcon is completely different. It’s pure attack/defense; there’s none of this Jeopardy sit-and-relax and think about a problem for a long time. Everything is kind of hectic and everything’s on fire. You’re in Vegas which honestly kind of sucks, so it’s like everything’s different from what we had been used to with like, you go into a room and there’s quiet music in the background and you sit and stare at your computer screen, thinking for a long time. Defcon’s very much you go in, everything’s loud and there’s bright lights, and you have to work as fast as you can before someone breaks into your server and starts breaking stuff.
JACK: Yeah, in the Attack/Defense style of CTFs, other teams are trying to hack into your computers and you have to block them from getting in. At the same time, you have to attack their servers to see if you can exploit them.
TYLER: The general setup at Defcon, or most Attack Defense CTFs, is all the teams are running a set of network services, up to ten or something.
JACK: These network services might be a webpage or an FTP server or an e-mail server.
TYLER: Each of these network services, any team that’s playing in the competition can talk to them. You have to find the hole by analyzing the code that was given for your service. You look at your own thing and you need to find all the security holes in it, or as many as you can, and use those to start exploiting other teams. When you start exploiting other teams, you can inject backdoors or do whatever clever tricks you can to make yourself stay inside of their system no matter what they do. As you’re doing this, you have to pull out some data from their system in the form of a flag which is basically just a single file that will stay on disk and then rotate every five minutes or so. Every five minutes you want to prove that you have access to the system by continuously stealing the contents of that file. Then simultaneously you need to defend your own network either by patching your services, by analyzing network traffic, anything like that so that you can prevent other teams from using the same attacks that you’re trying to develop against you, or any other attacks that they find.
JACK: This puts a whole new twist on the CTF game style. Now you have to strategically think which teams to attack, when to attack them, how to attack them. It’s probably better to attack yourself first, learn how you’re vulnerable, and then use that vulnerability against another team, and at the same time try to figure out how to defend yourself from that vulnerability. If you do it this way, you’re very quick, in and out of the network before they even know it. But here’s the other aspect you have to consider; any team that is attacking you, you can sniff their incoming packets and try to see how they’re attacking you. From here you can sometimes steal their vulnerabilities because they just showed you their hand.
TYLER: We were playing and during Defcon, because it’s Attack/Defense, you have to look at network traffic of things that other teams are sending over the network to your machine. One of the things we saw was some clear version of a backdoor. Sometimes after a team exploits [00:40:00] a challenge, they’ll put in, in addition to their exploit which gets them a flag, they’ll put in something that will persist after their exploit terminates, and keeps sending the flag back. We saw something and it was installing a crontab entry.
JACK: A crontab entry is a command that’s just set to run at a certain interval. Maybe every five minutes it checks to see what’s in a file and then sends the contents of that file to that team.
TYLER: We were like oh, this looks interesting. We went to our machine and we did crontab-L or whatever to list our crontab. It was like okay, there’s nothing here. They didn’t pwn us with this so we’re fine. [MUSIC] But as we kept going throughout the day, we realized that we were definitely getting exploited on that service and we had no idea how anyone was doing it. We kept looking and looking. We couldn’t figure it out. Eventually what happened was, it turns out they did add a crontab entry but after they put in the malicious code for the backdoor, they put in a raw carriage return and then they put in No Crontab Entries Found, or something. If you cat the file, it’ll read out the – the exploit will get displayed but then the carriage return will bring the whole thing back to the beginning of the line, and then over the exploit it’ll print No Crontab Entries Found. If you just cat the file you don’t see anything, but if you cat the file and pipe it to a hex dump, then you’ll see there’s a whole bunch of other hidden stuff inside of there.
JACK: You see how crazy this is getting? You’re in a room with some of the best hackers in the world attacking your systems like crazy, and they’re doing everything they can to hide the fact that they’re hacking into your box. There’s a feeling you get when you find out hackers are in your computer. It’s crazy stressful and intense. The blood rushes from your face when you find out someone else is in your computer, even if it is just a competition. Tyler was kind of upset that this team was sneaking backdoors into their server, so he wanted to do some sort of payback. They watched the network traffic for that team and saw that whenever they would grab a flag, they had a server open and ready for listening for incoming flags. Tyler had a plan.
TYLER: We started sending zip files, so basically compress a gigabyte of null bytes, which will compress down to like, a few kilobytes of compressed data because it compresses very well. Then we’d send that to their server that was listening for flags. Then on their end, they’re gonna decompress this gigabytes and gigabytes of stuff and try to submit it as a flag. It actually started bringing down their internal infrastructure for getting these flags and sending them off to the server. Not quite perfect payback but it was still pretty funny.
JACK: Now there’s some sabotage going on. I love it. The other team thought they had captured a flag and spent a bunch of time trying to unzip this file but it was just a large junk file that Tyler sent them. It just wasted their time and ended up bogging down their systems. Brilliant. This gives you a little idea of what’s going on in the Defcon CTF.
TYLER: Yeah, it’s like every second there’s something new going on where someone’s like wait, which version of the binary did we do? Did we patch this one? What is this network traffic? This looks like an exploit. The whole competition goes like this. It goes on as a three-day competition. By the end of it we’re exhausted and the score board’s open the whole time so we can see that. I don’t know what we got our first year, like seventh place or something. Kind of no hope pretty early on. It was pretty clear that we were getting screwed in that event.
JACK: Bummer. Seventh place? That means nothing. There are only prizes for first place. How many more CTFs and practicing does this team need to do to win this thing? But I guess they’re just college kids after all and still have a lot to learn. Back home to Carnegie Mellon they went. [MUSIC] A new school season started up which means more people joining PPP, and again they make their rounds to all the CTFs for the year. They go compete in all the ones they can, pretty much every weekend again, dedicating another year to CTFs.
But this time they focus on things that will help them prepare for Defcon. The year goes by, and the Defcon qualification comes up again. PPP tries and qualifies. They fly out to Vegas again to compete, but they didn’t do so great. They got something like fifth place that year. So, they have to wait another year. Back to doing another twenty CTFs in the year, back to Korea to compete, back to California, back to New York, and then back to Pittsburg to practice. Then Defcon comes up again, PPP qualifies, and they head out to Vegas for the competition.
TYLER: This year some combination of being more relaxed about the competition or the organization running more smoothly, or whatever, we were doing super well. I think at the end of the first day we were already in first place. The end of the second day we were already in first place. We’re still in first place when we came to our lead. We’re like oh man, we’re finally [00:45:00] gonna win Defcon. This is great. We’ve been working on this for so long. We’re up at night, we’re like okay guys, we’ve just gotta keep doing what we’re doing. Don’t screw anything up. We totally got this in the bag.
The last day, we go in. We’re running through stuff, and just before the end of the competition, it was either an hour before or thirty minutes before or something, the team that was in second place manages to solve some weird challenge that we didn’t even look at because we only had eight people. They managed to solve that challenge and they shot up past us and they won the competition. We got second. Then we were talking to them afterwards and we learned that their – you know, they had a lot of good people on their team and everything but their team was actually a group of eighty people. Literally eight times more people than we had, and they beat us, but they only beat us barely.
JACK: There was no limit on the size of your team that year, but PPP had a taste of blood in their mouths. They were so close to winning. They knew if they practiced a little more and they come back again, they have a really good shot at winning this. Another year of hardcore practicing; more analyzing of binaries, more practicing of machine code, more learning cryptography, more reverse-engineering.
TYLER: One of the other people on our team, Ricky Zhou, he went to high school with George Hotz. They both went to high school in New Jersey together. They actually kind of knew each other. George ended up at Carnegie Mellon for a little while trying to study stuff so we were like – we quickly were like okay, you need to play CTFs with us. Trust me, you’ll love it. It’ll be lots of fun.
JACK: Whoa, George Hots? You remember this guy, Geohot? At seventeen years old, George unlocked his iPhone. When you buy an iPhone, it’s set to a specific carrier. Yeah, well George jailbroke it so he could use any carrier he wanted. You might be thinking big deal, I’ve jailbreaked my iPhone, too. Yeah, but George was the first person ever to do it, ever. Well, the first person to publically admit to doing it. That made huge news. Then, a few years later, George reverse-engineered the PlayStation 3 and was able to read and write memory within it. This was a monumental feat. Those things were locked down really tight. Again, this made news, so much news that Sony actually sued him for doing it which created a huge backlash against Sony. Now this famous hacker was there at Carnegie Mellon and the PPP really wanted him on the team. George joined.
TYLER: [MUSIC] He’s just a really fun and hilarious guy. As soon as he shows up to our team meetings, it’s really exciting ‘cause he totally goes all-in for the CTFs. Like most of the people that do well at CTFs, part of it is just being able to sit and concentrate on a really difficult problem and do that for extended periods of time. He’s just very good at doing that. We’d have some problems where – I think we had some problem that was some really hard crypto problem that I think during the competition, no team solved. This was just some random competition we were playing in the year. That was the problem that he was working on at the end of the event and he was like, you know what? Screw this. I’m gonna go back to my room, I’m gonna lock the door, and I’m gonna keep working on this problem until I solve it. That was kind of his attitude for a lot of these things.
JACK: Okay, so this was a great boost for PPP. Now, with a few new teammates, more practice under their belts, they headed back to Defcon for their fourth attempt at the competition. They have their food orders all on a spreadsheet, and two helpers just running around getting them the things they need so they can focus more on just hacking as much as possible. They made sure to get a hotel room at the conference so that they didn’t have to spend any time driving around, and they even got rooms as close together as possible.
TYLER: We tried to get a suite to have all of our teams so they can work in a single place instead of having to work across a few different hotel rooms or sitting on a bed in someone’s hotel room.
JACK: Tyler, now the captain of PPP, and the team is feeling better than ever to compete. What they also liked that year was the team size limit was set to eight people. They think this was to their advantage. The team is prepared to spend as many waking hours as possible throughout the entire Defcon weekend to attempt to win this contest. It takes a toll on their body each time they go through it.
TYLER: Most people know that if we’re going there, they’re prepared to lose a lot of sleep and drink a lot of caffeine and all that.
JACK: They begin the competition. They see a lot of the same teams and faces that they’ve known before; some Korean teams, some American. These are the top teams they were expecting to see, and at this point they’re starting to understand their attack style and defend style a little more. Tyler thinks some of the other teams might even be sleeping in shifts so there’s always a group hacking while another group is sleeping. You never know what kind of operating system the organizers will have you hacking on. It could be Windows, [00:50:00] it could be Linux, it could be Unix. But when the contest started, all the servers were using ARM. My computer and your computer, it runs using X86 architecture. That’s just what desktop computers use in their processing. But ARM is what cell phones use, or microcontrollers. It’s just a bit weird. It meant they were on computers that they hadn’t really written many exploits for or understood really well. But Tyler thought this might be to their advantage.
TYLER: One of the things that our team usually tends to be good at is obscure weird things. If it’s ARM or MIPS, or just weird architectures that people don’t see every day, that tends to benefit our team more than others. We went into it and right away, on the first day, right out of the boat, we started winning. We shot up immediately and we were like okay, this is a good start.
JACK: PPP is looking good on Day One. There are a lot of game mechanics you have to think through the whole time. The contest shuts down at night and the conference room doors close so you can’t hack other people at night. What the teams do is they take these puzzles upstairs into the suites and try to find exploits all night long offline, basically. What if you find an exploit right before the room is going to close?
TYLER: Should we save this for tomorrow or should we throw it now? ‘Cause if we start attacking people with it now, they’ll have more time to analyze the network traffic overnight, but also if we wait tomorrow, maybe other teams will find the same bug overnight. There’s all these kind of weird game theoretic questions.
JACK: There’s lots of strategy that has to go on.
TYLER: I’ve heard from a lot of people that some teams don’t like to throw exploits at us because they’re worried that we’ll find the exploit and turn it around and throw it back at them real fast. Similarly, we usually don’t throw exploits against the top teams until we’ve thrown it against the teams we think are weaker for maybe thirty minutes, and then we’ll start to throw it against everyone.
JACK: Day Two now begins. They have a few hours of sleep and are ready for the caffeine to carry them through the day. I’ve talked to a bunch of organizers and players of this Defcon CTF and let me tell you, there’s so much craziness that goes on during these things. It’s bonkers. For instance, one year, one person from a team hid under the desk of another team to listen in on the chatter and the exploits they found. Another story I heard was that one team snuck an Ethernet cable into another team’s router so that they could be on the same network and try to hack into things that way.
The stories are endless and all the shenanigans that go on during the competition. Most of this kind of hacking is allowed. Really the only thing you can’t hack are the organizers. Day Two completes. The score board shows that PPP is still in the lead but now the scores are hidden so they don’t know how much of a lead they have. On Day Three, the scoreboard is completely hidden so nobody knows who’s in the lead. The contest ends on Sunday and the scores are tallied. The team goes to the award ceremony where the winners will be announced.
TYLER: [BACKGROUND TALK] We just sit down and they go through all the competitions and we’re mostly just exhausted and nodding off to sleep during the whole ceremony ‘cause we haven’t really slept in a few days.
GUY: Alright. Hi. I’m Guy from Legitimate Business Syndicate. First place will receive eight black badges. In third place we have Rayon ASRT. [APPLAUSE] In second place we have The Men in Black Hats. [APPLAUSE] And in first place we have PPP, the Plaid Parliament of Pwning. [APPLAUSE]
TYLER: We were expecting it because we worked pretty hard, we were doing well, but it was just a ridiculous feeling after working for so many years. This is year four of doing CTFs and year three or something of doing Defcon. We had put in so much time and energy into working at this competition that it was like – a relief isn’t quite the right word, but it’s a mixture of relief and excitement and happiness.
JACK: They went onstage to receive their awards. All eight of them got their own black badge. Even Geohot got one.
TYLER: After a few hours we’re just sitting around and looking at each other and nodding at each other like yeah, I guess we finally did it. Shit, we finally made it and finished first in the competition.
JACK: In my mind, this means that you threw your hat in and said we want to prove that we’re the best hackers in the world. Anyone who wants to challenge us can come here and challenge us and you proved it. Do you feel that way?
TYLER: Yeah, yeah. I think one of the cool things for us was also – most of the teams that were playing and that had won previously were [00:55:00] these big groups of professionals, people who work doing IT security or working as defense contractors doing security, or the real honest-to-God people who do this for a living. We came in as basically a group of kids. We just kept working our butts off until we could get there. Then to have this real win, there’s no way anyone can question it when you win Defcon CTF. It’s like well, if you beat everyone else there and you’re beating everyone else at all the other CTFs, you are just the best team.
JACK: When I go in there and I look around, I don’t know why I don’t see NSA hackers or some serious black hat hackers that are just like look, we’re gonna totally smoke these guys. They’ve got no chance in hell. How come I don’t see those competitors?
TYLER: I can guarantee you that they are there, having talked to some of them. There are definitely people from those groups who are there. Sometimes they like to stay up in the hotel rooms rather than be downstairs where people are taking pictures and stuff. You know, it’s not like the whole might of the NSA is up against you or something ‘cause that’s a little different. But it is absolutely people who work for governments are there, and there are people who do black hat hacking for a living who are there. It’s probably not the majority of people but it’s not an insignificant proportion of it.
JACK: You see what I mean here? Tyler and his PPP team proved they are the best hackers in the world openly, in a fair contest, for anyone else to challenge them. They beat out people from the NSA, Google, Microsoft, the Koreans, the Russians, you name it. Not only did they beat them here at Defcon, but they beat them all over the world in hundreds of other CTFs they played along the way. PPP was number one. [MUSIC] But now the team, of course they feel good, but they have these new skills and they’ve been doing so many CTFs, they’re like hey, let’s not get rusty here. Let’s keep it going. We’ve already won in 2013. Let’s try again in 2014. They go back to Defcon to try to defend their title as the best hacking team in the world.
TYLER: Yeah, okay. So, we won 2013, 2014. We lost 2015. We were hoping to get three in a row.
JACK: Bummer, they couldn’t get three in a row. But they decided to try again. They go back again in 2016 and win the Defcon CTF then. They go back again in 2017 and win first place again that year. They really, really wanted to win three in a row but they ended up getting second place last year in 2018. At this point PPP has won the Defcon CTF four times. That’s four black badges for Tyler. That is the current record for anyone or any team for number of black badges from Defcon. PPP is the only one with four wins.
Tyler and PPP will be competing this year again at Defcon 27 to try to prove once again their team is the best. Then they plan to go on to try to win three in a row from there. They’ve already made a legacy but now they’re trying to become legends. But their story just boggles my mind in so many ways. Tyler’s been to Defcon nine years in a row now and the only thing he’s experienced there ever is CTFs. He’s never seen a single talk or wandered through the villages or did any workshop or even go to any parties during Defcon.
TYLER: The one kind of exception was not this year or the year before, but the year before that, me and a couple of other people from our team were participants in the DARPA Cyber Grand Challenge which was the big machine CTF thing that DARPA ran. A couple of us participated in that with a company and we won first place in that, and then moved on to the CTF and got first place in that as well.
JACK: I should point out, the people who participate in these CTFs get a ton of job offers and of course, the winners also get even more job offers. I mean, who wouldn’t want to hire the best hackers in the world? Or even the hackers who came in the Top 10? This has been an amazingly great thing for all the members of the PPP’s career. Winning a Defcon black badge is just solid gold to have on their resume. I even saw the NSA one year at Defcon set up a booth and were actively recruiting people. Their booth even said ‘If you’ve won a black badge, please come talk to us.’
Another really cool thing that PPP did was they made their own CTF. It’s called picoCTF and you can play it anytime in the world. It’s on picoctf.org. You don’t even need a special computer. I’ve played through it. It’s great fun and I learned a lot along the way. You basically are given a set of little puzzles and you have to try to solve each one. It starts you out [01:00:00] with easy challenges and you work your way up to the harder stuff. It’s designed for colleges and high schools to get students to learn how to do security and hacking. Since it’s backed by Carnegie Mellon, it’s played by many schools around the world. If you want to get started with hacking, I highly recommend going to picoctf.org and start playing around on their CTF.
TYLER: I guess one other fun fact is that my wife and I actually met on the CTF team which is fun, too.
JACK: She participated on the team?
TYLER: Yeah. She joined the CTF team in 2013 as a Masters student at Carnegie Mellon and we started dating and she’s continued to play CTFs with the team. Yeah, and then we got married a year ago, so that’s exciting.
JACK: That is really cool.
TYLER: Yeah.
JACK: So, this has changed your life dramatically, being on PPP and competing at Defcon. Everything about your life has changed just because of that ride.
TYLER: Yeah, yeah. It’s pretty weird. My job is basically due to being in CTFs. I work at a security company that has – I’d have to sit down and count, but like, several other people from PPP are also part of that company. My wife I met from PPP, and yeah, it’s kind of inundated with reminders of CTFs.
JACK (OUTRO): [OUTRO MUSIC] You’ve been listening to Darknet Diaries. Thanks Tyler, for telling us your story. Good luck at Defcon this year. I’m going to Defcon this year too, and hey, if you’re listening and going too, let’s meet up. I’ve got a number of meetups going on there. Here’s where I’ll be; Thursday, August 8th during the day, I’ll be poolside at Mandalay Bay, hanging out with my friends from CMD. CMD is inviting you to come hang out with us too, but there are a limited amount of people I can get in, so sign up at darknetdiaries.com if you want to come hang out with me there.
Then again on Thursday night, you can find me at The Linq at the 3535 bar. Come on over and we’ll hang out there and get drinks. Nothing else is going on Thursday anyway, so let’s do this. Then Friday night, I’ll be partying with the folks from Tourcon up in the Chandelier Room in The Cosmopolitan from 8:00 p.m. to 11:00. You’re all invited to come, too. Let’s have drinks there. My schedule is going to be posted on darknetdiaries.com so don’t go blowing up my texts trying to find where I am. Just look for my whereabouts there and you’ll find me. This episode is created by me, the benjitsu white belt, Jack Rhysider. Theme music is made by the ba-da-ba-ba-ba Breakmaster Cylinder.
[OUTRO MUSIC ENDS] [END OF RECORDING]