Transcription performed by LeahTranscribes[FULL TRANSCRIPT]
JACK: Hey, my name’s Jack, the host of the show. Before making this podcast my job was looking at my clients’ networks to try to find ways to make them more secure. In other words I was on defense locking things down, hardening systems, securing applications, and trying to turn off everything that didn’t need to be on. The defense team is sometimes known as the Blue Team. I’m on the Blue Team. But one day we paid an attacker to come into our office and see how well I did at securing the network. He was a professional penetration tester and I made him sit right next to me. Attackers like this are said to be on the Red Team and this whole Red Team/Blue Team thing is just a term borrowed from the military where they had drills with attackers and defenders.
Here I am, the Blue Team, and there he was in the desk right next to me, the Red Team. The adversary, the enemy, a hacker. What do I do? Do I sabotage him so he can’t do his job? Do I block his IP from getting anywhere? It wasn’t that I didn’t trust him ‘cause he came from a very trusted company but it was that I was extremely curious at how he works. I wheeled my chair right over to his desk and I watched over his shoulder for the whole week. I was amazed at what he can do. I learned so much that it forever made a permanent impact on the way I see how attackers work. I want to give you that experience so in this episode we’re going to get geeky. We’re going to get really nerdy and crazy technical at times as we watch over the shoulder of a penetration tester to see exactly how they do their work and how they try to get the crown jewels of a company.
PROX (INTRO): [INTRO MUSIC] These are true stories from the dark side of the internet. Presented by Jack Rhysider, this is Darknet Diaries. [INTRO MUSIC ENDS]
JACK: The penetration tester we’re going to watch over the shoulder of goes by the name Tinker. He has a background in the US Marines and has been doing penetration testing for a long time, years. When you do penetration testing for that long you end up knowing a lot about computers, like far, far, far more than the average person and even more than the average IT person. Tinker really is top-notch at knowing how computers work in general so I called him up. [SKYPE CALLING] Hello? Tinker, you there? Hello?
TINKER: The cable just wasn’t pushed in all the way, that’s alright. It’s the nature of all – most issues are cable-related.
JACK: Okay, sounds like we’re gonna be in for a ride here. So let’s start out with just tell us your name and what do you do.
TINKER: Sure. My name is Tinker Secor. I am a penetration tester or Red Teamer, depending on the nature of the engagement. But generally speaking I hack into computers and I break into buildings in order to test my client’s security.
JACK: He’s a typical penetration tester and it’s amazing to me that this job even exists. There are a lot of people who do this as a career. I mean imagine if a brick and mortar-type store tested their security like this too. You’d have paid shoplifters trying to test how good the LP is. LP is loss prevention for those non-criminals out there. You’d pay cat burglars to try to steal paintings from museums and you’d have reformed street gangsters trying to quietly rob a casino. Maybe some of these jobs exist but [00:05:00] in the online world it’s actually very common. Like, I’m pretty sure that if you want to process credit cards at all, you need an audit done on your network which usually requires a penetration tester to come act like a criminal to see if they can hack into your credit card machines. Anyway, Tinker’s been doing this for a long time and he’s really successful at getting into networks. Today, we’re gonna follow along with him on his assignment.
TINKER: It was a large national client within the United States but it kind of stretched within North America and a bit in some other continents.
JACK: Often what a penetration tester will do is try to find a way into the network from the internet, the outside, basically posing as just a rogue hacker online who’s trying to find something open or a website that’s like, exposing data. Tinker had already done some of that for this client and they were happy with the results and they wanted to take this to the next level.
TINKER: They said look, we want to assume that a threat actor has breached the perimeter. We want to assume a threat actor has either broken into the facilities, implanted a rogue device, or maybe an insider threat kind of thing. Generally speaking, this test would cover all of it.
JACK: There’s a term in information security called Defense in Depth and this Chief Information Security Officer, their CISO, felt like their Defense in Depth was so good that they wanted to put it to the test. Basically this concept means you create many layers of security which makes it redundant, even.
TINKER: I started inside. [MUSIC] That’s what we call an internal pen test. The idea is okay, if you plan to fail and fail gracefully you want to see okay, what happens once somebody gets in? A lot of people say it’s game-over but there are so many different ways to breach.
JACK: Again, I want to give a warning here that we’re going to get nerdy and technical in this episode because this story he has paints a perfect picture of what a penetration tester does. I want to get technical because I think it will be a fun opportunity to see exactly how all this works. Oh and there’s also a few cuss words in this one, too. So this company wants Tinker to do a security assessment from the inside of the company. To do that they need to set him up with a temporary job in Marketing.
TINKER: They set me up basically to do content. I went in within the Marketing Department. I assumed the name Jeremy and I was Jeremy from Marketing.
JACK: You get it, right? This Jeremy from Marketing is actually a very good hacker and his goal is to see how much he can hack into in his first week on the job in the marketing department. The IT team and security team and even the marketing team have absolutely no idea that this new guy, Jeremy from Marketing, is an extremely trained hacker who’s highly motivated to just hack into the network and get everything. This isn’t a far-fetched scenario; sometimes you have temps or interns or new hires that get turned into spies to work for another government to see what’s in that network while they’re working there and can you really trust Jeremy from Marketing? Yeah, suppose he did all the interviews but still, I mean he really just did walk right off the street and sit down in your office. Are you gonna give him all the passwords to file servers and logins and to the company’s Facebook account? Ideally your hiring process should vet him but it’s really hard to know if that person is actually trustworthy.
TINKER: There’s really only two people in the entire company who knew who I was and that was the CISO and one of his assistants.
JACK: The CISO is the Chief Information Security Officer and he reports directly to the CEO. He basically got everything sorted out with HR to hire this Jeremy from Marketing.
TINKER: They said look, I can bring in anything I wanted. I could bring in all my hacking gear if I wanted to but I needed to make sure that I didn’t get caught.
JACK: Without being caught is the tricky part. If he had a bunch of antennas sticking out of his desk or even extra laptops all around, it would surely look suspicious. Alright, so he’s all set for his new job. It’s Monday morning and he’s off to work. On the drive into the office he starts to go over his methodology and plan of attack.
TINKER: Get in and do password reconnaissance, active reconnaissance, vulnerability, and misconfigurations that are in numeration, initial breach, lateral movement, pivot, escalation privileges, actions and target, exfiltration, and persistence may be in there if you need to, right. That’s kind of the standard approach.
JACK: That’s his plan of attack. [MUSIC] He drives into the office, parks the car. It’s a typical-looking office building; multiple floors. His office is just one of the floors.
TINKER: I showed up dressed in a button-up with a tie.
JACK: Shows his badge to get in and asks where the Marketing Department is. They introduce him to his new team.
TINKER: They said look, here’s your cubicle, here’s your team. The team was told that I was a contractor. This company used a decent amount of contractors so my being there, my role was fairly normal. I think there might have been another person who’d started a week earlier as an actual content creator within Marketing.
JACK: He takes a look at the computer that was given to him.
TINKER: I came in as an employee, as a contractor, but it was the same thing. They gave me a laptop that had a very [00:10:00] standard – their standard work station image, right. I could use that. It was fairly locked down and they did that on purpose ‘cause they wanted to say hey, what’s available to someone else or what happens if one of their employees clicks on a phish and they have that user-level starting point? They gave me that.
JACK: That’s another good point. When a phish or phishing e-mail is successful, often the hacker will then have access to that person’s computer who opened the phishing e-mail. If somebody in marketing did get phished, this is a great scenario to test whether or not they could get further into the network. I think it’s a good idea to test this.
TINKER: The very first day I came in with just the laptop that they gave me and maybe a Kali image burnt on a USB that I could maybe mount. I had in my backpack my own hack box, just a little Dell laptop loaded with Ubuntu as a base image with some Kali VMs, etc. That’s kind of the rogue device. I had a standard set of equipment starting off. Not a lot was expected of me within the first couple weeks is what my team told me but it was very much watch the security videos, I’m not supposed to click on a phish, that sort of thing. There was a couple things that they wanted me to start working on like an internal SharePoint. Again, nothing major. The culture was hey, we’ll get you spun up over the next two weeks and starting on week three and four you’ll start shadowing people and getting into it.
JACK: This gave him a lot of time to himself to see what was in the network.
TINKER: [MUSIC] The very, very first thing I did was plugged in their standard laptop and just get a feel for it. I wanted to know what’s the username schema. Is it first initial, last name? Does it match the e-mail? It doesn’t always match the e-mail.
JACK: One thing that companies do is give certain IT admins a second username, something like -adm at the end or -admin. A username like this gives you a clue that that person probably has extra access than others.
TINKER: The very first thing I can do is run – pull up command.exe on the workstation. I’m using their tools. I’m not using any malware. Just type in netspaceusers/domain and it will dump out the entire list of all users within that domain. I can do netgroupsdomain/admins, and dump all the domain admins. I can do netgroupsdomaincontrollers/domain and dump out the host names of all the domain controllers.
JACK: These commands spit out a ton of data, giving a list of all usernames, all admins, all domains, and he’s compiling this data to have it handy later in case he needs it. These commands he’s typing aren’t even hacker tools. They’re just standard Windows commands there to help IT administrators do their job. This is all part of the reconnaissance phase.
TINKER: Me running these commands as a user against the domain controller, that’s how a lot of default Active Directory environments are set up. I did this raw just so I could have it offline at night.
JACK: Active Directory is the mechanism that Windows computers authenticate to each other. Hackers love attacking this because it has so much data. It has information on all the users and all the passwords and it has tons of stuff that a hacker can use to escalate their privileges or move on to other systems. It’s a great place to start looking. There’s a lot of standard things to look for which are like low-hanging fruit; known vulnerabilities, best practices that the IT team didn’t follow. One such bad practice is to set the admin password for Jeremy’s laptop through a group policy because this means that the hashed password would be in the group policy and since Jeremy can see the policy he could grab that hash and try to crack it.
TINKER: This place didn’t have that so I tried a lot of the very standard things. Went through and checked some shares just using my own credentials, using guest credentials, no credentials, and did a lot of that stuff, just basic enumeration. [MUSIC] Got a feel for internal SharePoint, internal intranet, that sort of thing. What is available to the user?
JACK: During this time he’s also learning what kind of tools this company may be running internally. This is helpful because if you know for instance, that they’re running SAP then you can start looking up vulnerabilities in SAP. He started building a map of the network.
TINKER: The very first day I’m just – the idea is just sit very still, find out what’s going on in the environment. I kind of learned what’s going around.
JACK: Something a good penetration tester will do is try to be quiet as they can and not do anything to raise suspicion just so that they aren’t detected early and they know what normal looks like. He was careful at what commands he was typing into the computer so that he wouldn’t raise any sort of alarms.
TINKER: The workstation had a bit of antivirus and Endpoint protection. It wasn’t as robust as it could have been but it was definitely there. Endpoint protection is one of two things or both; it either prevents what it deems as malicious software but it can also do a lot of logging.
JACK: Next he took a look at what tasks and services were running on his laptop.
TINKER: Just doing CTRL + ALT + DEL and looking at Task Manager. Notice specific software solution that did a lot of heavy logging.
JACK: This means the computer he was on was sending [00:15:00] all kinds of messages to the log-collector telling it what was going on on that system. If he was doing bad things on that computer, chances are that was going to be logged and someone else could see that and catch him. He didn’t want to raise any suspicions so he stopped pulling data from Active Directory thinking someone might catch him doing it. Another thing he liked to do on his first day is be very visible around the office. He wanted people to know he belonged there and he was part of the Marketing team. He’d take walks around frequently, get some water, go to the bathroom, chat with people. This also let him look around the office a little and scope the place out, see what normal office behavior looks like. He comes back to his desk and sits down and starts to pull out his rogue laptop which is full of all kinds of hacker tools. After the break we’ll see what kind of fun he can have with this, what kind of trouble he can get into. Jeremy from Marketing pulls out a rogue laptop and boots it up.
TINKER: When I did plug in my actual laptop, my hack laptop, the first thing I did is run Wireshark.
JACK: [MUSIC] Wireshark captures all packets coming to that computer. It’s sort of like sitting on the front porch of your house and watching all the traffic going up and down the street. But you really only get to see what’s on your street, not the whole neighborhood. In fact you really only get to see what’s coming in and out of your own driveway. But still, it gives you a good sense of what kind of traffic is going around.
TINKER: I spun up Wireshark and started trapping a lot of – sniffing a lot of the packets.
JACK: Primarily he was looking for…
TINKER: What sort of hardware is on the system; laptop-wise or even server wise. What’s the host’s name schema?
JACK: Because while Wireshark generally only picks up traffic to that computer, it also picks up broadcast traffic, too. These are packets that are intended for everyone on that subnet and computers make a lot of broadcast traffic. By capturing these MAC addresses it will also tell you what kind of systems are on the network because a MAC address contains information on what manufacturer made that device. Again, knowing this lets him blend in better. Now he’s starting to know what kind of exploits he might be able to use.
TINKER: Again, the very first day, especially right at the beginning, all I’m really trying to do is just sit very, very still and listen to what’s going on around me, get a feel for the environment.
JACK: This reminds me of the quote from the spiritual teacher Ram Dass which goes like this, “The quieter you become the more you can hear.”
TINKER: One thing struck me is that they didn’t have a very good NAC Solution.
JACK: NAC is Network Access Control and it’s a technology that gives each individual computer network access. With proper NAC, only computers that the company has authorized are allowed on the network and everything else gets no access at all.
TINKER: It’s to prevent this very specific type of attack where you plug in rogue devices. You should only plug in devices that you know are yours. The problem is having to manage all those assets especially when you come from an open environment where bring your own device and that kind of thing can get into the internal network. Implementing a NAC solution is a simple concept but it’s very, very difficult and typically takes several months to roll out. They did not have network access control so I could plug in my rogue device. But as soon as I got done with the passive sniffing, I want to know what MAC addresses are associated with servers, with laptops, and whatever else I could find. There was sometimes even phones, right. What were the host names for each? I changed my host name to match their schema and I changed the first three octets of my MAC address to match their hardware, then I randomized the last three. I did do that in order to blend in.
JACK: He’s like Rambo now, painting himself with mud to avoid being detected.
TINKER: The long and short is I tried a couple different things and ended up in a position where I was confident that I could start doing more active reconnaissance without being found out.
JACK: Day one is over. Jeremy from Marketing quietly loads up his laptops and heads home for the day. He’s feeling confident at this point. He’s collected a lot of data and starts to get the feel for the environment and starting to think about what kind of attacks he can use. The next day he comes in, fires up Responder. [MUSIC] In my opinion Responder is an amazing tool. It’s like cheating almost for hackers. Here’s how I think it works. Okay, so if you have an office job and you use a computer, do you have shared drives on that computer? If you’re on a Windows machine you might have the M: drive or the I: drive or the Z: drive. This is some shared network folder that other people in the office can access, too. Okay, so suppose your Windows computer needs to connect to this shared network drive. There’s a number of things it has to do.
Usually the shared drive is like a host name. It’s not always an IP address so the first thing your computer needs to do to connect to that drive is to resolve the host name. That’s what DNS is for. Now, there’s a DNS order of operation here. Your computer will first check the internal host file to see if it has a hard-coded IP address for that server. If not it’ll then go to the DNS server to see if it knows what the IP address is. Normally the DNS server knows the IP but sometimes it fails. It fails [00:20:00] because maybe you’re on the wrong network, you’re not VPN’d in, the shared server might be offline. If you ask the DNS servers what’s the IP address for this host name and the DNS server doesn’t know, then what does your computer do next?
It asks everyone on the subnet hey; does anyone here know what IP address is for this shared drive? That’s when Responder kicks in. Responder is a lying, cheating, sneaky, ugly-looking guy who says yeah, I know exactly what IP address is for that server. Your computer says oh great, what is it? Responder says it’s me, I’m that shared drive even though it’s not. Your computer says oh, okay. Great, let me in then. The Responder says okay sure, no problem but first I want to make sure you’re allowed so tell me your password. Your computer then gives Responder the password.
TINKER: You’re not gonna send your raw password in clear. You’re going to send your authentication hash. This sent its typically Net-NTLMv2 or v1 and that’s a salted hash.
JACK: Do you see what happened here? Responder is a hacker tool that just lies and tricks a computer on the local subnet into giving computers the hashed password to that computer. It’s unbelievably good at this too, and it’s almost impossible to detect since you have to have sensors on that local subnet to spot it.
TINKER: Generally speaking I will run Responder twice a day for maybe fifteen to twenty minutes and even intermittently at that. I run it right in the morning and right at noon when people are logging into their computers. They’re logging in the morning, they’re logging in back from lunch. That’s when a lot of that traffic comes that you can track. Generally speaking, depending on how big the environment is, I’ll pull down ten, twenty hashes quite easily.
JACK: He fires up Responder in the morning and waits for hashes to come in.
TINKER: I pulled down maybe five to fifteen hashes total.
JACK: [MUSIC] Nice. With this, if he can crack a few of these, he can then work his way into the network and get some more privileges from someone else.
TINKER: I pulled down these hashes. Again, they’re Net-NTLMv2 hashes.
JACK: He loads these up into his GPU password-cracking rig which is offsite at his own office.
TINKER: It was eight Nvidia GTX 1080 TIs mounted in a 4U Rack server.
JACK: Whoa, what a monster. That’s like an $8,000 computer. Basically what it’ll do is take those hashes he picked up from Responder and load them up into this computer. Then he runs a tool called Hashcat to cycle through billions of passwords a second to try to find the matching password. A computer like that can try every word in the dictionary in like, under a second. Then it will go and try adding numbers to the ends of words, or special symbols like a dollar sign instead of an S. It will keep trying passwords more random and more complex over time until it finds a match. This is brute force password-cracking.
TINKER: The lowest I’ve ever gotten is like, a 20% crack rate, one in five. Somebody is statistically – I’m gonna get that much. Usually on those immediate standard stuff I’ll get anywhere from 50% to 75% crack rate. NTLM, I think at that time we could do something like 300 billion guesses per second. I want to say with Net-NTLMv2 it was only and I say only, it was only somewhere between I don’t know, six to twelve billion guesses per second. So not too many. I ran it against our standard dictionaries and our rule sets and I didn’t crack any.
JACK: His monster of a cracking station tried billions and billions of passwords and found absolutely no matches on any of the hashes. This was a big surprise. This never happens.
TINKER: Usually when that happens it means my tools are broken, like an update broke it or something. I went and checked all my tools, put in known hashes. I test hashes to make sure my tools work and those cracked just fine. I did troubleshooting on my own stuff and it was working. That’s when I stopped and went back into the intranet and looked up a security policy.
JACK: The security policy is going to tell you the minimum length of what a password must be and how many special characters and digits that have to be in it.
TINKER: Sure enough, I believe they had a minimum of twelve character passwords. At that point started to become passphrases. I’m a big advocate of passphrases. Password you can crack fairly easily but a passphrase, ideally four or five different words, completely random, that’s much more robust that what we have today. I said okay, alright, that’s fine. It’s still upper lower number and symbol, three out of four, and it still changes every ninety days. That means that people aren’t gonna create a really hard-to-figure-out passphrase. They’re gonna create something that they can remember and then iterate on it. I changed a little bit of my attack settings to [00:25:00] account for a minimum twelve character and basically just picked longer words and longer numbers at the end.
I did four digit numbers so you usually get the last four digits of somebody’s Social Security Number, the current year, something along those lines. I just picked longer words. I went onto the website and did a full word scrape from all their stuff including a lot of stuff from the internal internet to get the cultural thing. You get local sports teams, you get local schools, you get local street addresses, and any kind of mascots or what have you that they really identify with. You also get cultural phrases. I finally did that and I finally got, I want to say a good tidy handful. I finally got a couple of clear text passwords. But let me tell you, my equipment was sweating after that.
JACK: [MUSIC] Okay, now Jeremy from Marketing has a few other employees’ passwords and really, getting this was not that difficult. Running Responder is really simple to do and he’s using off-the-shelf parts to build a computer to crack these hashes. It sounds amazing but if you know what you’re doing it’s really not that hard. Now that he has a few usernames and passwords, he cracks a small smile because it feels like a big win but he doesn’t want to let the other people in Marketing know that he’s doing something. He begins to try to figure out what he can do with these accounts.
TINKER: There’s a couple things that you can do. You can immediately try to log into a workstation. I know their laptop is on the subnet and I do a very targeted spray with their username and their password against all of it.
JACK: What targeted spray means is that he’s subtly trying to remotely log into a computer using these passwords he found. But none of the logins worked.
TINKER: The error message amounted to good password but not authorized to log in, not within the group that can do remote logins which is fascinating, right? That is absolutely something that you can do but I very rarely see. By default that’s not set up which to me, that was kind of my first shock to the system. Aside from the fact – I’m like, here I am. They require twelve character passphrases and they don’t allow common users to do remote logins for their own boxes. I’m starting to go okay, something’s going on here. What sort of place have they put me in?
JACK: [MUSIC] Okay, these users cannot log in remotely but they have to be able to log in normally like when they’re at a workstation. He logs out of his company-issued workstation and he tries to log in with one of these usernames. Earlier he was pulling stuff from Active Directory but because there was so much logging enabled in his laptop, he had to stop because he didn’t want to bring attention to himself. But now that he has someone else’s password and he can act like someone else for a little while, he can use that to gather more information with.
TINKER: What I did with these credentials – ‘cause I didn’t want to use my own credentials. I didn’t want anything to be tied back to me as a person. I used these stolen credentials and at that point I logged into SYSVOL on the main controller, pulled all of Active Directory. You can mount SYSVOL and get all the group post preference. You get scripts in there that the domain admins will run and other IT will run. You can sometimes pull hardcoded credentials out of those. I pulled all the users, usernames, and host names, and a myriad of other things. With the domain controller I pulled out all the information of all the users including the groups and had a lot of recon at that point.
It was rather successful, that first go. That’s just users. Even though I wasn’t able to log into the user laptop, I was able to at least interact with the domain controller in the way that Windows allows for. But still at this point I don’t have much, right. JACK: Right. So now we’re nearing the end of the second day and Jeremy from Marketing is really struggling to get anywhere in this network. Sure, he has a few passwords but he’s very limited at what he can do with them. Usually by now he’s deep in the network with starting to get access to Active Directory servers, something, something bigger than this. But he’s got nothing so far.
TINKER: At this point I’m like okay well, I still have good user credentials. What am I gonna do? The first thing I tried to do is just log into their e-mail using either OWA or Office 365 or whatever single sign-on that they were using. I was able to get in. [MUSIC] They did not have multi-factor authentication set up on e-mail. As luck would have it though, one of the very first e-mails that I read just rifling through someone else’s e-mail was an e-mail saying hey, be advised next week we’re implementing multi-factor authentication in e-mail so be prepared to set that up. I’m like woo! I got in early enough to where I didn’t have to do that.
JACK: Now he starts looking through the e-mails to see if he can find anything of importance; maybe IT e-mailed them passwords at one point or something else that might be [00:30:00] helpful. He found a password but it was for a third-party tool like a tax-assessor’s website or something. Really, that’s it. He even looked for internal nodes that Outlook sometimes stores for users to see if they just wrote their passwords down on that or something but there was nothing there. No help. He tried a new approach.
TINKER: Get into their single sign-on, their internet.
JACK: What I see companies doing today is they create a single portal for employees to log into which then provides them access to all the tools. This is a called single sign-on because you log in once and gets you access to many things.
TINKER: Single sign-on that’s not set up properly or securely is a hacker’s dream. All of the things are in one tidy little group and you have full access to it. I’ve taken down entire organizations where single sign-on was there, like a hub of applications, if you will. One of them gave me a whole lot of access. This place though, I got in from the outside which I tried to do because I didn’t want it to originate from the inside. From the outside it required multi-factor authentication just to log in.
JACK: When he says from the outside he means from outside the company. The single sign-on portal can be accessed from anywhere in the world. It’s right on the internet but ah, multi-factor authentication makes it much harder to log in. This is where you need both a password and a six-digit token code generated on your phone or something to get in. He tried to get into the portal from his laptop on the inside of the company and it didn’t require a multi-factor authentication. Yeah, now he’s in. He looks to see what’s there. A bunch of different apps, payroll stuff, client databases, control panels, okay. He’s feeling like he’s getting somewhere now.
TINKER: [MUSIC] I clicked on one of the apps and each individual app required its own separate multi-factor authentication login. I was like what? What kind of lockdown prison is this place, right?
JACK: It’s the opposite of single sign-on.
TINKER: Well exactly, and so what they’ve done is they set it up properly. They had one place for everybody to go to but man, you better have your soft token or something set up. At this point I’m getting kind of heated. I’m like Tinker, you’re losing time. You better hack into something. I had gotten a couple of things; I hacked into e-mail but it didn’t give me much. I already have some findings. I’ve got a report here that’s building of things they can tighten up but nothing significant. Pen testers, your dopamine, your adrenaline is going off those really big hacks, right. I still at this point have not gained access to another workstation beyond my own.
JACK: He takes a little break, gets a drink of water, resets himself, and sits back down.
TINKER: I need to lat move. I need to start getting onto some servers. I need to start going and at least targeting some crown jewels here but I’ve got nothing. One thing caught my eye though, in the single sign-on, was Citrix. I, as an attacker, I love Citrix ‘cause what Citrix is, is it amounts to Remote Desktop through the browser. Generally if there’s something from Citrix, generally it’s a server of some sort that’s hosting internal applications or something else it’s serving up. But I see Citrix and I usually go for it ‘cause I can really get a good thing there, maybe dump some memory. I click on Citrix and it asks me for multi-factor authentication and it says it’s gonna send it via SMS to this user’s cell phone. But it only gives the last four digits of the cell phone.
JACK: While yes, it’s possible to hijack the cell phone, do SIM swapping or something, that’s technically illegal and he’s not allowed to do illegal things as a penetration tester. All he knows about this person is their name, their username, their password, and the last four digits of their cell phone.
TINKER: But I have their e-mail so I type in the last four of their phone number into the search bar within this person’s e-mail and I pull up one of their signatures that has their full phone number. Okay, I have their full phone number, I have their name, I have who they are, I have everything that amounts to them within this environment. Let’s bypass multi-factor authentication.
JACK: [MUSIC] Okay, so here’s what he’s gotta do. He’s gotta click on that login to Citrix which will then send a text message to the phone of that user and he’s gotta somehow get that text message, enter it into this website all within sixty seconds before the code expires. This isn’t gonna be easy.
TINKER: I have this phone number. I called the person. I gotta make sure none of the people around me are hearing me at this point, right? But I put on my headset, put on my own phone, and I called this person. They answer; Jane from Accounting. I lie to them and then I say hey, I’m somebody [00:35:0] else from IT and we’re gonna migrate your Citrix since it’s within single sign-on. This person has no idea what I’m talking about. They’re not IT and so they’re like, all they hear is computer gobbly gook. They’re like yeah, that sounds fine. Why are you calling me? I’m like well, I’m gonna send you a pin number that I need you to read back to me that authenticates that it is your account but I need you to know as your IT, I will never ask for your password.
Again, this gives them a sense of security. I obviously already had their password but it gives them a sense of security and says okay, well they’re not gonna ask for my password. That’s what matters so I’ll read off this pin. I said okay, I’m about to send you a text message. You’ll receive it from my server, Citrix, right, and they go okay. I go ahead and click Send MFA and they go okay, I got the text. You just need me to read this to you? I go yup, just read it to me. She read it to me. I typed it in within the minute ‘cause you only have about sixty seconds and it logged me in. I go okay, well just letting you know, and this is right before lunch, go out and take a long lunch or don’t interact with Citrix for at least two hours while we do the migration.
She said okay, great, sounds good. I go okay, sounds good. Bye. I’m like okay great, finally. I’ve got into Citrix. I log into Citrix and there’s no applications. There’s no computers. It’s an empty Citrix instance. I’ve hacked my way into a broom closet. There’s nothing here. I just bypassed multi-factor authentication through a solid social engineering exploit after cracking this person’s password which was hard to crack to begin with. A day and a half has arrived up to this very point and I’ve got nothing, absolutely nothing.
JACK: At this point if you were to look over at Jeremy from Marketing you’d see him sweating and shaking.
TINKER: I found that generally when I’m doing a soc eng or social engineering attack or when I’m doing a physical break-in or what have you, when I’m doing it I’m calm, cool, and collected. I’m usually sweating like you wouldn’t believe but my demeanor is on point. Afterwards though, man, that adrenaline rush, it comes crashing down and I will, I shit you not, I will physically shake. I’m probably one of the most loud, outspoken introverts you’ve ever met. I am confident in front of people.
I used to be a Sergeant in the Marine Corps and when you got a platoon full of trained killers and you’re trying to get them to do what you want them to do you have to develop confidence, or at least a projection of confidence. You gotta lie through your teeth, right. I will have confidence; I’m socially adept but it still drains me, especially when I’m doing something like this that I have to put on a heavy mask and so yeah, I get out of that. I’m sweating. It took me probably about a full hour to prepare for it as I did an in-depth research on this person. I’ve got nothing.
JACK: Usually he’s a lot further along at this point on his penetration test and it’s just making him really worry that he’s not gonna find anything.
TINKER: I go screw it; I’m gonna go all out at this point. As people start leaving to head home I say hey, I’m gonna stay back and finish trying to knock out some more of these onboarding videos and nobody seemed to mind.
JACK: [MUSIC] He sits and waits, sitting in his cubicle watching everyone leave the office. He keeps peering over and seeing if anyone has left. He sits back down and waits some more.
TINKER: I waited until the cleaning crew came. I waited until the cleaning crew left. I think I was the only one on that floor. I believe there was maybe a couple other people working on different floors but I was the only one on that floor.
JACK: He starts walking by every cubicle looking to see if any computers were logged in while that person went home. No computers were left logged in but he does see some people did leave their computers behind.
TINKER: I said you know what, I’m just gonna steal laptops. I don’t even care at this point. I go start plugging into my colleague’s laptops that left their laptop there but two things kind of jumped out. The biggest thing was they had encrypted hard drives so even if I mount and boot from USB which I was able to do, I couldn’t mount their hard drive. It was encrypted and so I couldn’t pull off any kind of local admin hash.
JACK: Ah, again blocked. All these little safeguards in place are really giving him a hard time to get anywhere. Even when nobody is in the office he still can’t access people’s laptops. He’s tired, he’s hungry, he’s nervous, and his frustration is just building.
TINKER: I said screw it, I’m gonna go after the IT shack.
JACK: Okay, so the IT shack is the room where the IT help desk keeps all the computers. Like, they probably have ten, twenty, a hundred computers in there. If he can get into this room he’d have access to a lot of workstations and he thinks maybe if he gets his hands on these he can get somewhere.
TINKER: Going up to the IT shack, during the previous [00:40:00] couple days, whenever I’d get up and walk I’d try to make people see me so that they know I’m supposed to be there. I’m doing kind of reconnaissance on where everything was. By that time I knew where the break room was, I knew where the guards hung out, I knew where the IT shack was. First thing I had done previously was check where all the cameras were and there was not a camera looking at the door to the IT shack. I knew that visually I was fine but when I went to the IT shack I wanted to make sure there was no one around the corner. I wanted to sit there and pause.
JACK: He stands and waits right around the corner from the IT shack door and he tries to listen to see if anyone is around.
TINKER: In order to really cast your hearing out to hear as best you can, there’s a couple things that you can do. One is slow down your own breathing to slow your heart down ‘cause if your heart’s beating that’ll actually fill your ears with a pump, pump, pump of blood so you need to calm yourself completely down. I tilt my head down. I looked down because I want to focus. I’m not necessarily closing my eyes but I want to focus on my hearing so I’m making sure that I’m not looking at anything in particular.
Then I open my mouth slightly and the reason why you do that is you have your jaw muscles; when you have a clenched or closed mouth, your jaw muscles will actually come right up near your ear canal and kind of close it off slightly. You can try this even at home. Get to a place where you have your jaw closed and just listen for a while. You can do this at night and then open your mouth comfortably wide open. You don’t want to strain yourself but comfortably open to move your jaw bone away from your ear canal. You’ll find that that opens your ear canal wide open and you can hear a lot more. I say get context; not only can you hear further but you can hear more things. Does that make sense?
JACK: Yeah, I love pen testers. It’s like you’re Felix the Cat; you have a bag full of tricks and you never know when you need it but you just have them ready to go.
TINKER: That’s the thing, we try to be Jacks of all trades and you try to study as many things as possible. I’ve pulled that trick out of being in the Marine Corps when you have to do night missions. Little things that you pick up along the way that I tell you, they come into play.
JACK: After standing around the corner from the IT shack door for a good thirty seconds listening for anything, it seems like the coast is clear. Time to move in. He gets his lock picks ready and is prepared to move to the door and start picking the lock. He turns the corner and to his surprise a door stopper is stuck in the door which is just barely keeping the door open slightly. Alright, lucky break. He won’t have to pick the lock now. But this gave him a totally different sensation. Was someone in there? If so, is he gonna have to wait even longer for them to leave? He tries to look through the crack but he can’t see much. Screw it, he’s going in.
TINKER: [MUSIC] I walk in real quick and I see stacks and stacks of laptops.
JACK: Nobody was in the room. Someone accidentally left the door open all night.
TINKER: I come to find out later on it was literally a person had forgotten their keys and so they left it propped open to go to the restroom and never came back to shut the door.
JACK: He’s standing in the IT shack in front of like, seventy-five laptops or more.
TINKER: I’ve got an option here. I can stay in the shack and do all my things there or I can take what I need and move over to my desk or conference room, whatever, and do it elsewhere. There’s a give and take of both. If I stay in the shack no one will see me hacking into these computers and so if I stay in there I protect myself against the bulk majority of people that will be walking through in the middle of the night if they came in after hours for whatever reason. But if I stay in there and an IT person comes in, which plenty of IT people work overnight or have to come in for a call or something, I’m caught red-handed. I’ve got no excuse for being in there.
I make a call that I’m gonna start hauling as many of these computers to my desk and then from my desk, bring them up and if someone comes by hopefully I’ve got them tucked away or whatever and I can pretend that it’s mine or whatever. I can hear them coming, I can hide it and go back to work. I start taking armloads, like full handfuls of these laptops to my cubicle area and stacking them underneath my desk. I ended up grabbing – I made, I don’t know, three or four trips and ended up grabbing about thirty laptops before I said you know what, this is probably enough. I went back to my cubicle at this point, shut the door, and just started trying to boot from USB and mount as many hard drives as possible.
JACK: He starts going through each laptop one by one, spending hours on them trying to find if any of them have an unencrypted drive. He finds two in the whole stack that were either old images or didn’t get encrypted. Now that he has an unencrypted hard drive he dumps the local administrator hash from that laptop. Once he has this hash he starts running it through that monster password cracking station he has and he tries to crack the admin password.
TINKER: I crack it rather quickly. It was actually the company name and the year with a capital first letter on the company name, a very weak password. [00:45:00] I was like are you kidding me? Everything else I’ve found has been amazing and this is it?
JACK: Just to test the password he tries logging into his own workstation with it and the admin password worked. Bingo. This means the admin password that he just found is likely the admin password for all the user’s laptops in the office which should allow him to log into any user’s laptop. Finally he’s making progress. This is a big break. He starts putting the laptops back in the IT shack.
TINKER: I took pictures of where they were so that – they were out of order by this point but I put them, I mean, stacked about as precisely as I possibly could. Anything I touch, I put it back as closely as I found it so that it doesn’t look like it’s been disturbed. I’m like okay, well great. I got a lucky break. I can now use this to spray everything, right? At this point I’m beat and I go home and I sleep.
JACK: Feeling rested and happy that he has an admin hash and a password, he comes back into the office finally ready to dig deep into the network.
TINKER: I come back in and okay, I’ve got the best thing that I have; I’ve got a local admin hash. First thing I try to do is pass the hash which just uses the actual hash as a password.
JACK: Which is a great technique. Often when you’re logging into another computer your computer hashes your password and gives it to that other computer to log into so if you have the hash, just give him that and that could authenticate you. But in this case it wasn’t working. There was some kind of error but last night he cracked the local admin password. IT administrators will often reuse this password so this local admin password might actually work on every laptop in the whole company. He tries to remotely log into another computer using this local admin and password.
TINKER: But it wasn’t working. The local admin hash was not letting me log into – again, it comes up as valid credential but you’re not allowed to log in. I’m like, this is asinine. Even the local admin needs to be able to log in. It’s not letting me.
JACK: He logs into his own laptop again as local admin to try to figure out what’s going on.
TINKER: When I log in with the local admin password they don’t have access to the full computer. [MUSIC] The local admin, the local administrator, does not have full access to the rest of the hard drive. It doesn’t have access to the user level which – this doesn’t make sense ‘cause that’s how it’s set up. That admin has access to everything. Again, I’m punching myself in the face. It turned out that they were using a third-party non-Microsoft tool to do access control and user control, etc.
JACK: This is quite impressive. While the password was probably used for everyone’s laptop the admin user doesn’t have that many rights at all. I’ve never heard of this myself. But yet, this is another safeguard that this company has put in place in case this password got leaked which was really hard to find to begin with. This would stop them even further.
TINKER: Now I’m angry. At this point I can pop a shell from my computer but – to my rogue computer. With this pass I can log in anywhere and get a shell if I have physical access to somewhere but it doesn’t give me any rights. I don’t have anything. I’ve got this tiny little niche.
JACK: I would be angry too. So many of his exploits and techniques should have given him access to the whole company by now but this company was foiling his every single move and all his techniques. Now he’s tired of trying to fly under the radar. He’s ready to try an exploit on another computer that might make a little noise. From there, if he can get into a computer he can see if there’s anything good on that and move around to another computer. He scans his own computer to see if it has a vulnerability that he can exploit.
TINKER: I tried a variety of them but the one that worked was unquoted service path which the way Windows works, is if say you want to run a program at startup and it’s designed to run this program at startup. But one of the folders that you run this program in has spaces in it. If you don’t put quotes around that full path, what Windows will do is attempt to run up to the space as if that word, say it was something that said Citrix space Server. That’s a BS one. It would try to run Citrix as an executable first before it tried to run Citrix server as a directory. If you go in there, if you have ReadWrite and you can create a program, a malicious program that’s named Citrix as opposed to Citrix space Server, it will run your program as in this sense, system because this system was calling it. I found a directory that let me do this. It wasn’t Citrix space Server but I’m like okay, great. But I ran a check to see if I had ReadWrite and it said I didn’t. At this point I’m like well, fuck it. I’m done. This is horrible.
JACK: [MUSIC] Without the ability to write to the remote computer, he’s unable to exploit that thing. Because it said he didn’t have the ability to write, he just gave up at this point, totally out of ideas. He put his elbows on his desk and he put his head in his hands, completely [00:50:00] dumbfounded. He’s now on day three and still has not gained access to any computer outside his own and a couple of powered-off ones in the IT shack. His report and findings so far looked dismal. This has been the hardest assignment he’s ever had. Now day three is over and he heads home. Morning comes. It’s now Thursday and he’s getting ready to go into the office.
TINKER: I call up an associate of mine. I told him here’s everything I did, but I ran the check to see if I had privs and it said I did not have writability. He goes well, did you try it anyways? I’m like oh, goddammit. No, I didn’t. I just assumed. I went ahead and tried to write to it and I could. [MUSIC] Even though Windows came back and told me I couldn’t, I was allowed to write. This kind of tells you don’t listen to the output of the tools that you’re trying to hack into.
Turned out again, this third party software that ran all the access control, the third party allowed them to write even though the native Windows didn’t. Third party superseded native Windows. At this point I now have a meaningful way to escalate privileges to system level and I tried it out. I went with my colleague; he wrote a stager and I wrote the malware for it. He wrote an executable that will then call my PowerShell, reverse shell. I think it was just a tweaked version of Veil or some sort of PowerShell, remote shell. We tested on my own workstation, unplugged from the domain. At this point I’m getting kind of gutsy here and it worked.
JACK: Okay, so for this exploit here’s what needs to happen. First he has to put the exploit on a USB drive and then physically take it to another computer. He would do this while a person wasn’t at their desk.
TINKER: Log out of their user, log in as the local admin, drop two sets of malware.
JACK: Drop it in as in copy it from the USB to the computer and then log out as admin. Then when the user comes back and logs in, his malware should give him remote access to that computer.
TINKER: Even though I found this way to do it, it’s still under – a person has to break in by this point to have physical control. I’m like you know what? Screw it. We’re gonna do this.
JACK: He thinks over this plan. This is a risky move but if done right could get him access to that person’s computer. So whose laptop would be worth getting into? Maybe the CEO’s. Hm, yeah, but they weren’t at this office. Who else? The IT team. [MUSIC] Perfect.
TINKER: I’m gonna go straight for IT. I’m gonna hit IT and I’m gonna take them down. I’m gonna get system level remote access.
JACK: With this remote access you’d be able to do everything including reading the CEO’s e-mails. The plan was to wait until lunch when he could go over to the IT team’s computers and put this malware on it. He waits and waits, peering over his cubicle from time to time. He watches a few more of those onboarding videos for Marketing and waits until lunch. The Marketing team asks him to go to lunch and he’s like no, no, I’m fine but they kept insisting and he’s like no, really, I want to stay so his team leaves him behind. He thinks okay, now’s a good time.
TINKER: I set up a listener on my rogue machine and I go start walking. I’ve got a little thumb drive with my malware on it and their antivirus didn’t detect it ‘cause we kept it very low level. I go over to the IT area and I shit you not, the bulk majority of IT are sitting there eating lunch at their desk. I’m like a) that’s not healthy. That’s not good work. You need to get away from your computer. You need to stand up. You need to walk. But are you kidding me? This is not saying this is a valid defense technique, seriously. One person can be there if they really need to but people need to get up and move away from their computers, even IT. I’m frustrated.
I start walking and pacing around. At this point I’m getting kind of heated. I’m losing my cool. I go around a corner; I finally find an area that doesn’t have anybody and sure enough it’s Finance. I’m gonna take down Finance. I go up there. As I’m up there I see one lady sitting down next to one of these cubicles and I’m just gonna go for it. I tell her; I go look, I’m IT. I’m about to do some updates. She goes okay, sounds great. I go ahead and do it. I pop it up and about thirty seconds it takes me to log out, log back in, drop the malware into the correct folder, log out again, and then leave.
JACK: He goes back to his desk and waits. Now what he’s waiting for is that lady from Finance to finish her lunch and to log back into her computer. He pulls Metasploit up and waits and watches. Metasploit is like a hacker’s tool bag. It contains hundreds of exploits and tools to hack into stuff. He’s got this hack all set up. He’s staring at his screen and it says the listener is running and waiting for mode activity or something like that. If everything is set up right, when she comes back and logs into her computer he’ll then have remote access to that lady’s computer in Finance. The screen will say Meterpreter Session One [00:55:00] Open. He waits for activity. Nothing. Nothing. He peeks over the cubicle wall sometimes to see if he can see her but he can’t see anything. He waits longer, longer. He just keeps waiting. Come on, lady. The wait is killing him. It’s now been forty-five minutes at this point and he’s starting to think it didn’t work. She had to be back by now and for some reason, whatever reason, the malware just didn’t work.
TINKER: I’m about to give up. I see Meterpreter Session One Open, right, and I’m like oh yeah, there it is. Then I see Meterpreter Session Two, Session Three, Session – it popped eight shells. It tried to call this thing eight different times. I’m like yes, I’m in, alright! [MUSIC] I start rifling through this person’s computer. I get persistence. I actually get a couple passwords for finance, some small ones. Right as I’m about to start dumping memory I lose my connection. The session closed. I’m like oh, no. No, no, no. I have been without sleep, I’ve gone too far. What the hell happened to my shell? I get up and I make a beeline right to that lady’s laptop ‘cause I’m gonna go pop another shell, you know. I’m like get out of my way!
As I go, I round this corner and this precious little old lady reminds me of my grandmother; she’s looking up at this IT guy and she’s like no, I don’t understand. They told me you guys were updating my computer. Right as I come around heated, she turned and kind of glances at me and goes him! It was him! I’m like oh, fuck! I let out this high-pitched seventh-grade girl scream, you know, and I turned around and right as I turned around there’s two more IT guys right there. I’m like oh, shit. [LAUGHING] And they’re looking right at me like who the fuck are you and why are you calling yourself IT? They sat me down and they said who are you? What are you doing with these computers? Kind of thing.
JACK: When a penetration tester gets caught like this, they have to think. Should I try to escape this situation or should I just tell them I’m here working for the CISO? In this case he decided to say I’m working for the CISO.
TINKER: At this point I’m at the end of my rope. I’ve done a very thorough test even more than what I’ve gone into here over a good week. I’ve stayed late, I’ve done everything I can think of and even learned some new tricks along the way. While I was able to find several different things, an easy password on local admin, even a shared password even though they used the other ones, some clear text credentials here and there. A lady gave me her pin over the phone. A lot of these things they could tighten up. Long and short, they stopped me and we brought in the CISO. He said hey, good job. Here’s the situation. He’s okay. We went into an initial debrief with what I had done and how they found me. I asked them; I go look, I ran the safest frickin’ shell that I could run. I even tested it against your antivirus and your antivirus didn’t catch it. I was only there for thirty minutes. How did you find me? They said you were running PowerShell from a finance computer and finance doesn’t run PowerShell. The only people that run PowerShell are IT and maybe some of our Devs, you know, DevSecOps or DevOps.
JACK: Power Shell is kind of like a super command line tool in Windows and yeah, only technical people ever use it or need it. The Finance Department would never run it so this sort of behavior is like anomaly-detection. That lady in Finance has never, in all the years she’s ever worked there ran Power Shell. But this exploit did and since it was so out of the ordinary is how he got caught.
TINKER: But yeah, it was one of the toughest places and I like telling that story when Blue Team kicked Red Team’s ass because it showed what worked. Like I said, I still found a lot of different things that they could tighten up. They weren’t perfect by any stretch of the imagination but they had such robust security that they were able to not only detect me but act on it. It’s not enough to detect an attack. You have to do proper response and containment and I tell you what, they had all three.
JACK: I imagine the IT team was proud of what they did to stop him but they remained focused and serious as Tinker went over the report.
TINKER: They were taking ready notes. They didn’t gloat. They didn’t rub it in and they also didn’t take offense of the things that I found. They were very professional and they said they appreciated it and looked forward to my full report. It was the epitome of professionalism.
JACK: I like this story because not only do we get to see what a penetration tester does but we also get to see what steps a company can take to make it really hard for hackers to get in. Because the harder it is, the more resource a hacker has to have. They have to have more time or more processing power or more people or more exploits or something. The harder you make it for them the more motivated [01:00:00] they have to be to get through it. They’ll probably just give up and move onto something else if it’s too hard. Just to recap what worked here for this company, they had multiple layers of security, defense and depth. They had a minimum of twelve character password policy which made passwords hard to crack. They had two-factor authentication almost everywhere. They limited access to each user which made it hard to do any remote logins. The local admin had very limited access and the logging that was on everyone’s computer allowed them to detect and find this hacker within minutes of him doing an exploit. All this added up creates a nightmare scenario for Tinker and will probably be enough to create a nightmare scenario for any other hacker.
TINKER: The CISO was very – it was just professionalism from the top down. You could tell that this was a culture of continual self-introspection and self-awareness as it related to their environment and continual improval. He went in with the idea that at a certain point, you can’t have a perfectly secure system or no one’s going to be able to use it. If someone can use it you can – an attacker can emulate that user in some form or fashion. He’s like, we’re getting to the point where we have risk acceptance. If you have to break into my place and physically access a computer and do all this kind of stuff, at that point the only people I’m really worried about are really high-end criminal groups; the NSA in Mossad. If it takes the NSA in Mossad to hack into my place, fine. We’ll accept that.
JACK: Thank you so much for sharing this story with us.
TINKER: Cheers, cheers. Thank you for having me here to tell that story. A quick shout out to the Dallas Hackers Association. I’ve never met a more vile bunch of criminals, thieves, con artists, and hackers in my life but there’s some good folks.
JACK (OUTRO): [OUTRO MUSIC] You’ve been listening to Darknet Diaries. Thank you to Tinker for telling us this amazing story and teaching us about pen testing. You should follow him on Twitter because he tells a lot more stories like this. His name there is @TinkerSec. Also thanks to Proximity Sound for doing that voice intro. That was really cool. Darknet Diaries is going to do a bit of a rebrand in the next few weeks with a new logo, web page, shirts, stickers. I’m super excited to roll it out so look for that soon. This show is made by me, the President of D-Corp, Jack Rhysider. Intro music is by Breakmaster Cylinder who you could always find hanging out at the Red Wheelbarrow BBQ.
[OUTRO MUSIC ENDS]
[END OF RECORDING]