This is a write-up for a talk I presented at BSides Cymru 2019. It’s my first attempt at doing this kind of a write-up; I am a relatively new speaker and I figured it would be important to have this because:
- I forgot to say a lot of things on the talk (shouldn’t have happened but hey, it did);
- I meant to run live subtitles for the benefit of those who cannot hear me speaking; this didn’t quite work,
- I complain a lot about having to watch videos instead of just reading and getting to the core content, so here is me eating my own dog food.
I am of course inviting you to have a look at the video if you missed it, it does show that I can indeed launch calc.exe remotely, otherwise if you prefer to read and learn more about the topic, please read on! But please mind that I am typing this from memory and my notes and after I did the talk, next time I will write if before and will try to keep it more consistent.
 Who is Meadow
Meadow is a 6th generation nation-state sponsored… I don’t like titles, that’s why I like to laugh a little at the long and pompous titles and here on the right is my business card. I don’t call myself a hacker because I think hackers are the romantic heroes from the movie of the same title, I consider myself a (hardware) security engineer. Although I may need to stop calling myself that as the recent Twitter shitstorms showed… because I don’t have an engineering degree. What I am presenting here is a reflection of what I’m doing day-to-day: trying to find that one odd criminal that will do something odd and uncommon and own us all - because they didn’t get a memo that what they’re trying to do is impossible.
You might have seen me on one of my recent talks: BSides London 2018 where I talked about the Green Padlock of Doom, we are not seeing the green padlock anymore (almost), but the problem is still there; I also did a talk earlier this year for BSides Leeds on how we need to go deeper in hardware hacking - and focus more on how stuff works because it pays off. And this year I did a first-ever hardware badge for BSides London 2019; and a writeup for this badge is close to being done (right after I finish this post I will come back to it!).
 What is going to happen today
In this talk I will focus on three things:
- I will talk about daily struggles when trying to differentiate between skids and noise and the real criminals lurking somewhere in the dark;
- there will be a demo of the evil keyboard I have made as a prop for this talk;
- and, as no talk is good if it only presents problems and no solutions, I will try to show why a device like the one I will be showing is a problem and what can we do to fix this problem.
The talk is a snapshot, or a nightly build, of my research into this topic. It’s not about the keyboard - I would very much like to do a separate talk just about the hardware (if anyone’s interested, that is!), but rather about the problem that HID attacks create and how it is a go-to for modern criminals.
 How to recognise different types of criminals from quite a long way away
The thing about recognising a proper criminal is that they’re probably not where you’d expect them to be, nor would they behave in such a way. This is why the ‘skid risk’ sign is in order: more often than not we are burried under skid noise making us look away (and still, more often than not this is on purpose). I use the ‘skid’ term quite loosely - it’s not that only random script kiddies are generating this noise because, honestly, if a simple ‘skid’ method works this will be a way in for the criminals, because they, other than us, nerds, want to get in and out the easiest way possible. And herein lies the problem…
 Hackers are not criminals
But before that, a public service announcement that I keep on repeating whenever I can: hackers are not criminals and every time you say that hackers have breached a database and published 8 billion passwords God kills a kitten. And the image here is what makes it even worse, this evergreen image of a ‘hacker’ in a hoodie that is ‘organising a massive data breach’. No, stop, DON’T DO THIS, please.
At the same time those images and the media noise associated with it makes us forget who the the real enemy are here, and those enemies are not skids: of course, if a person can own your network using a stock Rubber Ducky they will not try to do something more sophisticated, because why? Those are basics that you should have covered.
Back to the problem: those are basic things. When you spin up a VM that has a web server and open it up to the Internet you will instantly start getting hundreds of ‘hacking attempts’ at your phpmyadmin, your unpatched RDP, your SMBv1 - which you should have disabled/patched already. (Or deleted, in case of phpmyadmin). But this the noise that keeps our attention away from things that matter: with software/networking you have your 7 billion ‘hacking attempts’ per day. With hardware you have USB HID attacks you can buy online: all the BadUSBs and RubberDuckies out there: this is the noise you should be able to filter out because you can easily get those devices, analyse how they work, even grab their PIDs and VIDs (every USB device identifies itself with those, more on that later) and know when they are plugged into a machine in your estate. I know, it’s not that easy if you have a really big estate, change always takes time.
 A criminal world outside our bubble
In this noise, though, there lurks a real danger and of that you should really be worried about. Skids are not your problem, but those people are. There is a whole world of criminality out there that you are probably aware of: drugs and arms deaing, human trafficking, proper organised crime stuff. And they know that there is money to be made in information security. We like to think that this world is Somewhere Else and we are being hacked by, well hackers. That is not the case anymore, if it was ever - this criminal world is poking in and is bringing their own habits: not only violence, but…
 Offline is cultural, not from watching movies
… also staying offline, or at least staying quiet. Those people will not go on Twitter or even DarkWeb and brag about it, they will not try to sell it because that attracts attention, obviously. They will make something that is, effectively, a 0-day althogugh it’s not like you can patch a problem and make it go away. They will stay offline because it’s cultural, it’s not because they watched Mr. Robot, where this screen came from, but because it’s their nature. For us, online people, that is sometimes a big conceptual problem to overcome:
 Just because you don’t see it doesn’t mean it’s not there
we have never seen a trace of this anywhere, so it’s probably an one-off? Well, we don’t know, that is the thing. The attacks I am talking about in this talk are so extremely easy to pull off it would be a, well, crime to think that no-one ever did that before. I really think the problem exists, but we are just too busy looking at the noise to notice. With this talk and this prop keyboard I have here I would like to prove that spit-and-tape scenario is really feasible.
 S in USB doesn’t stand for security
This brings me to my prop and the demo of what I put together, but first: a word on why I focused on USB.
 S in USB stands for S…trouble
The reason why we have this problem today is because the USB protocol was never designed with security in mind, because why would it, in 1995 and earlier? And because it is quite unique in that two sides have to work in bringing the change, the manufacturers and the OS vendors, it’s quite difficult to drop one implementation and replace it with another. So we had USB 1.1 released in 1995, and layer upon layer we crawled to USB 2.0 in 2000, then to USB 3.0 in 2008, still layer upon layer - and your keyboard from 1999, if there is such a thing (I honestly cannot remember when first USB keyboards appeared) it will still work in your USB 3.0 port.
Of course USB3 and USB4 released a few weeks back, they have new security features - more so because they allow a direct memory access, and that’s a completely different attack platform (and USB3 is insanely more complicated than 2.0) - but they are backwards compatible with USB 2.0 (I may be wrong about USB4 though, I hope it drops the old standards altogether). How much, then, it matters that we have new security features in newer protocols?
 It matters precisely zero
 Work on new standards is important
This of course doesn’t mean that the new developments are not important. They are, of course, but for us who have to worry about all the tech we have to secure today, it doesn’t matter that much - until, that is, you can safely disable USB 2.0 and your company will not shut down because of that.
 You can have security over USB
All that I’ve said doesn’t mean that you cannot have security over USB; a perfect example is the FIDO2 world with the hardware keys that are compliant with it (Yubikey) - they take an iherently insecure standard that is USB 2.0 and make it secure, but that security is controlled on both sides - by the hardware manufacturers and the operating systems. There’s no way keyboard vendors would willingly make their keyboards more secure, while increasing the cost, so that we can get rid of the old HID attack.
 How complex is the USB protocol
Exploiting USB is not easy, because it’s a seriously fast protocol that relies on timings and you cannot stop and analyse what is happening because the host will kick you out. A lot of things are happening in the first phase, the setup or enumeration phase: on the right-hand side here you have a dump (from Saleae) of the first half a second of a full speed USB connectivity and here we are on a line 97000-something (left-hand side shows a low-speed dump, which is 10 times slower). I don’t have an USB analyser and even if I did I’m not sure if I’d trust it unless it was a really good one (Teledyne LeCroy, for example). In this particular case I tried to use a Windows-based Eltima USB Analyzer 4.0 but… it crashes my keyboard so now i’d need another analyser to find out why… draw your conclusions from this. I am doing something quite weird here and I need to see everything - and that means a LOT of scrolling, as the setup phase can be over 300k transactions, or lines in a text file. In this case we see the host asking the peripheral (address 0, endpoint 0, as this is an enumeration phase) for a device descriptor and the device responds with:
- I will send 18 bytes;
- it will be a device descriptor;
- I am USB 0110 (bytes 2 and 3, they are 0x10 and 0x01 because USB is little-endian so the least significant byte goes first);
- few empty bytes;
- my packet size is 8 bytes so I will send the remaining 10 bytes in the next 2 transactions.
And so it goes, back and forth. It takes a while to get a grasp of it but my point here is that it’s perfectly understandable and exploitable if one has enough time or they’re pressed against a wall with a deadline, as I was. Everything is documented online, there are no secrets here.
 This attack is not new
It is very important for me to point out that this type of an attack is not new, and I did hear a few times a very valid point in response to me submitting this talk: this is just another HID attack. That is quite true, have a look at some examples of the tools of the trade:
… and plenty more, if you start searching the rabbit hole goes deep (and those are only the recent ones). There are also devices that allow you analyse USB and perform attacks similar to mine:
Why am I talking about this, just before showing off with my invention? To point out that just because those attacks exist in the wild doesn’t mean they are not the problem. They are and even more so when someone comes up with something original that doesn’t exists anywhere else, like a proper criminal would.
 D e m o
[Video shows me doing what I’m describint below]
(The “Wild General” name is just a random codename generator thing, I like to name my projects this way as it’s easier to refer to them this way than ‘the keyboard I made for Wales’).
- they keyboard is connected via an active USB hub to my Surface I am presenting from;
- I have SSH-ed to my c2 in Azure before the talk;
- the keyboard, once powered up, started trying to connect to an AP on my phone, I have programmed the credentials earlier;
- once connected, the keyboard has completed its USB setup process and can be used as any other keyboard out there;
- once it manages to connect to the WiFi each keypress will be sent to the host (my Surface) but also to my c2 in Azure in the form of keyboard scan codes;
- and it is indeed true - on my c2 terminal I see lines of ‘ping’ every 3 seconds, and those are essentially that, pings, but also…
- … every keypress is send back as well - and it’s not limited to letters, because the data between the real keyboard inside and the host is passed back and forth transparently it doesn’t need to translate and understand it, it just forwards it. What that means is that not only letters, numbers and such are being sent out but any keypress combination including modifier keys, but just as well any media buttons, volume controls, whatever is in the standard…
- one way connectivity is not enough, I can compose a message and send it from my C2 back to the keyboard: I can make it type ‘Hello BSides Wales’ remotely;
- and that, of course opens up a whole world of possibilities because I can also send a string to make it press Win+R, c, a, l, c, Enter - and it opens up calc.exe.
With spit and tape then I have a full two way comms with my wild-general evil keyboard.
 Few components, some Dremel work, some spit, some tape
It took a bit of engineering; some may say extreme engineering, to squeeze it all into the keyboard chassis: I think cutting a PCB with Dremel is not something you’d see recommended in a hardware hacking guide but in this case it was whatever it takes. There was Dremel, Apache, Perl - whatever was at hand to make it work. Just cut off the bits you don’t need! What is inside then?
 What is inside?
In this case you can peek inside because it’s a demo, normally you wouldn’t be able to. I will not tell you how this could be done because I am usually on the receiving end of such tricks and there’s no reason for me to make my life more difficult.
There are two MAX3421 controllers, one for the keyboard side, one for the host side, and Arduino IoT 33 that is controlling them and talking over Wifi to the C2. And hot glue, of course, as it’s indispensible in those situations (however me as an engineer, I hate it) There’s also a voltage regulator that takes the 5V from USB and makes it a 3V3 required by both MAX3421 and the Arduino.
 Deep cuts on the Arduino
It also required some non-standard approach to the Arduino itself as well. I had to file off one side of the castellated pads of Arduino because it was about 1mm to wide to fit above the up-arrow key. Again, probably not something you’d expect from a good-engineering or even hardware hacking handbook but this is the ‘whatever it takes’ approach, we need to be aware that there are some who will do it. And, as you could see, it worked so who cares if it’s not elegant?
 Can it be detected?
Realistically, that is. Imagine you have an estate of 20k enpoints, each of them being a possible target to this type of attack. What can be done here?
 Trying to detect the keyboard
There are several ways to go about this:
Monitoring PID/VID and other strings: that’s the least effective thing that can be done. You may know that each USB device identifies itself with several, device-specific strings (product name, manufacturer) and hex values (PID, VID - product and vendor ID). There’s a problem here: they can be easily changed. This is true for the devices I spoke about before, as well as the really simple solutions like an Arduino keyboard where those parameters can be changed by modifying few values in some header files. With my keyboard - it takes all the data from one end and passes it to the other, so whatever you saw with a ‘clean’ keyboard, you will see the same with my keyboard. So that is a big no.
Measuring current drawn by the device: this has some legs, but not many. First, there’s a common misconception that all USB hosts measure current: they do not. Part of the setup phase for an USB device is a question “how much current you want?” to which the device answers with A Value. It’s just a text response, nothing more. Second, my device draws more current than a normal keyboard because I didn’t care, but I could make it so that it would draw more or less the same amount of current. So, again, nope.
Analysing the current: now this is a good one, and I would have no way to combat that. What I mean by analysing the current is using a device like ChipWhisperer - it does many fantastic things but one of the core methods it uses to perform them is analysing the current in a very, very granual way (with a high resolution). It’s so granual (fast) that it can detect changes in the amount current drawn as the device is flipping a bit from 1 to 0. With that you can create a pattern for a normal keyboard and then compare it with others (using ML, possibly). And as said, there’s no way I can make a device that will behave exactly the same as a ‘clean’ keyboard. So it’s a win? Well, 20k keyboards to check…
Detecting WLAN connection: yeah, no. First, the device can connect to a local WiFi in which case you have a chance to find it (taking into consideration that everyone has a phone now), or map the traffic, but… that’s what I meant talking about skid noise earlier: this device is doing https GET requests to an endpoint in Azure and getting HTTP response codes in return. You can detect that, unless you are burried under the phpmyadmin noise and know what to look for. But I can connect it to an AP on a phone and give it to someone who visits the office frequently (like a cleaning person); they don’t need to do anything, just be there with the phone. You can detect that WiFi connection as well… again, if you know what to look for and are not burried under the aforementioned noise…
Disabling USB 2.0 - and that is a win! Seriously, if someone could pull this off, that would kill off all the attacks of this sort. The problem is, even though it is possible, no one in their sane mind will do this because there are so many devices that rely on this, it’s just not possible. If you can, though, then you’re golden.
 The problem is bigger on the inside
Bigger because when you think about this - if someone is able to come into your office and swap a keyboard, they can do lots of other things: plant an old school bug, a listening device, for starters. And those can be so small nowadays that you really have no chance in detecting them. So, what now, sit down and cry? Well first and foremost, don’t give way to paranoia because there’s honestly no way to check everything. And if you’ve seen MG’s O.MG Cable (which you should) - this really extends to every cable you have, Apple or otherwise. No hope then?
 Are we screwed?
There is hope. And, no talk is good when the message is that we are screwed and thank you, see you next time. So my role here is to present you with a possible solutions to the problem presented, even if the solutions seem very distant from the core of the problem.
 Attribution is important
But first, a word about attribution: however I like to joke about some attributions out there, especially if the method is fully documented: me as a criminal can read as well and can see how you arrived at your conclusions and next time I will drop you a surprise egg. But at the same time I deeply believe that attribution is important and we, the defenders, should be doing our best to try to find out where a device (if we find one) came from, who made it and so forth, because ultimately we want those attacks to stop and the only way this can happen if law enforcement, like NCSC, has some help from us. It may seem like a futile excercise the result of which will be shelved and no-one from LE will ever look at it again, but it’s worth the effort.
 Multifactor yourself
And this brings me to the key point of the talk. I understand that this topic has been out there for a very long time and possibly everyone knows now what MFA is but in this case this really makes an attack like mine pointless. Sure, the criminals will get your emails (sent), your web history (if typed, not clicked) and so on, but the main loot: passwords, passphrases will be useless to them. Of course there will be some pass(words¦phrases) that will still be of value but probably not really worth the effort. Don’t be the low hanging fruit. Yes, it’s probably not nice to divert the attacks to those who are lagging but we have been talking about MFA for so long now… (I know that it is not that easy, ‘just deploy MFA’. I KNOW. But awareness is the good starting point and I hope that talks like mine can help to drive the point through.)
 Zerotrust everyone
I think what the slide says sums it all up: your network is my network - yeah, and? If you don’t know what ZeroTrust is you really should find out. The model, and the implementation that Google did under the BeyondCorp name is a shift from location-based authentication to authentication based on MFA. That is: it doesn’t matter that you’re on a corporate network, because that gives you access to precisely nothing. You still need to authenticate with every service, and not only you but all the automated services as well - to reiterate the point, you owning a network gives you nothing. With my keyboard you certainly can do it, it can serve as a gateway if configured this way but again: so what? This furthers the pointlesness of such attacks, if your corporate network is your local coffee shop network it really doesn’t matter that someone made an evil keyboard, Meadow’s Wild General.
 Too long; fell asleep
Time to wake up and sum it all up! If you only have space to remember one slide from the talk, please let it be this one:
- Good criminals don’t talk and don’t brag about what they did, don’t take the fact that you haven’t seen anything about the particular attack as a proof that maybe you dreamed it up. There’s a whole criminal world out there that Does Not Talk;
- USB 2.0 is problematic, to say the least, but it’s not going away. Complaining about how bad it is for security doesn’t help, we need to move past that and try to manage the risk associated with it;
- Do your best to filter out the skid noise: phpmyadmin and RubberDucky, those are problems you can address with minimal effort and they will help you to focus on those odd events that can get you into real trouble;
- there is always a way in and out, the time of building bigger, taller walls to prevent exfil are behind us and with the current level of hardware development: honestly, there WILL be a way. It’s time to move forward and…
- MFA yourself and ZeroTrust everyone. Make the crimials go elsewhere, owning such an infrastructure is really next level and you will have some time to prepare for that while criminals are busy elsewhere;
- don’t buy snake oil: when vendors tell you they ‘protect against USB attacks’ what they usually mean is… USB flash drives. This is so common it hurts. I will encourage you to build a keyboard like this and show them where it’s at…
- … and that’s why we need more of such devices like this, to keep applying pressure on vendors - both the ‘security vendors’ and those who provide us with operating systems: the more we show that it is a problem, the better!
I got one question, and a very good one (although I didn’t hear it exactly because of the noise), am I paranoid about every cable and every device around me?
I am not. I could be, especially that I really, really know each hole that can fit a device that will somehow exfil the data (and that inludes listening devices, cameras, you name it). But I am not paranoid because you can spend every waking moment searching for them and still miss that one. This is the sad reality that we have right now and it’s just a risk that we have to accept - while trying to minimise it of course. If I started searching I’d probably find something. But that is not where modern security should be because there will always be another way in.
 Thank you!
Thank you very much for reading and/or watching, as always it’s been an amazing experience for me and I hope to continue on this and other hardware-related topic on the next security con :)