Transcript 0:00 Hello and welcome to episode thirty-six of the IoT For All podcast. I am your host, Ryan Chacon, and I am joined today by Shannon Lee as my co-host. On today's episode, we have Ken Munro, a partner at Pen Test Partners. 0:13 Ken is a security researcher with a focus on embedded systems, which is a natural fit for the IoT space, obviously. Ken's focus is to responsibly find and disclose security vulnerabilities to device manufacturers. 0:27 So simply put, him and the other partners and workers at Pen Test Partners basically go out and buy smart devices and break them to try to find security flaws. A pretty awesome job if you ask me. 0:40 He regularly blogs and writes about various, um, findings for newspapers, industry magazines, in an effort to really get beyond the unhelpful scaremongering put out there by many security vendors. 0:52 He has also become a voice for reform and legislative change in the largely unregulated IoT space, briefing UK and US government departments, as well as being involved with various EU consumer councils. 1:04 He also has a pretty awesome TED Talk, which I'm sure we will link up somewhere in the description, that I definitely think you should check out. Now, on this episode, we get into a lot of different topics. 1:12 We start out by talking about what the device testing process looks like, how liability is handled in cases where a device would be tested, issues are then found, but the manufacturer is not really open to do the change or the feedback. 1:25 The device then gets out into the market, and then issues occur. So who's really at, at fault there? Um, we talk about why security is often an afterthought in IoT, unfortunately. 1:36 What size companies is he seeing the lack of security, uh, in their devices impact the most when it comes to being put out of business due to a security failure then resulting in, in, um, with their devices, and then in affecting the actual customer? 1:49 Um, we jump around a little bit after that and talk about the reporting process when issues are found with devices. 1:57 And then outside of cost, what are the contributing factors, um, to the cause of these sec-security vulnerabilities? You know, where does, where does it, um, where do the issues really lie? 2:06 You know, who, who's causing these issues to occur? And then also, what can be done to fix security holes once a product is launched out in the market, people are using it, um, if anything at all? 2:17 Um, this kind of goes into over-the-air updates, you know, if you have that capability, if you don't, kind of what the situation entails and what your options really are. Um, how should... 2:26 I guess, how wary should consumers really be of products coming from smaller companies and startups? 2:31 A lot of times we go on sites like Indiegogo, Kickstarter, and we see these awesome tech gadgets and we wanna buy them, but what should we really be thinking about when we're buying something from a, uh, smaller, maybe less proven company or startup? 2:44 Uh, we jump or jump back into talking about how IoT companies are dealing with the growing threat of cyber criminals and the potential threat to their businesses. 2:53 What is it going to take to start driving change across the industry? And then the, we kind of start finishing up with a few other topics. 2:59 Um, we get into, like, device regulations in states like California and Massachusetts, and how those are being adopted or how could they potentially be adopted by other states. 3:07 Kinda like, what do we need to see happen in those states with these new coming regulations in order for other states to start adopting these, um, these changes as well? 3:15 Uh, we also get into what advice can be given to consumers when it comes to buying an IoT device. And the last question I ask Ken, which is kind of interesting, is should we be scared of IoT? 3:26 So make sure you stick around and listen to that answer. Um, basically, I, I promise you there's ton of value in this episode. It's one of my favorite episodes. 3:33 I know I say that a lot, but each episode is just, you know, something different, something more exciting, so I'm hoping you guys really enjoy this one. 3:39 So without further ado, we hope you enjoy this episode with Ken Munro of Pen Test Partners. Welcome, Ken, to the IoT For All show. How are things going this week for you? Hey, going really well. 3:49 We're getting prepared for, uh, Black Hat and DEF CON, the, the hacker summer camp next week. Oh, can you... I was gonna ask you to introduce yourself, but I'm actually already curious. It's in Vegas. 3:58 [laughs] Can, can you tell me more? What are, what, yeah, [laughs] what does that entail? Yeah. 4:01 It's, it's a, it's a great excuse to go to Vegas for a week, uh, and do lots of really cool re- security research, uh, looking at some, particularly for us, embedded systems, so IoT devices. 4:11 But we're also really interested in, um, vehicles, car security, and aviation as well. That sounds awesome. My, my excuse to go to Vegas every year is CES, so [laughs] at least- Oh. Well, yeah. 4:23 Like DEF CON, Re- and Black Hat are like the CES of the, like, security world. Oh, okay. Yeah. Gotcha. Oh, yeah. Well, I would say- Totally. Well, they're CES for me. Yeah, I'd, I'd love to do CES. Just it... 4:33 for me, it's like, um, going like a child in a sweet shop, you know, looking for candy, because- It is... every one of those smart products at CES, I just wanna have a go at. 4:40 I want to test the security and make sure it's, uh, safe. Right. Right. We should take him with us next year. [laughs] Yeah, yeah. We'll, we'll, we'll get free passes if you wanna come with us next year. Oh. 4:49 Just let us know. The, the, the trail of destruction would be, uh- [laughs]... would be serious. Yeah. 4:54 We just have to make sure we get all your, like, candid thoughts recorded so that we can turn those into some, uh, nice audio episodes. [laughs] Um, but yeah, how about, how about we have you, um, introduce yourself. 5:04 Um, obviously, I'm sure the audience already knows by now that it's gonna be into some level of security talk with devices, but if you can give a quick background, um, you know, kind of who you are, how you got into this space, that kind of thing. 5:14 Yeah, sure. So, um, I'm an IoT security researcher, and I work with a team of, uh, really cool guys and girls who, um, look at the security of, uh, of systems. 5:23 But we're particularly interested in embedded systems, hence why we stumbled into the Internet of Things four, five years ago. And, uh, what we do is we, we do a lot of our own vanilla research. 5:32 So we go out, we buy smart things, and then we break them. 5:35 We find security flaws, the objective being we can share those with the vendors involved, with the industry, and make things better for everybody so that manufacturers start to improve the security of the products for us, so we, as consumers, become safer. 5:49 That sounds great. Before I get into it, I just want... the audience obviously knows Shannon's voice, but Shannon is here with me as well. Hello. 5:54 [laughs] Um, so, so Ken, tell us a little bit more about that process of the testing of products. Is this something that you guys are going out, finding products, and testing yourselves? 6:04 Is this something that customers, or sorry, that manufacturers, um, and device builders are coming to you and saying, "Hey, test this out for me. Find flaws so we can fix them ahead of time"? 6:13 You know, is it solicited, unsolicited? How does that work? 6:16 And then how do you kind of report anything that you findWithout causing too much of a problem with, you know, maybe the device manufacturers being upset [chuckles] that their device is- [chuckles]... 6:25 not as secure as maybe they thought it was. Yeah, sure. So, uh, it's twofold. 6:29 So we do a lot of work where, um, device manufacturers, certainly the more, the more forward-thinking manufacturers, come to us for advice early on. 6:35 So we can give them some guidance so they choose the right chip, ma- do the right code, and everything's nice and safe. But unfortunately, we also do a lot of our own vanilla research. 6:44 So we go and buy our own products for our own use, reverse engineer them, find out what's wrong with them, and try and report those vulnerabilities privately to the vendors involved. Okay. 6:52 The problem is, is because IoT's relatively new and a lot of the companies are relatively startup, is they're not used to security researchers, and they're not very good at receiving reports- Mm-hmm... 7:03 about problems with their products, and that's when things often start to go a bit wrong. 7:07 So sometimes you'll see in, in, in the media reports of IoT security vulnerabilities, those are where the disclosure process has gone wrong, where we've tried really, really hard to give away everything we've found completely for free so the vendor can fix it, and instead of embracing that and going, "Okay, cool. 7:22 Thank you. Move on. Fix it," they start threatening us- Mm... or just ignoring us, stonewalling us, and that- that's really not a great way to be. Does... How does that play into liability down the road, though? 7:36 Like, if you guys have gone through this process and, you know, made them aware and documented security breaches and they don't fix it, gets to consumers and something happens, is there a liability on, on their part? 7:48 Or how does that, how is that usually handled? So it really depends. Um, there's very little liability regulation- Okay... around the world for, for that sort of issue. 7:57 Now, obviously there are some laws around, so in, in the EU we have GDPR, which, um, can be, can be used if an organization's breached. 8:04 But often, oftentimes we're trying to get information to vendors before that happens, so we can stop people's data being exposed. Mm-hmm. 8:11 You know, so we can stop, uh, children's data being used, children's voices being exploited. We're, we're trying to stop that happening. 8:17 But so often, you know, often when a, when a organization's in startup mode, they're trying to get a product to market, security's the last thing on the list of things they're- Right... thinking about. 8:26 Yeah, we've actually had other security guests on the show. They don't, not as... It's a little bit different kind of, um, application than, than you are. 8:33 They're, like, maybe a security company than a, than a, than someone who's testing these products out, breaking them, finding out what's wrong with them, and they said the same thing, that security is, like, an afterthought, um, for a lot of these device manufacturers, and that's a real problem in IoT. 8:49 Yeah, it's, it's a huge issue because, you know, let's say, you know, you're, uh, you're making your product, you're probably prototyping using some hobbyist chips to get your product working. 8:57 Maybe you go out to get some funding. Everything's great. You've got a product. You're starting the marketing machine going. 9:02 And then somebody, someone along the lines just before you go live says, "Hey, did, did we do security? Did, did we protect people's data?" Mm-hmm. By that point it's j- it's just too late. 9:11 And that's- Why do you think-... unfortunately- Like, why, why do you think people are waiting for so long? Is it because they don't have the in-house resources? It's expensive? It's difficult? 9:20 It's, it's expensive if you get it, if you try and bake it in too late. Okay. And I think the big problem is security just gets thought about too late. 9:28 If you thought about security on day one, it'll have almost zero impact on the cost of developing your product. It should do anyway. You're ultimately preventing- However, the problem happens-... 9:36 them from having billi- sorry, having y- an ultimate breach and them being fined millions and billions of dollars, though, at that point. Yeah. 9:44 So we've seen IoT vendors pretty much disappear, go out of business as a result of either a breach or a security vulnerability not being dealt with well. 9:55 Or we saw one, um, several years ago, um, one, one of the very first we, we started looking at where, um, the US Senate, some senators asked questions, formal questions of the IoT vendor, and they disappeared. 10:06 That was it. Mm. That was, that was... You know, what should've been a great successful startup doing cool things, bringing new tech to the market for our c- children to play with and learn from. Right. 10:14 And instead, 'cause they got security wrong, they went to the wall. Jeez. So what size companies are, are you seeing kind of doing this, like, disappearing or, you know, going out of business because of the security? 10:24 Are they s- all sizes or are they usually s- more smaller startups 'cause they just can't afford that? 10:29 Yeah, and that's, that's the key actually, is, is i- in startup phase, if you don't get security right at the beginning, it can be very expensive to bake it in later. Okay. 10:36 So if you were to compare that with someone like, I don't know, a Google, um- Mm-hmm... 10:39 who's got, you know, huge resources, huge reputation at stake, you know, y- yes, they've made security mistakes, but they've got the resources to fix them and also the willing to fix them. 10:47 Whereas if you're a small startup and, you know, it's gonna cost, you know, tens of thousands of bucks to r- bake some security in at the last minute, what do you do? Do you go out of business? Right. 10:56 Do you stop shipping? Do you recall your product? Or do, or do you just carry on and hope? 10:59 Yeah, I mean, there's a reputation at stake too for a lot of these larger companies as well 'cause the media attention that they get for having a device that gets out there and doesn't work or has security breach and really causes issue with the users. 11:10 But the users are still using it. Totally. Right. Yeah. Yeah, exactly. Um- Yeah, totally. 11:15 It's, it's kind of a, a real shame when that happens, but y- the, the time when these vulnerabilities make it into the press is, into the media is when, you know, the, the vendors haven't responded, you know? Mm. 11:24 Where they've done a, a bad job. Yep. You know, where they just haven't listened to the researchers. When you are finding these issues with devices, are you, um, 11:32 k- going through a formal process to report them to them or is there ever a situation where you f- you guys just buy a device if you're curious and trying find issues and then it gets out to the public or is it really confidential until it's, you know, I guess you have permission to kind of put that, the, the issues you found with the devices? 11:49 Like I... For instance, in your TED Talk you talked about a number of devices which I'm sure you... 11:53 I, I'm guessing you've, you've worked with the cus- company to help improve the security and kind of found what you, you know, the, the mistakes they've made. 12:01 Is that a normal kind of situation or are there ever times where you guys just buy stuff, break it and then talk about it? So we go through a, a process called responsible disclosure which, um, people at Google follow. 12:14 Okay. So we'll try and approach the vendor. We'll, we'll, we'll tell them. We establish contact. And only if there's, there's been no response typically for around 90 days- Mm-hmm... would we then go. 12:23 But we- our objective is to try and get these problems fixed. Right. You know, we, we want the world to be better. We want things to be safer. 12:28 But if the vendor refuses or just refuses to engage, discuss or anything, then actually I think we have, we have an obligation to consumers- I agreeHow do you guys decide on what to make or test? 12:39 [laughs] That's a great one, isn't it? Mm-hmm. Often people give us a tip. You know, they'll say, "Hey, have you looked at this?" Okay. 12:45 Or we'll be out on, on Amazon just having a look at devices, or just random things turn up. My colleagues see things, "Hey, let's have a look at this one." And yeah, I've g- I've got a crazy selection. 12:54 In fact, on my desk right now, this is random, I've got a, a kids smart tracker watch, a fingerprint and Bluetooth padlock. Uh-huh. I've got a cryptocurrency wallet. 13:03 And really weird, I've got, I've got a, a smart, um, heated drinks cup, which is quite cool. [laughs] Wow. Yeah, those are all... 13:10 I mean, uh, if you're being able to play with these gadgets all day, I'm sure that would be a- [laughs] I'm just... 13:15 If you define your job to somebody and say, "I get really cool technology products and break them," I'm sure that, like, it's probably the dream of so many people. 13:22 But you guys have a responsibility, and at least you take it upon yourself to have this level of responsibility to consumers, so you know, that's a, I think, a very noble thing but also a very important thing. 13:32 Yeah, I mean, and, uh, what are our motivations? Our motivations are d- are broadly to try and make things better. Right. 13:36 'Cause, y- you know, a- as part of that, along the way, you know, I'm sure people will come and find us and maybe ask for some help, and that's great. 13:43 But actually, our, our prime goal is to try and make, um, security better. Mm-hmm. And, uh, a, a couple of great examples we've seen. So we did some research on a, uh, a smart talking doll called My Friend- Mm-hmm... 13:51 Kayla. She's a kids doll, interactive. Kid can talk to her, it, and the doll can respond. 13:57 And we noticed, crikey, four and a half years ago, that actually you could listen to the audio from that doll from the next door house or out on the street, and you could also speak to the child through the doll. 14:07 So someone who was creeping on your kids- Yeah... could spy on them, their audio, or talk to them. So we tried to re- tried to report this. Vendor dismissed it as a prank, which we were quite offended about. 14:18 We tried to escalate it. Just weren't paying attention. And down the line, this doll has been banned in several, um, countries in the EU, and there's also... 14:29 It was, uh, transpired that she was the catalyst for California Senate Bill 327, which bans insecure IoT for California residents. Wow. 14:37 So kinda cool that something good came out of some of the research that we published. Yeah. 14:41 You know, I'd far rather that actually the vendor had fixed the bug and everything was good, and we had, you know, kids talking dolls. But instead it's ended up like this. And how did you guys come across the doll? 14:50 Was it- [laughs] Yeah. Well, I think it was coming up to, to Christmas- Mm... time. Was it four, four and a half years ago? Maybe five years ago now. And, um, I think she was being trailed quite heavily on the TV. 15:01 Ah, okay. There were a lot of commercials going on. And one of, one of my colleagues saw her and said, "We've got to get this talking doll." Right. 15:06 So of course, we went out there, we bought one at great expense, and within a couple of days, one of my colleagues had figured out how she worked. 15:12 This little sweet, innocent talking doll, and within a few minutes, we'd managed to make her swear like a docker. Geez. But then we realized there were, there were more serious problems- Mm-hmm... 15:22 afoot with a lack of security on Bluetooth. Wow. Yeah. That's in- that's insane. That would be, uh... I mean, just having the doll start, you know, using profanity around my, around a child would be... 15:33 You know, I know some people are like, "Ah, just, you know, that's just unfortunate and entertaining." But the whole creeping on and listening and talking, I think that would be... That's a little nuts. 15:41 Which is interesting because I don't- Yeah... 15:43 understand why, you know, consumers know about this, or maybe they're not educated enough to really understand it, but why are we still buying all of these products and not taking the steps to- Yeah... 15:50 put them on a private, uh, wireless network or whatever it is? Yeah. I think a lot of it is, is a lack of consumer understanding. 15:58 So, you know, I think many, many of us just make an assumption that if a product's on the market, it's gonna look after us, our data safety, right? Mm-hmm. You, you, you kinda hope that. 16:05 You'd, you'd hope that, you know, governments and consumer organizations were, were championing the cause of safety. But I think many of them have been caught on the hop. 16:12 Um, IT, IoT is moving so quickly, it's moving so fast, that regulation i- is struggling to keep pace. 16:19 Um, there's a lot of efforts going on in, in the US, there's a lot in, um, in the EU to regulate the space, but regulators are playing catch up, and that's the big problem we have right now. Wow. Yeah. 16:27 The government's always behind. [laughs] Yeah, yeah. Yeah, although, do you know what? It was just l- just last month actually, I think, uh, US Senate, there's been another draft bill about IoT cybersecurity submitted. 16:38 So, you know, that, hopefully that'll get, uh, into committee stage soon, and we'll start getting some, some regulation in the coming months and years. 16:43 So outside of cost, um, uh, the cost it takes to kind of, you know, fix these security issues or even just have them tested ahead of time, what is contributing to these vulnerabilities? Is it poor design? 16:56 Is it, you know, uh, you know, when these devices are just... I don't know. I don't wanna say stupidity, but is it stupidity by basically having devices with default passwords? 17:04 You know, who's really at fault here and kind of- Well-... what's really driving these vulnerabilities in the devices to, to really cause these problems? I think it's, broadly, it's a lack- Okay... 17:13 of understanding in, in the, in that startup community. So let's say you were doing a startup, you know, 8, 10 years ago. Maybe your thing was a web app or more a mobile app. 17:21 And all you had to do was get the product to market, and you could fix it. Like, at the drop of a button, you could push an app update, and it would fix any bugs. 17:27 So you could go to market with a product that just about worked, and any bugs that were found, you could fix on the hoof very quickly. The game changer with IoT is there is hardware involved. 17:37 So you're going to have to push or book production in the Far East or wherever you choose to d- design six to nine months out. And because it's hardware, it's much, much harder to update- Right... in the field. 17:49 In some cases, we've found IoT devices you simply can't update- Mm-hmm... over the air. So they're so, the s- the security hasn't been thought through. 17:57 So it's bringing a whole new d- um, dynamic into startup, that of embedded system security, and that's the big challenge. 18:04 So people come to, to market going, "It's all right, I can fix it when it's in, in the market," and then discover they can't. Yeah. So that startup mentality is, it's, it's, has to be really different with IoT. 18:14 Definitely. I mean, you go on any kind of even crowdfunding site like Indiegogo, Kickstarter, and you see smart devices all over the place. I'm su- I mean, I, I cannot imagine people that are- Yeah... 18:25 crowdfunding their, you know, products are devoting the time to the security side, not because they don't want to, but again, it's, it's not the easiest thing, like you said, lack of understanding, and it's not cheapYeah. 18:38 And it-- what, what I also find a little frustrating is that, um, if you go out for advice- Mm-hmm...on, on day one about security, so the critical things you can get totally wrong on day one are choosing the wrong chipset- Mm-hmm...to build on, the wrong architecture. 18:50 If you get that wrong, there is nothing you can do. 18:53 And we did some work on a, a smart tea kettle- Okay...quite a few years back now, and the-they, they'd chosen the wrong chipset, which meant you couldn't remote, remotely update. 19:02 It didn't have any security features you could use, like a, a trusted execution environment, s- uh, decent storage for, for keys and things. 19:09 And 'cause they got that wrong on day one, they were really backed- Right...into a corner. So it's so important. Ask those questions on day one. Get some advice. Go, "Okay, which architecture should we use?" 19:18 You know, "How, how do we do our embedded system?" And you might find for an-a-another cent or two per chip, actually, you've got all the functionality you need to make a really robust- Mm-hmm...and secure product. 19:28 But if you didn't choose that one, then you're in real trouble. Wow. How... 19:33 So once these companies launch these products, and you're saying a lot of them are very difficult to kind of fix over the air, what, what can be done, if anything? 19:43 Um, and how, how's that process usually work from your experience? Wow. So if you can't update over the air, then- Yeah...uh, you're in a really difficult situation. Do you issue a product recall? Okay. 19:55 Yeah, 'cause that, that's a, that's a hugely expensive thing to do. Um, and it was likely to take many, many small startups to the wall if they have to physically replace their product. 20:04 And we've seen IoT vendors pushed into a point where they, they've really had no choice. Yeah. Which is scary. 20:11 And in some cases, we saw some of them said, "All right, we'll just give you a huge discount on our, our, our, our version 2.0, where the security problems are fixed." Right. And that, that's kind of cool. 20:20 You know, that, that's one way of solving a problem. 20:22 But yeah, you, you really don't want to be backed into a corner where your only way to get out is to either potentially face a class action or suffer the cost of a huge product recall. 20:33 And we have seen class actions in IoT as well. There's, um, a couple settled a couple of years back actually for, um, in, in some cases, excess use of, um, customer data- Mm-hmm...beyond terms and conditions. 20:44 And we've also seen some, uh, FTC enforcements around IoT as well. 20:47 So as a consumer, like, how wary should consumers like myself be of these newer products that are coming from, let's say, smaller companies, smaller startups? You know, you wanna... 20:57 You know, you think they're cool because the marketing is great, you know, the video they make online about how cool this product can be is, um... You know, we're thinking about that stuff, right? 21:04 We're thinking about how this benefits us and how it changes our maybe lifestyle or daily routine, but we're-- sometimes even for a consumer, the security is an afterthought. 21:12 So how wary should we be of these kinds of products, and what should we be looking out for? Well, so don't get me wrong. Right. I love cool tech. 21:21 Um, one of my favorite things recently, I love skiing, and yeah, big problem when you listen to audio or taking calls is answering calls. 21:26 I got some really cool, um, smart, um, uh- Mm-hmm...speakers to fit in my, my helmet. Brilliant. Really cool idea. They had a walkie-talkie function. 21:35 But along the way, we discovered that actually anyone could track me- Jesus...and anyone could listen in on my calls. [laughs] So that, that's kind of not so good. 21:42 There's a lot of beneficial to it, but you have to be careful. So think of it... Yeah, right. [laughs] Yeah. Yeah. Which is g- was pretty, pretty awkward, really. Um, so I, I love smart tech. 21:51 I, I love, you know, cool new gadgets, but the challenge is, is distinguishing one from the other. Which, which one's the secure one and which is the vulnerable one? And actually, it's really, really difficult. 22:01 So one key I, I would recommend is if you're gonna buy a product, start looking around the vendor's website. What do they say about security? Okay. Do they say, "Oh, yeah. Hey, we do SSL and TLS"? 22:11 Or do they actually have a page where they talk about their secure development processes? Do they talk about how they fix vulnerabilities? 22:18 Do they talk about how they support their product for two, three, four, five years after they've sold it to you? Or is it really just, "Hey, it's got military-grade encryption, so it must be good"? 22:26 And that, that worries me. Yeah. That's crazy. Jenny, I know you had a question. Yeah. Um, I really wanna address how IoT is being weaponized. 22:32 And as you probably know, there's been several news reports about breaches in the past week, including the Capital One data breach. 22:38 And Armis disclosed an 11 zero-day vulnerability, um, exposing two hundred million IoT devices. 22:45 Um, so you say that we need to do this research to look into, well, which device is secure, which device is not, but these two hundred million IoT devices all had VxWorks OS. 22:56 So with this many devices being susceptible to remote takeover, um, how are IoT companies dealing with this evolving threat of cybercriminals and the potential threat to their businesses? 23:07 Okay, so I'm gonna go back three years. Mm-hmm. I'm gonna look at Mirai- Yeah...uh, the botnet, 'cause that was one of the first weaponized exploits. Now, Mirai was, was described as an IoT botnet. 23:15 It wasn't, or at least certainly the version one of, of Mirai wasn't. It's evolved. It has become one. 23:20 Back in the day, when it first started, Mirai was actually a bunch of exploits carried out against CCTV digital video recorders, 23:27 and roughly three hundred thousand of those were weaponized to create a denial-of-service attack that brought down several social networks. But the vulnerability was from one piece of code written by one vendor. 23:38 But because Mirai was, um, I guess presented as a, as an IoT botnet, everyone missed the opportunity to see the single point- Mm-hmm...of failure. 23:47 And there are single points of failure, and there are ways to mitigate as well. A great example, looking at the recent VxWorks bugs. 23:53 Well, hang on, as long as we don't put these IoT devices on the internet or use decent access controls in front of them, then actually we're in a much less serious position. 24:02 These devices aren't out there on the public internet. However, there is an area that really bothers me right now, and what have we started seeing is security flaws in IoT actually start to accelerate. 24:13 And the reason for this is, is actually partly a result of security. 24:17 We're starting to see, um, vendors coming to market and realizing they don't have the skills to set up their own, um- Mm-hmm...cloud service platforms, so they outsource to, to, um, a vendor that provides IoT cloud platforms. 24:29 We've found vulnerabilities in the platforms. 24:31 So instead of being a vulnerability in, for example, my smart hot tub- [laughs]...which is one we looked at at Christmas this year, actually the hot tub led us onto an IoT platform, which led us into some smart med tech-On the internet and vulnerable. 24:46 So this, this problem, this interconnectedness is actually starting to get worse. 24:49 So with all these big, like, big situations that are getting in the media, and then obviously conversations like we're having now where we don't... You know, they're smaller situations, smaller devices. 24:58 You guys are encountering these, you know, everyday consumer products that are, um, at risk and beco- causing problems. 25:04 What do you think is really going to-- or what do you think it's going to take to really start driving change across the industry? And that goes all the way up to government regulations. Yeah. 25:16 So I thought when I started out, what, five years ago in IoT security research, I thought that market- Right... forces would drive security. I thought that, that vendors would see some of the train wrecks- Mm-hmm... 25:26 in, in the, in the media and go, "Hey, we don't want that to happen to ourselves." Um, you know, "Let's actually ask the question about security, you know, on day one or, or day seven of, of startup." 25:35 Uh, but actually it's not. It, it, it's actually got worse, and I d- I no longer believe that market, market forces will- Okay... affect the change that's required. Uh, I think, I genuinely believe we need regulation. 25:46 And I'm not a huge fan of regulation. Don't get me wrong. Right. You know, I like free market and market forces, um, driving change. 25:52 But in this case with IoT, I think the only way we're gonna do it, particularly as we see more and more startups- Mm-hmm... 25:58 in the Far East, um, around the world as everyone's bringing products to market, we've got to have regulation. We've got to protect our consumers from vendors who- Right... 26:05 just don't listen and don't understand security. And that's, that's why I think California went ahead and regulated. I think they had to. 26:13 First of Jan 2020, we've got protections for California residents, which means, you know, by the size of the California market, I think- Mm-hmm... 26:20 by default, we're going to see many vendors starting to follow that across, across the world. I believe that Massachusetts has followed suit with very similar regulation. Interesting. 26:28 So we should all move to California is essentially what the- Yeah, deal with those crackers. [laughs] Yeah. [laughs] I've had them. I lived in San Francisco. [laughs] So yeah. 26:37 But what do you think it's gonna take to kind of get the adoption to grow from, you know, state to state? Do you think it's just gonna be just seeing more of these situations come up? 26:45 Is it going to be state representatives talking to each other and seeing the benefits that it's having within their borders? 26:50 Or, or, you know, how do you think it's gonna kind of bleed into these other states, you know, outside of California, Massachusetts? Well, I think the, the joy we have- Yeah... 27:00 California being such a big market, would a vendor build a product that complied with California- Totally... and another product that didn't? You wouldn't. 27:07 So you're actually, you're actually leading the market with-- by California- Yep... stepping up. Um, there's also efforts going on in the UK. We have, um, we have some regulation that's being consulted on right now. 27:17 So the UK is- Okay... kind of leading a charge there too. And around the EU, there's also some, um, there's some progress being made. So it-it-it's time for vendors to wake up because they're gonna start seeing- Yeah... 27:27 their market start to shrink if they don't comply with decent security advice. They're going to see territories they just can't sell in anymore. So why would you do that? 27:36 Why, why would you not go, "Hey, you know, now's the time. Let's get our- Right... security baked in today so that January first next year is not gonna be a disaster for us on the, the retailer shelves in California." 27:46 And, and what kind of, um, where-- Can you give us a little more insight into the regulations? Like kind of what, what does the device need to go through in order to be able to be sold in these states? 27:54 Is it, like, is it a kind of, um, certification process, or what exactly are they looking for? No. 28:03 So that's, that's probably the only place where the California regulation is, is a little, um, vague at the moment, is they just talk about reasonable security controls, and obviously there's going to need to be some definitions there and some case law I'm sure as well. 28:15 Yeah, reasonable can be interpreted- Abs-... any way you want. [laughs] Especially when you're buying hardware- Yeah. Totally... from overseas. Right. 28:19 And actually, in fairness, NIST are also doing some work about to define what reasonable is, which I think is a really good idea. Okay. 28:25 Um, in the UK, we've actually got some very high-level, um, points which actually should, should start pushing, um, manufacturers in the right direction. 28:33 In the EU, the draft regulation or draft, um, guidelines they put out are actually quite complicated and actually really quite specific and to my mind, probably a little bit too much at this point. 28:42 You know, I think it, it's a big step. Okay. Do you think that'll prevent companies- I think-... from taking that step then? Because- It's too much. 28:48 I, I-- My personal view is I think right now some of the, um, some of the guidance in the EU is probably a little bit too much. Um, I think we're starting from a- Okay... 28:56 very low base right now, and I think we need to encourage these vendors of IoT products, you know, step by step. You know, let's do the basics. Let's, let's start getting rid of default passwords. 29:05 Let's make sure that products are supported. Let's make sure that the vendor commits they're gonna support their product for a period of time. And you know, then I think the next step- Mm-hmm... 29:13 and this is also something EU's working on is, how's about some labeling? So in-- across the EU we have, you know- Sure... con-- um, power consumption for electrical appliances at home. Really easy, uh, A to F, um, 29:26 uh, diagram so we know how much power my refrigerator consumes. I want to see that in- Right... IoT. So this product's got uber security or it's not so good, or actually it's a train wreck. Mm-hmm. 29:37 And then I think, you know- Mm-hmm... you and me, we can make those decisions- That's it... at the, uh, in the retailer going, "Do you know what? That was cheap. I don't care about my data. 29:43 I'll buy the rubbish one," or, "I'm gonna spend my money and buy something that looks after my data and keeps me safe." 29:51 To a, to a point earlier about why, you know, somebody would build a device and not, you know, becau-- and kind of miss out on California and Massachusetts, I wonder if early stage, uh, device manufacturers will kind of skip around the security to bring their product to market faster, get buy-in, and, um, establish somewhat of a user base outside of the more regulated states to g-get further funding to, you know, either go after security or grow, you know, build the device out to maybe a later phase. 30:21 So I wonder if that's a potential path for a lot of these people to kind of not worry about security until later on. Yeah. 30:26 Do you know, I-I-I'm gonna come back to this point, which is if, if you do security on day one, it's, it's not a huge cost. If you, if you bake- Okay... 30:34 um, things like the OWASP, um, [lips smack] web and mobile app security design, um, processes into your procurement contracts- Mm-hmm... 30:41 with the people who are gonna write your mobile app and your web app, so contractually they're bound to produce you a secure application. Uh, that, that, that'll save you some money. Okay. 30:49 Um, if you-- when you go to your chipset suppliers and you say, "Hey guys, right, you're gonna make sure you comply with X, Y, and Z," right? Some basic things. And they'll go, "Oh, okay. All right. 30:58 You need this chip."Yeah, it's a few cents more. Is that okay? It's like, yeah, 'cause we can build security on that. Great. Great. 31:06 So I guess as we're trying to-- we're wrapping up here, um, what can consumers be doing to protect themselves, you know, when buying devices? 31:14 You know, how can we be sure devices are either s- more on the safe side or, you know, gone, you know, in, in that regard? Like what, what should they be looking out for? 31:22 Like we've had a couple, uh, experts on here talk about, you know, es- like if, for instance, you're going to buy cameras, don't buy any cameras from China. You know, just... 31:29 Or if it's a white labeled camera, make sure you try to figure out where it came from. 31:32 But outside of that, kind of as a whole, are there any kind of just high level, catch-all type pieces of advice you would give consumers when it comes to buying smart devices? 31:41 Because like, like you, myself, I'm very into buying gadgets and technology where we're not necessarily thinking about security aspect until later. 31:50 So what would you kinda say to people who are similar to you and I, who are looking to buy these kinds of things, and what we should be, you know, paying attention to? Wow. Okay. So that, so cameras in China. 32:01 Um, uh, it's very difficult, um, on that because actually, essentially all cameras are made in China. So e-even the ones that, you know, have got Western brands, they've just been rebadged. 32:10 If, if you take them apart, they've still got the same- Yeah. Right... PCVs inside that were made in the Far East. Yeah, with a very small number of exceptions. Mm-hmm. 32:17 Um, so it's, it's really, really difficult to advise consumers what to do. What I, I, I... Some simple advice, though. Number one, do you need it? Mm-hmm. And I know for me, a fan of tech, I, I need it. I really do. 32:28 But do you actually need it? And then think about what it does. What does it collect? Is it a camera? Has it got a microphone? Has it got a speaker? Does it collect your personal data? Where does it send it? 32:37 So y-y- I would be more concerned about devices that collect more of my data. I want to know more about the security of them. Okay. Is it a kid's baby monitor? 32:45 A baby monitor, you're gonna put it in your, in your child's nursery? Wow. Okay. That, that, that's quite brave. Are you, are you sure you want to do that? But there are a few points that you sh- also look at as well. 32:55 Keep your products up to date. Check on your mobile app for updates. Check for firmware updates in the product so you can push them. They often fix security bugs. I think that's really, really important. 33:05 But do, do decide, you know, do you actually need this smart tech? Right. That's important. Really, really important. There are a couple of really simple checks you can do in things like Wi-Fi and Bluetooth. 33:15 So if there's no, um... If you don't have to, um... Try connecting someone else's phone to your Bluetooth device and see what happens. Mm-hmm. Yeah. Go maybe get another phone and try it. 33:27 Try connecting to the Wi-Fi device. Has it got an easy password, one, two, three, four, five, six, seven, eight- Right... or something similar? Can you change it? No? Okay. 33:34 But there is one thing I think consumers must be responsible for, and this is gonna be boring, is it is your responsibility to set a strong password. Okay. I know it's boring, but it is really important. 33:45 Use a password manager to generate you a complicated password. Mm-hmm. Store that password safely, and make sure your IoT products and your mobile apps that talk to them have got strong passwords on the user accounts. 33:55 Otherwise, you don't need a hacker- Yeah... who understands IoT. Anyone can hack you. Yeah, that's what's interesting because I, like for my parents, for instance, they're, you know... 34:04 And it's funny 'cause I was watching something the other day about... 34:07 Oh, I was watching a show last night on TV, and the son was talking to his parents who were trying to understand how to enable FaceTime on their computer, so they had to put in their Apple ID and Apple device, and they keep their passwords written down on a piece of paper, like in a book, which my parents- My dad does that. 34:22 [laughs] Yeah. My parents too. 34:23 So like, you know, those extra steps, though, of using a password manager and, and getting, you know, say, non-technical people to understand how to do that and, you know, create a password that's not 34:35 something they can easily remember is kind of, is, might be s- scary to them, or maybe kind of, um, they're, they're worried they, you know, that they'll lose their, their information or they'll forget their password even though there's usually an option to recover it. 34:47 So it's an interesting kind of piece of advice because people like you and I, I think, and Shannon understand the benefit of doing that, but a lot of people feel like it's an e- extra step, and it's just not, you know, it, maybe it's not worth it to them or they don't understand why. 35:01 Yeah. Although people are often, um, I think, I think in security we say, "Hey, you know, writing your passwords down in the book, that's really bad." Sure. 35:08 Do you know, actually, compared to reusing the same password everywhere- Yeah, definitely... write it down in a book. Yeah. It's fine. Why not? And, and put it in a, in a safe at home. Yeah. 35:15 It, it's better than using the same password everywhere. Definitely. I'm a victim of that, so I use my, the same password. [laughs] Well, before I- [laughs] Now I'm gonna go change everything. 35:25 Yeah, it's the most frustrating thing when I try to use the same password. I know. It's like, you can't use a password similar to the password you used six months ago. I'm like, shit. 35:31 [laughs] I'm not gonna remember any of these passwords. But no, we use, uh, I know, I know at least you- That's why we use a password manager. Yeah, we use password managers. 35:36 Uh, but the last question I have before we kinda get into the just kinda having you, um, give the audience ways to connect or, or follow along with anything that you're finding is do you think... 35:47 I'm trying to put this in a way that's not scary to people, but should we be scared of IoT? Like should we be scared of connected devices? 35:55 Or, or how would you kind of answer that statement without terrifying too many people? So the bit that concerns me is you're putting your trust in someone that you don't know or understand. Sure. 36:11 It's very difficult to work out from an IoT device or the mobile app it uses where your data goes. Okay. And without some tech skills, you're gonna struggle to do that. 36:18 And in many cases, that data disappears off to the Far East, just maybe bouncing through a couple of servers, then coming back to a, a Western vendor. Who knows? 36:26 But that bothers me a great deal, and I think as, as consumers, we're sleepwalking into, frankly, a surveillance society that's not of our own making. Mm-hmm. 36:35 We're, we're buying something cool, trusting an IoT vendor that they've built it safely. Maybe, maybe it's just accidental, but data's disappearing into other territories without perhaps the controls that we'd expect 36:51 in our own countries. Sure. So I think we are inadvertently walking into a surveillance society, and we're inadvertently building-Innocuous cyber weapons that could be used against us in time. Interesting. 37:04 So I think there needs to be just some- Yeah... care because we could so easily give foreign powers or disaffected individuals the ability to knock out- Right... power grids, to steal our data, to- Mm-hmm... 37:16 create fraud- Right... to creep on us. [chuckles] And I just think, you know, we're at a, a bit of a tipping point right now. Regulation's on its way. Hopefully, the future's- Mm-hmm... looking brighter, but we're in a, 37:27 an unpleasant- Yeah... place at the moment, from my point of view as an IoT security researcher. Yeah. I mean, you're, you're right in the middle of it. I mean, you're, you're touching everything out there. 37:35 So if it, you know, that's what you're thinking, then we all need to be very aware and attentive to what we're doing. So, so we really appreciate your time, and this episode has been, I think, great in my opinion. Yeah. 37:43 Loved it. Yeah, I, I've loved it. Um, if people wanna kinda follow along and learn, you know, maybe what, what you're working on, anything that you guys find, you know, how do you kinda get information out to the public? 37:52 What would be the w-best way to do that? Yeah. So we, we blog- Okay... most days with IoT security research. The, the website address is pentestpartners.com. Okay. So pentestpartners. 38:03 And it's full of crazy stuff on there, lots of advice, so you know what you go about when you're looking for a secure- Right... chipset and how to write your mobile app safely. 38:10 Um, but also a, a train wreck of things that we've- Okay... broken, that stuff that just wasn't looking after your data- Sure... and privacy safely. And they have great titles, too, so- Okay. 38:19 Well, maybe we'll ask you-... it'll entice you. [laughs] Yeah. Yeah. Well, uh, if you guys are ever looking for additional visibility, we can, we can cross-post some of that onto IoT For All. I think that'll be... 38:27 Our readers would love it, so. Yeah, definitely. So, this has been great. Thank you so much again for your time. 38:31 We'd love to have you back on, you know, maybe in a couple months as, you know, new things come up and new exciting stories develop on your end, um, to share with our audience. Great. Thank you. It's been great. 38:41 Thanks again for listening to this episode of the IoT For All podcast with, uh, our guest, Ken Munro of Pen Test Partners. Hope you guys found it very valuable. 38:49 Um, I know Shannon and I really enjoyed it, not just because of the content, but we thought Ken was just an awesome person to talk to. 38:55 Um, it's always fun to learn about different devices and the job of basically finding these devices, bringing them into the office, and trying to break them, um, and how that whole process actually works. 39:04 But what I've been doing lately in a lot of these episodes is kind of taking some big, um, big points that I thought were important, um, that we thought were important here, and sharing them with you afterward just in case there were things maybe you skipped over or you didn't catch and, um, hopefully, um, provide a little bit more value to you. 39:19 So one of the things I thought was really interesting, and we've talked about this with other security experts, is the fact that security in IoT is often an afterthought. 39:26 Um, security, um, really becomes, and Ken talks about this, really becomes expensive if you try to bake it in too late. The big issue is that security gets thought about too late in the development process of the device. 39:39 But if you were actually to think about it and really start incorporating it early on, maybe from day one, it would almost have zero to very little impact on the cost of your product, and I think that was something interesting, someone who myself is not a hardware guy, um, learning about kind of h- where cost really comes in, uh, with security. 39:57 So thinking about it later, trying to bring it into the process, into your device later, actually costs more money than just thinking about it and handling it up front on, for many reasons. 40:05 Um, another thing that kind of ties into this is along the, 40:10 I guess, development life cycle of a product is ch- how important choosing the right chipset to build on is, um, and, and getting that system architecture correct from day one. 40:20 'Cause if you choose the wrong chipset or you choose, you know, the, to set up the system architecture the incorrect way, um, there's really nothing you can do once that device is out there. 40:30 So, you know, really making sure that if you are a manufacturer of devices or you're looking to build a smart device, to really ask those questions ahead of time, get the advice on how to set up your embedded system. 40:40 Maybe it, it costs one to two cents more per chip to give you the functionality and the security you need, um, but it's probably worth it in the end. 40:49 You know, with- without this correct chipset and correct systems architecture, you may be in a lot of trouble later on down the line if any security breaches are found. Um, thirdly, 40:59 um, w- we, when Ken starts talking more on in the consumer space and more so at consumers about what we can do to, um, buy devices that are more secure and o- on our own and do the right, the right due diligence, some of the things he mentioned I thought were important to note. 41:15 So looking at vendors' websites, you know, what do they say about security, if anything? Do they just say something like it has merchant-grade encryption? 41:22 Um, you know, something just kind of buzzwordy that we might not really understand but sounds secure? 41:27 Or do they have dedicated pages to their security testing process, their update process on their firmware or, you know, how they handle security? 41:36 What does their support process look like to provide updates and upgrades throughout, you know, the next few years if you own the device for an extended period of time? 41:44 All those things are think- something important to, to look into before you are, um, maybe to buy a device if you're really focused on, and you should be, on security. 41:53 And lastly, I, um, I think something that's really important is when Ken starts talking about just the general high-level advice, um, he has when it comes to buying a smart or, or connected device. 42:04 So the first t- point he mentions is, do we really need it? As much as a device looks fun and interesting, you know, how, how important is this device in, in our daily life? 42:12 Is it something that we really need, um, or really want? Um, just because that kind of starts putting things into perspective there. The second thing is what type of data does the device collect? 42:21 Does it, does it, does it have a camera? Is it taking video? Does it have a microphone? Uh, does it have a speaker? You know, what personal data is it requesting from you? 42:28 And then thinking about where that data is going to possibly be sent and who has control over that data. You know, do you trust them? 42:33 Um, have you looked into figuring out what kind of security they have on that data to protect it so it doesn't get put into the wrong hands? 42:39 Um, and then one thing that's kind of often overlooked is who is the intended end user? You know, is it something that's going in your children's room, in a nursery perhaps? 42:49 Um, and how much, like, h-how important is it that the data that's being collected from that end user is protected for you? 42:58 So if it's something that's going in a children's room, I mean, does it really, do you really want eyes on that or somebody being able to kind of creep into that room, um, remotely, like he talks about with, with the doll that they tested, broke, and then found these security flaws in. 43:11 And the last two points are 43:13 looking in as a consumer into our connected devices and making sure that we're updating them regularly, meaning that we're looking into updates from the company and the manufacturer, and we're actually hitting that button to upgrade the firmware, the software, whatever it may be. 43:25 And then finally, making sure, and he said this is boring to say, but it's super important, is making sure we're setting strong passwords on our connected devices. Something that we think we do. 43:33 Maybe we think our device is secure because we have a super strong password, but, um, making sure we're really doing the due diligence to do that and having different passwords for different devices is very important. 43:43 So to password managers and so forth are something to definitely consider using if you do not do it already. Um, but yeah, those are some big takeaways I thought were really important. 43:52 Hope you guys found those to be valuable and a variety of other stuff we talked about to be valuable today. If you did, please be sure to subscribe to the podcast. 44:00 We love seeing more and more of you, um, subscribing week to week, listening to our episodes, downloading them, and giving us feedback. 44:06 Um, if, um, you do enjoy the episodes, please leave a rating or review on whichever platform you're listening to us on. It not only helps others find it, but it lets us know that you're a fan, which we find really cool. 44:18 Um, but other than that, we really enjoyed, um, this episode. Hope you guys did too, and we look forward to, um, having you listen to our future episodes