One pillar in our roadmap for federal AI legislation outlines a policy framework for protecting against catastrophic cyber and national security risks. A core part of that work is building a better understanding across government of what changes as we move toward an AI-vs-AI cybersecurity landscape where both attackers and defenders can operate at machine speed.
In this conversation, Anne Neuberger and Sam Jones join Jai Ramaswamy to go deeper on what this shift looks like in practice. Neuberger draws on nearly two decades in government—including serving as deputy national security advisor for cyber and emerging technology—to explain how AI is transforming the threat landscape, most notably making attacks faster, cheaper, and continuous at scale. Jones brings the builder’s perspective as CEO and cofounder of Method Security, where his team is building autonomous cyber systems for both offense and defense observing firsthand how AI is accelerating everything from routine tactics to exploit development.
Together, they discuss what “cyber resilience” means in an AI world: continuous testing and red-teaming that was previously cost-prohibitive, clearer benchmarks for critical infrastructure, and faster recovery when disruptions happen. They also walk through the policy measures that can help defenders keep pace, including enabling high-fidelity information sharing and modernizing procurement so the U.S. can deploy the defensive capabilities needed for a new cyber era.
Topics covered:
02:01: How AI changes the threat landscape
06:23: Net new risks in an AI-vs-AI cyber world
10:06: Building trust to deploy new technology in no-fail systems
12:40: Cybercrime at machine speed
16:03: Who benefits more from AI: attackers or defenders?
18:14: Tactics to remove friction for defenders
21:11: Real examples of incidents where AI could have changed outcomes: Colonial Pipeline and Change Healthcare
26:03: What cyber resilience means in an AI world
29:42: Measuring resilience
35:44: Information sharing and antitrust: lessons from financial services and telecom compromises
43:44: The builder’s view: what Method Security is building for offense + defense
48:12: Little Tech realities of building with a small team and selling into government
51:38: The role of procurement in ensuring defensive systems keep pace with adversaries
54:17: What’s next: in-year buying flexibility and closing thoughts
This transcript has been edited lightly for readability.
Anne (00:00)
AI fundamentally gives attackers the ability to jiggle every virtual doorknob continuously.
Sam (00:06)
The institutions and networks and enterprises that underpin our way of life need to be tested continuously and not just in some lightweight scan, like seriously acting as an aggressor and seeing if there are holes throughout the enterprise.
Anne (00:21)
The threat landscape is fundamentally changed is also why the cyber defense landscape will potentially fundamentally change for the better too.
Jai (00:35)
Welcome to the a16z AI Policy Brief. Today we have Anne Neuberger and Sam Jones. Anne and Sam, would you mind just introducing yourself to the audience?
Anne (00:44)
Absolutely, sounds like a first grade literature book, Anne and Sam.
I’m Anne Neuberger, I’m a senior advisor at Andreessen Horowitz, and a distinguished fellow at Stanford. I was previously deputy national security advisor for cyber and emerging technology in the last administration. It’s great to be here.
Jai (01:01)
Welcome.
Sam (01:02)
Sam Jones, I’m the CEO and co-founder of Method Security and have been working in defense tech my entire career including cybersecurity and been doing AI since it’s become a meme and such a hot topic and excited to get into it today.
Jai (01:15)
Well, really appreciate having both of you on the podcast today. Now, Anne, Sam, both of you have written, I think extensively, on the issues surrounding AI and cybersecurity. Very hot topic, clearly one of the risks that I think people are most scared about. And I think the real question is whether this technology really changes the threat landscape, and if so, how, and if not, why not? But I’m going to leave it pretty open-ended just so we can get the conversation started. But Anne, given your extensive experience, I’d love to hear your thoughts on that kind of a landscape.
Anne (02:01)
Absolutely. AI fundamentally transforms the threat landscape because typically an attacker to achieve their objective, whether it’s stealing information or whether it’s disrupting a system, needs to do some degree of study of the network, of the administrator who has rights on the network, where are their vulnerabilities on the network, and then what’s the right way once you get a foothold to navigate through and achieve the attacker’s objective.
AI fundamentally gives attackers the ability to jiggle every virtual door knob continuously. So even if a new patch comes out and the company is bit late or they deploy code that has new vulnerabilities in it, it can be found so much quicker by an attacker who can just continuously be looking for either misconfigurations in the network, coding errors that allows that can be exploited, and to do so continuously at scale across a target set, so it fundamentally transforms that. I will say that what’s so interesting about cyber is that the first steps of either exploiting or defending a network look exactly the same. It’s finding the mistakes, the misconfigurations and vulnerabilities. What you then do with it is the key. So I think that, what I just talked about, why the threat landscape has fundamentally changed is also why the cyber defense landscape will potentially, if we can get our act together quickly, fundamentally change for the better too.
Jai (03:24)
Sam, your thoughts?
Sam (03:26)
From my perspective, just because I’m constantly building with this stuff and Anne aptly pointed out, regardless of your objective is to attack or to defend, those first steps are the same. And that’s what we build systems to do, whether it is to defend or to attack. My team and I are building with this stuff constantly. And we’ve seen a huge evolution over the last two and a half years, which is the lifespan of Method Security.
Where last year I would characterize how the threat landscape was changing is that purely known tactics, techniques and procedures were accelerating and everything was just getting faster, but it’s all known things that were happening. And that still poses a great threat to old defensive systems that can hardly prevent basic attacks and basic threats. And we were seeing a lot of criminal groups that maybe were novice in their skill level just be upgraded. And that poses significant challenges for enterprises that have bureaucracy in place to adopt and deploy defensive techniques because they’re just purely outmatched at that point.
But this year, what we’re seeing is pretty different. AI is now being quite effective at exploit development. And so it’s actually helping generate new tactics, techniques, and procedures to complement that existing acceleration. It’s getting doubly bad in the threat landscape, all the more reason that achieving resilience and defensive posture is an urgent problem.
Jai (04:53)
So I think what I’m hearing from both of you is that the way AI changes the threat landscape is – I was an old cyber crime prosecutor and the days of Matthew Broderick, load hackers are long gone. These are organized, transnational criminal organizations, nation state actors and a combination of the two.
But what I’m hearing is that one of the differences is this now allows a less sophisticated class of actors to get involved because the level of sophistication you need comes down because AI is commoditized intelligence and therefore a lot more people can do it. And then speed, scale, all those other things really are different.
One of the things that it seems to me that is in a sense good about this technology is that the capability aspect of AI is not asymmetric. There are some technologies that will benefit attackers more than defenders. But with this technology, I think what I’m hearing is that there’s a sort of symmetry that both attackers and defenders can use this. And so talk to me a little bit about that. And based on that, what are the real net new risks? Because I think it’s always helped to think not just, what are the risks? What’s new about this technology? What do we need to worry about and then deal with from a policy perspective?
Anne (06:23)
So I want to answer the two really good points and the two questions are fundamentally linked because AI does make attack far easier, much more continuous, and also continuous in the types of attacks that are possible and evolving them. And as such, defenders who aren’t using AI in defense are in a much worse position because a human against a machine in this game just doesn’t win. Humans and machines fundamentally can. But there are certain parts of cybersecurity that speed now matters much more, and they have to be automated.
Think about this. We’ve long put in place, particularly post the rise of insider threats, monitoring to say what’s a pattern of behavior, particularly for a particular role, where a person works, what their typical work pattern is, when they come to work, what are the kinds of data they are authorized to look at if you’re working on Asia bond markets. It’s something odd if you’re looking at South American, something, South American equities or whatever it is. And as such, but in the past what we would do is to avoid false positives and potentially disrupting somebody’s work, that would alert and send it to a network administrator who would investigate and say, was this legitimate? Somebody’s just working on something, a new project, or was it something malicious? Now, there’s a need to automatically turn off the account and investigate afterwards because you can’t risk that speed of what’s possible in attack. So that is the net risk. Very new kinds of adaptable attacks and that if defenders don’t deploy AI-based defense with speed and determine, frankly, the risk equation differently, we’ve often been worried about legitimately false positives. I think now, because of the speed issue, we’ve got to really turn the dial on automating as much of the fence as possible and then ensuring that we can do investigations rapidly quickly to turn things back on if needed.
Jai (08:28)
I mean, that puts a premium on the types of solutions that are out there, it seems to me. And Sam, you’re building some of these solutions, but that can be really disruptive. I mean, if I’m somebody who needs this information, particularly in the financial markets or something where actually it’s not just for the attackers, but for the business, real time is real cost. How do you think about developing solutions in that kind of environment?
Sam (08:54)
I work back from the buyer psychology, if you will, whether that’s an enterprise like a Fortune 500 enterprise, CISO, or a military DCO or OCO commander, both of all those groups have to accept serious risk when employing new technologies, because their domain is no fail. You know, they’re not making sloppy internal development apps that could be fun. They’re not making images or art. If they’re work goes down like they get fired for a specific reason…
Jai (09:26)
Or there’s an actual threat to the infrastructure or to safety that’s at risk.
Sam (09:30)
Which then they get fired anyway, so they rightfully are skeptical of employing too much new technology too quickly. But at the same time they also know they need to adapt to win and so what we have been trying to do is come from the stance of AI is almost like a raw intelligence material is inherently Not trustworthy. And so how do you build systems around that raw ingredient that no one should off-the-shelf trust such that it can be employed in production?
Jai (10:00)
Can you dig a little bit more deeply into what it means to be not trustworthy? What do mean by that in this realm?
Sam (10:06)
For a no-fail security system, whether that’s an inline security control that’s going to make a decision of classifying this as good or bad behavior and shutting down an account or something like that, or deciding to delete data, whatever it might be, leaving things up to too much nondeterminism is really scary for folks that this is their job and there’s no tolerance for failure.
AI, especially this class of large language models, is so non-deterministic and it’s evolving so quickly that you’re not going to take chat-GPT off the shelf and decide to put it into a critical security control. There’s just no way. And so there has to be guard rails like before and after and all around anything at the AI core of some kind of security product. That’s what we’ve tried to do very thoughtfully, both for defense and offense. And it’s going over well, but it’s a lot more to build than just making a really cool prototype that is on the internet that looks really cool. We’ve been trying to architect for what is actually viable from someone who will get fired if this doesn’t work.
Jai (11:14)
It’s interesting you raised that point because oftentimes when we’re talking about AI, the benchmark seems to be perfection. But I want to push you on that a little bit because human beings make mistakes all the time. But are you saying that the psychology of organizations is such that when the human being makes the mistake, you’re going to be forgiven for it because they see, we’re giving you a certain amount of tolerance because you’re a human being. But if you put money on the line or you start relying on technology, my standard is different than it is when judging a human being.
Sam (11:48)
A little bit and we’re probably going to see some similar psychology with Waymo’s and Tesla’s going out to the road, which are definitively safer, but you know, we’re accepting a different type of robot risk. I think what’s different about cyber is that these technologies can be deployed so prolifically across the enterprise that they can make so many decisions that compound even lower error rates, like to a higher degree that it’s almost like you’re not in control.
Jai (12:12)
So there is a new risk in that. The compounding effect is a net new risk given the speed.
Sam (12:18)
Exactly. Ultimately, what security teams want is to be in control. And AI off the shelf does not allow them to do that. And so there’s a lot of systems around it. And that’s kind of what we focus on.
Jai (12:28)
And you’ve both talked about speed. What other things, and now we have this sort of compounding risk, any other net new risks that you can think of that this technology introduces?
Anne (12:40)
You know, I’m struck by your comment of being a former cybercrime prosecutor.
I’d love to ask you back that question, but first reflecting on it, when we think about cyberspace, there’s really three sets of actors. You mentioned it at the beginning. There are pure criminals out for financial gain, the rise of ransomware in an entire, pretty sophisticated ransomware ecosystem in the last few years. The first set of groups who gets access and sells that access, the second who builds exploits, the third that does negotiations, the fourth that does the process of the payments and the decryption. We’re talking billions of dollars in ransom revenue that really drove this ecosystem. And the challenges of pursuing them was first, you have some countries that offer safe haven to the actors themselves, other countries that offer safe haven to the virtual asset service providers who don’t follow anti-money laundering rules. So actually the prosecution of the much more complex cybercrime ecosystem became far harder. The reason you saw groups of countries working across, and transnational rules don’t really exist, so they’d essentially be cobbling together an international system to deal with a criminal problem that crosses borders. Obviously, there’s also the set of threats, which are more countries seeking to disrupt at a convenient period of time. And then there’s the loose gray space of countries who usefully use hacktivists or actors to achieve their goals. And I think the speed on the action, while it can help in using AI to draw across, let’s say, here’s the malicious infrastructure that was used for an attack, it has become far harder to find the evidence because attackers can also clean up that infrastructure far more quickly as well.
So I think the attack infrastructure has become easier to set up, tear down, and our laws and policies in actually pulling the thread to investigate it and bring folks to justice and begin to deter it remain pretty much the same. So that speed is now playing against the ability to deter and counter these actors.
Jai (14:48)
I think that’s right. In the cybercrime space, one thing that really changed the game was the Budapest Convention. Sharing of information is hard in a prosecution. Cybercrime happens in real time. Legal processes take months, sometimes years, to put in a mutual legal assistance request. And so I think there probably does need to be some additional kind of infrastructure built to deal with the even now increasing speed.
But in some senses, my thought is that and I’d love you to validate this or not, as we move to what you described, Anne, as an AI versus AI world, I’m gonna date myself, but I think of Mad Magazine, Spy versus Spy, when I think of AI versus AI, but where AIs are essentially fighting against each other, for measure or countermeasure, offense, defense, how does AI tilt the playing field one way or another? Does offense have an advantage? Defense have an advantage? Would love to have your thoughts on who benefits more from this since both will benefit.
Anne (16:03)
Yeah, and I’m looking forward to hearing Sam’s thoughts too. Defense is fundamentally harder. And that’s why I think defense benefits more, in short. An attacker has to find one way in and especially on a large network, especially on a hybrid network, some of it on premise, some of it in the cloud, there’s likely to be a misconfiguration, a system that just was down when the patch went out or code that got pushed and somehow there was vulnerable code there. So it’s far harder for defense to always ensure that that entire space is safe.
On the other hand, so much of cyber defense today is still manual, and is still bringing together lots of different data to find what’s anomalous. And once you figure out what’s anomalous, you can figure out what’s malicious and then put in place the actual algorithms to then address that in the future. So I actually think given defense is harder and given defense today is still far more manual, AI will make a bigger difference, and leveling up the two has both become far more capable in each space via AI.
Jai (17:07)
And so I think what I’m hearing from you is that there’s more low-hanging fruit in the defensive sector. We haven’t, in a sense, maximized our efficiency on the defense side because it’s so human intensive, maybe for organizational institutional reasons, maybe just because it’s harder from a policy perspective to adopt defensive measures. I’d love to think through that a little bit more.
Do either of you have thoughts on how we get policy out of the way so that it doesn’t become an impediment for companies, for the defenders to adopt these technologies? Because we know that the criminals and the nation states have no restrictions on whether they can adopt this stuff. And so they’re adopting it in real time, whereas there are sometimes legal, policy, other reasons, maybe reputational reasons why the defenders don’t want to adopt this policy. What are the policy levers we have to encourage just to get them out of the way so that defenders can also adopt these things in real time?
Anne (18:14)
You know, I’ll start with a quick thought that actually builds on something Sam said earlier. This conversation, which is to say, AI is going to make attacking and attacks far more capable. So defenders have got to move out with AI. If CEOs say that, to their security teams, to Sam’s earlier point, that enables them to say, the key is you’ve got to move out fast. There’s risk of doing, there’s risk of not doing. The risk of not doing exceeds the risk of doing here. So move out and I’ll have your back. It’s some of the early rollouts we run into issues because any new technology, as you’re deploying it, the tech is immature, something can go wrong. And I think that what I’m struck by is sometimes defenders rightfully are worried that in early adoption, there’ll be issues and they’ll be caught up with that. So I think that push to say, we have no choice, we have to deploy and ways for companies doing those early deployments to compare notes. What have we learned about TTPs? What have we learned about bringing together different kinds of data? Which products are exposing the data needed so that you can actually use AI to bring that data together and run a next level of defense on that? What are the most effective code vulnerability scanners that actually generate patches that you can trust and deploy those quickly too? So you can close that cycle.
So I think a big part of it first is the executive leadership recognizing the dynamic we’re talking about and giving top cover for rapid deployment. And then second, the ability for companies to come together and actually share what they’re learning with a level of fidelity.
Finally, on a policy side, there are some areas where regulatory barriers exist in enabling that kind of data sharing. HIPAA is a good example. The rules around health information sharing, GDPR, EU rules around data sharing that prevent that, particularly cross-border. And that’s going to be key to getting the tech to work in the way that defenders need.
Jai (20:13)
Yeah, on that regulatory issue, one of the things that I think has been kind of disappointing to see is in the financial space specifically, you see that fraud detection way outstrips and the use of AI in that world way outstrips anti-money laundering. And a large part of it is the regulatory barriers. There are a lot fewer regulatory barriers to adopting AI in fraud. Whereas in AML, there’s model validation and other concerns that arise. And it can take months to deploy new AI. Those are the kinds of barriers I think about. But can you think of, in the incidents that you were exposed to given your kind of senior government experience, are there any that you can talk about where AI and the technologies you’re seeing would have made a meaningful difference in the defensive side?
Anne (21:11)
Yes, very much so. I’ll give two examples, one that is responsive by the way to your AML point.
You know, I recall after the Colonial Pipeline attack of May of 2021, that really opened a lot of people’s eyes to the fact that cyber attacks could be not just stealing data, but also fundamentally disrupting everyday lives, right? Cars queued up at the gas station, people couldn’t get gas. And frankly, our one pipeline that runs down the Eastern seaboard shut down for almost a week. That’s a major impact.
And at the time we were trying to figure out what was the scale of the issue because many companies that were affected would pay a ransom quietly and we didn’t have visibility. Things changed after that, but I recall a conversation with the CEO of a large bank. And he said to me, Anne, we know the ransom payments. And I said, really, how do you know it? He said, well, suddenly you have a company that never touched the crypto market before. There’s a cyber attack that’s public and they come in and they’re buying crypto and like, you know, immediately moving it to that point, right? It was just interesting.
But similarly, you know, when I think about, to your question, the most disruptive cyber attacks that occurred in the last few years. One of them was a company called Change Healthcare, a major division of United Healthcare. A third of all medical transactions in the United States go through them. A hospital billing, a pharmacy dispensing a prescription, and essentially, they were hit by a cyber attack, the clearinghouse. So all these transactions were not happening.
At the time, and the CEO said this publicly, it was caused by [not deploying] multi-factor authentication, an authentication for a user other than a password. And we know passwords have been compromised so many times. enabled the attackers to get on. And then the network was also not properly segmented and configured, so the attacker was able to move along. That is the kind of thing where, from an AI perspective, firstly, an AI monitor that’s looking to see an account that has just passwords. Find them, because today you shouldn’t have accounts. Certainly you shouldn’t have admin accounts with just passwords. Find those, lock them, which will get them to be changed. Similarly, in actually identifying a network and figuring out where it can be segmented, how to optimize it so that an attacker that gets a foothold can’t move along. Finding vulnerabilities at scale, all of that, is achievable from an AI perspective today in a way that previously for a complex network was a lot of manual.
Jai (23:37)
So again, the sort of automation versus manual dichotomy. And Sam, from your perspective, where does AI help the most in a real incident? Is it time-to-detection, understanding? Is it automation of processes? How do you think about where AI becomes most effective?
Sam (23:59)
Most effective right now in processing huge amounts of disparate data sets and kind of the like higher tier of the investigation, it’s certainly useful in the like time to detect in some sense, but a lot of times that time to detect you have a lot of weak signals that you’re trying to aggregate together to find a stronger set of signals. And there’s still a lot of data plumbing to actually make that possible. It’s not like you can just shove all of your like endpoint logs into Claude code and say like, is it good or bad? Like there’s still a lot of non AI data engineering work to even make that possible. And so in reality, a lot of the real time stuff is pretty hard, but fusing contextual data sets like intel, data that’s been shared with you past incidents and what you’ve done like across your, know, whether it’s JIRA or some other ticketing system, what did we do in this case? Like, is this normal? That’s pretty powerful and basically an unlock for free that most teams have.
But a lot of times, these AI systems need complementing other software systems that are just hard and messy to build, which prevent the full unlock of AI, if that makes sense. When you have someone that’s researching adversaries, researching what they’ve done in the past, researching a clean set of data that they’ve combed through on the incident, that’s basically free. But stuff that’s larger scale is actually quite difficult still from a technical perspective.
Jai (25:27)
And on that, I want to double click on that a little bit because I’d love to get your view on what does cyber resilience look like 10 years down the road when some of this stuff is actually out there and being used? And then a compound question, which I’ve always been taught we shouldn’t ask, but I’ll ask it anyways, is this just a question of deployment or is there, and you sort of mentioned this, but is there actual innovation and building that needs to take place? Talk about that. If it’s not just deployment, what are the things we still need to invent to be able to do this?
Sam (26:03)
I’ll answer your first question first with maybe some policy twists on where I think it’s most important to achieve resilience, where it also happens to be the most difficult. But I think, I mean, 10 years from, I don’t even know what’s gonna happen in 10 years. I look at like a year or two ahead, but we need to be tested. The institutions and networks and enterprises that underpin our way of life need to be tested continuously and not just in some lightweight scan, like, hey, you don’t have any internet exposure, like you’re good.
There’s no way that is actually helpful at all. Like seriously act as an aggressor and see if there are holes throughout the enterprise and use AI to do that to make sure that you’re okay. And enterprises and organizations that are playing in critical industries, this is where policy comes into play, probably need to have some kind of like continuous testing or red teaming because otherwise, I just don’t think you can play in some of these critical spaces, whether it be energy or like another recent incident that I think was pretty noticeable to most when we’re a bunch of airlines were targeted about a year ago or so. And they were using a third party contractor to infiltrate, to get access to systems and basically shut everything down. And, you know, nobody could travel in the U.S. for a while. It was hacking 101. Like it wasn’t even hard, but there were so many doors open now that you need to basically test those things.
And yes, a Fortune 500 airline should have the budget to be able to do that, but they need to be held accountable if they’re going to be playing in these different sectors to actually go enforce that. And then I think all on the commercial side, all comes back to the board of directors and the management team, which is, this is no longer optional. We have to do this. And yes, there are some policy things, you know, reporting breaches to SEC is useful actually as a forcing function. But I think you also need to know, like, we’re going to get hit hard every single quarter if we’re going to be playing in this market. And we’re going to be up to par here. I think that is something that we should actually be looking at doing.
Anne (28:10)
I just want to add, because I love Sam’s point so much, which is today the hardest problem I used to think we had in cybersecurity was, what is measurable resilience? I recall before Russia’s invasion of Ukraine, we knew an invasion was likely. And at the time, working in the administration, the President turned to me and said, Anne, how likely is it that Russia will conduct offensive cyber attacks against the US? And at that point, we had mandated the first ever minimum cybersecurity requirements for pipelines after Colonial Pipeline. But for every other sector, the US government had zero visibility on their required minimum cybersecurity baseline. So we actually didn’t have an answer to that. What I loved about Sam’s point is today we know attackers can be continuously jiggling digital doorknobs. We want the key critical infrastructure companies to do that first and then to tell us what is the likelihood that we believe that our network could be disrupted and not recoverable for a period of time. Because I think we want to have the resilience benchmark that a critical power, water, pipeline company, if they are disrupted, can be recovered in four to six hours. And I think if you’re continuously red teaming by AI agents externally and internally, you’re far more likely to know what are the paths in and how many of those have you been able to close or address.
And before that, to be frank, it wasn’t an answerable question at a cost ratio that was reasonable for companies to do.
Jai (29:42)
I actually want to drill down on something both of you have mentioned, which is metrics, measurability, evidence. Because it seems to me that for systems to be resilient, it can’t just be anecdotally based. It’s got to be evidence-based, measurement-based in some sense. And we also know just general management theory, what you manage to what you measure. And so as we think about evidence-based cybersecurity and the types of metrics we need. Sam, I’d love to hear your thoughts on what are the metrics that are useful? What are the benchmarks that are useful in developing a system?
Sam (30:25)
I think you just need to look at the enterprise as a whole and test, especially if we’re talking from a policy perspective, internal security teams will have much more fine-grained things that they’re working on to realize their security program. at a higher level, the efficacy to respond to a legitimate threat actor is the ultimate measure. And so I think everything works back from that.
Of course, historically, as Anne pointed out, the cost of actually emulating those threat actors as a red team has just been too cost prohibitive. Thus it has never really happened at scale. That is all changing now and it can actually be employed nationally at scale. I would say even today, that’s kind of the work that we do. And so I think something about can these organizations defend and recover from like legitimately looking threat actors at some kind of continuous cadence along certain metrics, whether that’s, you have to be able to prevent and block like this lower class of adversarial behavior. And then for everything above that, you need to be able to recover within some few hour period, depending on your industry and depending on the criticality. I think that’s like the ultimate measure. There’s probably something more to unpack that to make it a little bit more continuously enforceable, but we should be deploying probably nationally certified teams that are doing this for every publicly traded company or critical infrastructure at scale. And I think that’s you know, obviously a huge workforce opportunity for us, but also maybe one of the coolest ways to employ AI to really work on national resilience.
Jai (32:01)
And any other benchmarks you can think of that are important.
Anne (32:06)
I think the key one is, as I think about the different kinds of threats, criminals in countries, can we feel confident? The average American deserves to know that when they turn on their water, the water filtration system is working. When they go to the gas station, they get gas. And I think as we look at countries like China and North Korea, China in that case that we know is pre-positioned in critical infrastructure in the US and in our allies around the world, with the goal of disrupting it, either for military objectives, if you disrupt an airport’s operations, American service members deploy through our regular commercial airports. One, they can buy time in just delaying a military deployment all the way through to concerns that a Chinese-focused disruption would be to foster panic or, frankly, to put political pressure on the domestic population of Americans saying, why are we getting involved in a crisis around the world; Americans question that, how does this fit American interests?
From a national security objective as a country, we really do want to know that the country’s core infrastructure and military bases are secure enough from a digital perspective. So I think we mentioned that before, but I really want to put a metric on that. I think AI makes that achievable if we create a digital twin, for example, of parts of the power grid and test different kinds of attacks against it in order to identify the most effective and highest ROI cybersecurity.
There’s a cost in cybersecurity that’s been at the root of the policy failures and challenges we’ve had. The last time the Hill tried to pass a cybersecurity bill was in 2015, the Lieberman Collins bill that failed. And it failed because in that case, there were differences of opinion on a number of areas. But also the issue was who bears the cost. And there’s always this debate. I’ve had CEOs come to me and say, we’re up against the government. Our government should pay. The answer is, okay, but there’s some amount of cybersecurity you’re responsible for in your network and then agreed above and beyond that could be the government. But the rich work of AI now makes it a tractable problem from a cost perspective. And as such a measurable one from a national security perspective. I think it opens up a whole new interesting line of policy work that we can do between government and private sector to fundamentally get at a more secure digital ecosystem as our economy and our security is now moved to the digital ecosystem.
Jai (34:35)
Yeah, I mean you raise an interesting point which is that the infrastructure for the most part is private sector and that’s one of the hardest things about cybersecurity that the government does not control the thing that is vulnerable and and and you also had mentioned something that I think it’s worth drilling down a little bit more which is information sharing within the private sector If you talk to some different industry perspectives or people who have perspectives, they will say that they have some fears around information sharing from antitrust, from privacy, from a bunch of other things. We talked about policy levers. I think that it’s fair to say that the government sometimes can be skeptical that there are real legal barriers, but from the private sector perspective, there’s uncertainty and we know companies don’t like uncertainty and they’re not going to take risks where they feel that there is that uncertainty. But can you talk about the challenges around information sharing and whether there might be some clarity or policy levers or maybe not that are available?
Anne (35:44)
I’m very sensitive to those concerns. I worked across four administrations. I came into government in 2007 intending for one year. And as my husband likes to say, one year became 19 years. And I was across Republican and Democratic administrations. One thing that’s interesting to me is that the core of cybersecurity, how do you protect the nation in cyberspace, remains a bipartisan goal.
To your point, what role and responsibility the private sector has, what role there is of regulation, where government requires the private sector to do the things it needs to do, because there’s a level of assurance that we want to provide citizens, has been the crux of the debate. And I think to your point, after the salt typhoon attacks, Chinese attacks against telecoms, a number of the telecoms raised the issue you raised, which is antitrust concerns.
In some, I should say, that sectors like financial services have long had extensive intra-sector sharing. It’s the most sophisticated sharing. I think it builds on some of the existing processes there were across banking. But extensive sharing happens today across banking and financial services. And it began 10, 15 years ago. And they built it because they managed to first also crack the code of concerns that competitors would expose each other for brand impact. And essentially what they said was, if anybody leaks this once, you’re out. And the benefit, because attackers use techniques again and again, the benefit to a company of being a part of it is significant. So they’ve managed to retain that shared approach.
But at the time, when the telecoms raised it, I actually, you know, brought together NSC lawyers, Department of Justice lawyers, and said, please dig into this. They dug into it and they said, there are no antitrust concerns because companies are not competing on their cybersecurity, they’re competing on their products, cybersecurity is a shared goal. So we brought in the CEOs of the major telecoms and cybersecurity companies in the aftermath of the Chinese compromise because it was so significant. And we actually brought DOJ to the table and said, please answer this directly. And they did. That being said, I think that it’s fair for companies at the beginning of an administration to say, we want to hear this said to us again because of a concern that sometimes a perspective may change. But I think as we dig into it, one can see the legal perspective of it’s not a competitive factor. So as such.
Jai (38:12)
I think that’s fair. I think the hardest thing from the company perspective, and now I see it, I sort of had your perspective when I was at the DOJ, and now as a CLO, I’m seeing it from the private sector side. The challenge is that so much depends on an administration’s interpretation. We know through court cases that guidance is not considered regulation, and is not binding. And because increasingly we have a bit of a whipsaw problem, companies would like to have more certainty either in some sort of notice and comment rulemaking legislation, something that they can point their boards to and say, we’re not going to take on liability. And here’s why, to the point of, no kind of compliance officer wants to be fired because they made the wrong call. So I think that that’s where the rubber meets the road, I think in good faith, the government can say, we don’t see a problem here and yet the uncertainty can still exist. And there’s an interagency problem, there are many agencies involved, but that to me seems to be some of the friction that could exist. And you’re right, I think financial services has solved it, partially because they’re used to information sharing. There are all sorts of ways in which they share threat information on the cyber side, on the money laundering side, that allow that so they built that muscle memory and they’re now comfortable with it. So probably some of that is my guess.
Anne (39:45)
Your comment is also reminding me of another issue we saw. And again, look at the salt typhoon or the Chinese compromises of telecoms as a good one because the impact of that compromise was so significant. Essentially, the Chinese had compromised many large American telecoms, in some cases for a number of years. So they were positioned to collect conversations at will. They could also geolocate based on the nearest cell tower individuals. So really broad in terms of potentially the number impact and the depth of the impact. The other issue I saw in those interagency discussions to your point is we brought together the FCC as the regulator and the Department of Justice and the FBI. And there was a lot of resistance by the law enforcement community to being at the same table. But the FCC brought a set of tools, responsibility, and most importantly, knowledge of the sector that was very important, because our core goal was how do we prevent this from happening again?
And what I saw was at the beginning of those meetings, I chaired those meetings, I always said, we’re here for one purpose. This is the goal. We want to prevent this from happening again. We’re one team. So no particular agency moves out and does their own thing until we have a coordinated approach because there are a set of tools and instruments of US government policy. And some are competing, exactly to your point, Jay. And to be most effective, we need to work that together, I think we perhaps, we, I’m no longer in government, I think government can do a better job sometimes of recognizing that there are sometimes competing objectives across a regulatory approach, a law enforcement approach, a partnership approach, given the private sector owns and operates this infrastructure, and really actually documenting in these kinds of incidents–here’s how we’re going to work through these different tools most effectively so companies know what to expect. You’re raising a very fair point that it’s probably pretty unpredictable.
Sam (41:37)
Yeah, the financial sector is definitely the shining beacon of I think this working well, but I think that does have a lot to do with their concentration of budget and talent. And a lot of other industries that we would deem critical infrastructure do not have that luxury. I’m thinking power, I’m thinking water, I’m thinking grid, oil and gas to some extent probably has that, but that is where we probably need to have different types of policies applied to that infrastructure because the proactive information sharing that relies on pretty talented people to implement and run those processes probably won’t work there where they might have a single IT person that’s underpinning an entire water treatment plant. And if I were an attacker looking at the homeland, I just want to disrupt and cause chaos. That would be one of my target number ones.
We probably need to look at a different policy that is maybe more like enforced deployment of technology that is like government managed, which is a little bit similar to how the PLA runs like some of their like nationwide. And Anne has pointed this out in her foreign affairs piece last year where they have persistent monitoring across all critical infrastructure because there is no such worry about data privacy. I’m not saying we should relax to that extent, but we can’t believe that folks that have no budget and basically one IT person are going to effectively implement security controls even with AI. And so we probably need more enforcement there because that’s the underbelly that’s gonna get hit the hardest.
Jai (43:12)
That’s a great perspective. And I’d like to actually drill down a little bit more on that because one thing we really haven’t talked about yet is your perspective as a builder, a founder, a CEO. You bring, I think, a unique perspective. So, I would love to hear, to the extent that you can share, what is your company building? What are the things you’re most excited about? Give us the kind of ground-eye view of the founder.
Sam (43:44)
I am a defensive optimist like Anne on this, but I think it’ll be a rocky road to get to that ultimately. So that’s maybe a long-term view at least that I have, but we build autonomous cyber systems both for offense and defense with the mission of delivering cyber resilience to the United States. And that to us is a very dual use problem. The United States’, as we’ve been talking about, critical infrastructure is composed of public and private. And so that’s kind of the composition of our customer base.
But the company in many ways is a long time in the making from myself and my cofounders of our experience. My CTO and I both started our career working in government as cyber operators. I was working at the Air Force. He was working at the NSA. And we didn’t really have the tools we wanted to win. We obviously didn’t like the bureaucracy either. We wanted to just build. And so we both found our way back to Palantir. I was there 12 years ago. I think he joined like 13 or 14 years ago. My third co-founder joined 13 years ago where we built up a lot of data trade craft and knew how to build hardcore software systems, but then got a clear view into board level priorities as it comes to software and things like that. I joined this company called Shield AI. I was their 20th or so employee, which I know is another portfolio company, figuring out how to put AI onto drones back in 2018 when it didn’t quite work that well, but we got it working and now obviously they’re hugely successful.
And this company is kind of, we’re trying to fuse all these different experiences and disciplines into building the ultimate company that can actually deliver on this mission. And that’s why we started the company. We’d always wanted to start a company and it was like, who else is going to do this in a seriously trustworthy way and has the engineering background to actually pull it off. I think a lot of folks in the security industry are tired of the security industrial complex, if you will, which is just these small niche point products that are really, really cool tools, but not built for national priorities. We asked ourselves, who else can do this in this AI moment that’s going to get out of control? We felt like we had to start this company in a way.
What we do specifically is we build a lot of the systems that, and Anne alluded to how, I liked your comment on whether you’re trying to defend and close the doors and adjust the security controls or attack, it’s the same actual start to the workflow. That’s the workflow that we deliver in our product in a kind of hyper AI enabled way, but that doesn’t sacrifice trust and scale. And that is a lot of like Palantir software training on, you know, what was access controls for them or guardrails for security AI agents for us.
And so when I talk to a Fortune 500 CISO and I say, your ultimate measure of resilience is can you emulate a realistic and relevant adversary and know that you’re up to par, they would say, sure, but nowhere near in production because I don’t trust it. And then we go through how we’ve actually built the system and they’re like, okay, I trust it. Let’s do it. And that is basically like the technical barrier to get to what we’re talking about in terms of resilience and we build that. So it’s a lot of defensive work in the Fortune 500 and in the Government space, as you could imagine, is increasingly a lot of offensive work, whether that is red teaming, which is offense for defense effectively. How do you scale those teams that have historically been like cost prohibitively expensive, but now you can unleash them across the Department of War or the US government, but also offensive cyber operations where we want the ability to deter at machine speed for reasons of both defense and offense, but that is not something that is a toy-like product and is a very serious capability which we build.
Jai (47:37)
I’d also like you to kind of share with the audience the Little Tech perspective. A lot of our policy framework at a16z is built on distinguishing between the policy environment that startups face versus what a big company faces and can deal with. But talk to us a little bit about Method Security–how many people are there? What are you like as a startup company? Are you the prototypical couple of guys in a garage? Just give us some color on the nature of your company.
Sam (48:12)
It was basement versus garage, but we are 21 people full time as of yesterday, actually. And we’ve been keeping the company lean intentionally. Software engineers are so much more productive right now when really thinking about what is the composition of company.
Jai (48:29)
Is that because of AI coding?
Sam (48:31)
Yes. And so we’ve been trying to think through first principles like what does the engineering team look like. We’re also forward deploying a lot of our engineers into customer situations to build better products, but also to just stay engineering focused. Out of the 21, 18 of us are engineers. So we’re very engineer focused. And I think that’s ultimately our strategy around raising the ultimate ceiling of method and building tech that has compounding returns. But we are taking a very small team and trying to sell to Fortune 500, the US government and from a Little Tech perspective that is very, very hard. And it’s almost like we’re on a suicide mission.
Jai (49:11)
Explain that, why is that?
Sam (49:14)
If you have not sold, delivered and sold software to the US government before and you go into it reading all this goodness on Twitter, it’s the new thing and that’s where you should put your career, you are in from a world of hurt. And I think unless you have that pain tolerance and deep desire to deliver for the mission, you will work on it for nine months and then start pivoting to something else.
And so it’s a really long-term bet both from the company and kind of how we’re building the personality of the company because – and this is a specific Little Tech thing that we have that kind of touches on AI, the government and building a business right now – things in the commercial space, especially as it relates to AI capabilities are moving so fast. The government is increasingly even further behind in their understanding of what is possible. Even with all the work that the administration is doing to their credit, actually the gap is accelerating because the innovation is accelerating too fast. And so we have stuff that is a lot of the things that we’re talking about here, we have deployed with Fortune 500 organizations. They can emulate adversaries at scale, safely at Fortune 500-level environments.
And so, you know, instead of saying, let’s go out and to commercial and there are some change engines that might get this, but as a whole, like it’s still like a lot of inertia. So what the government will do in that case is look to do some like early R&D experiments on what our capability already does in production where we’re saying like, it’s already delivered, right? You know, you don’t need to wait two or three more years to do this. It’s actually already ready.
And that is still like a huge cultural battle that we have to fight all the time. So I spend a lot of my time educating senior leaders that this is actually possible now. You can call up my Fortune 500 clients to reference here and ask if you’d like, but that has a lot to do with the color of money, you know, getting things accredited, changing what was supposed to be an early pilot with like FY28 delivery to we can actually deliver it next week.
And there’s a huge mentality change that has to happen there. And cyber is probably one of the top places where things are accelerating most quickly.
Jai (51:38)
I mean, it’s interesting because there has been some movement in the procurement space with the recent NDAA. One, I’d love you to sort of comment on how that may have impacted your ability to compete in this space. And two, there’s still some challenges, what you mentioned, I think the term of art is commercial-first, right? That’s the goal.
The world has changed. There was a time when for technology, the government was the principal buyer. And now things are progressing much more quickly in the private sector for a whole host of reasons. But is that kind of what you’re getting at as a sort of commercial-first principle and there’s still work to be done in the commercial-first space that would be meaningful for your business?
Sam (52:25)
Yeah, mean, the commercial sector is a bigger market for us, candidly. I think there’s a unique moment in time where we’re trying to put extra emphasis in the government space just because we need to figure this out and we need to figure it out today. But the commercial security market is definitely bigger as a total addressable market for us. And ultimately, I think cyber is the ultimate dual use use case, in my opinion. We want to deliver the capabilities that the government deserves and that it needs.
Yes. I know if you’re like, which side of Harvey Dent are we on that one, probably both. That is only possible if you’re commercially competing your technology with serious enterprise buyers that have no friction to buying what they want every second of every day. Otherwise it’s a gotts-like thing that’ll inevitably be a dead science project.
Jai (53:18)
And has the NDAA, does the NDAA have a meaningful impact on your business and what challenges remain that you would like to see addressed?
Sam (53:26)
It does. I think the Little Tech perspective is the top down authorities and decisions are very helpful, but it takes years for it to trickle down the defense and government apparatus to the 06 or GS 15s that are actually making the decisions. It might take two years for that culture to get down there. So it is definitely helping. And this administration is doing a very good job of injecting people that are change agents at multiple levels. But I don’t want to give people the hope that a top-down authority actually has direct effects the next day. It takes like a long time and it takes a huge battle on the inside.
Jai (54:09)
Are there any other sort of policy changes that you’d like to see in future NDAAs that would be meaningful for your business?
Sam (54:17)
I think increasing flexibility for in-year purchasing decisions for organizations is probably the most important thing to keep up with how fast commercial innovation is happening because the current program budget execution cycle is still like a three to five year cycle. And that’s just de facto three to five years behind. And so giving organizations real R&D and procurement, like increased R&D and procurement spend and gear that they can use at their discretion is necessary.
Jai (54:49)
Fantastic. Well, Sam, thank you and thank you Anne. It’s been a wonderful conversation and I appreciate your coming on the podcast.
Sam (54:54)
Thank you.
Anne (54:54)
Good to be here.
This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. Furthermore, this content is not investment advice, nor is it intended for use by any investors or prospective investors in any a16z funds. This newsletter may link to other websites or contain other information obtained from third-party sources - a16z has not independently verified nor makes any representations about the current or enduring accuracy of such information. If this content includes third-party advertisements, a16z has not reviewed such advertisements and does not endorse any advertising content or related companies contained therein. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z; visit https://a16z.com/investment-list/ for a full list of investments. Other important information can be found at a16z.com/disclosures. You’re receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters you may unsubscribe immediately.
