Security for developers: Threat Modeling
Disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer or any other entities with which I am affiliated.
I’m trying something different with my blog. Many of you know I’ve historically written about my thoughts on security and various products. However, one of the major themes of this blog is to advocate for more engineering principles in security. As a result, I’ve also written posts about how security and engineering can work better.
These were popular, and it seems that I’ve grown an audience of software engineers interested in security. I think this is great! As a result, I commonly get requests for resources on how software engineers can learn more about security. The amount of resources on this topic has ballooned over the past few years. However, it’s rather disorganized, and like most things in security, there’s a lot of noise. I decided that I’m going to start putting out a series (“Security for developers”) on topics to help developers improve their security skills. This will also help developers at companies that don’t have the resources and/or need to hire a dedicated security person.
In this newsletter, I’ll start with something basic: threat modeling.
What is threat modeling, and why is it important?
OWASP provides a general definition for threat modeling.
Threat modeling works to identify, communicate, and understand threats and mitigations within the context of protecting something of value.
A threat model is a structured representation of all the information that affects the security of an application. In essence, it is a view of the application and its environment through the lens of security.
Well… that’s a lot to process. In more practical terms, threat modeling is a process of identifying security risks in a particular system and deciding what to do with those risks.
This is important because as a software engineer, the threat model ensures the creation of relevant defenses. It’s extremely difficult or impossible to eliminate all risks, short of actually not building the product or application. That’s because some risks will always exist. Part of the threat modeling process is to ensure that the relevant risks are being mitigated.
It’s easy to think that the risks never end. What ends up happening in most engineering teams without security expertise is that they either overestimate or underestimate a security risk. This leads to the improper allocation of security defenses. Having a security person helps the team reason through this, but it’s also possible to do this as a developer. Let’s take an extreme example. A small web application with a few users likely does not have to worry about state actors. However, companies like Anthropic and OpenAI might worry that state actors might steal their model weights. As a result, the risk of a state actor attacker for the small web application is much lower, and it’s not worth it for them to protect against that risk.
This brings us to the business reason for threat modeling. I believe this tends to get lost or is typically poorly communicated by security leaders. The goal of threat modeling is to ensure the proper resources are dedicated to mitigate the appropriate amount of risk. Every company has a different risk profile and willingness to accept different risks because each company operates in a different space and environment.
How to get started with threat modeling
OWASP and other sites have detailed articles on how to approach threat modeling. I think they are a bit too complicated, especially if you’re just getting started, but they make more sense once a dedicated security team exists. However, threat modeling doesn’t have to be so heavyweight. It’s easier to start simple and then iterate. Also, it’s better to do some basic threat modeling than none at all.
There are two components to a threat model: the threats/risks and the mitigations.
Before I discuss how to think about a threat model, it’s important to know that there’s rarely one encompassing threat model for a particular part of the application or infrastructure. It’s likely the threat model has many “sub” threat models. This will make more sense as we work through it.
I think using the OWASP basic framework is a good way to do the very minimum. Just ask yourself these 4 questions:
What are we working on?
What can go wrong?
What are we going to do about it?
Did we do a good job?
This is a bit vague, but at least it gets you started. Here’s something a bit more structured that follows this train of thought.
Many threat modeling guides say to “decompose the application.” However, I like to start by asking: what are we trying to protect? I would say if you’re doing this for the first time, it’s good to start with the most important assets, and then it’ll be easier to figure out the “sub” threat models. For example, some companies want to protect their customer data. Others, like Anthropic and OpenAI, want to protect their model weights.
Next, we should understand what an attacker could do with these high-value assets. A common and straightforward method that captures risks for many high-value assets is STRIDE.
Of course, this might not apply to all sensitive assets in an organization. For example, accessing patient healthcare data might not lead to spoofing. There’s a larger risk of information disclosure or tampering.
However, this is the next step! Figuring out what threats might happen to those assets and their likelihood. To do this, an easy way is to use the qualitative DREAD method. The categories are the following:
Damage – how bad would an attack be?
Reproducibility – how easy is it to reproduce the attack?
Exploitability – how much work is it to launch the attack?
Affected users – how many people will be impacted?
Discoverability – how easy is it to discover the threat?
From there, once you’ve ranked the threats based on their likelihood and other broader business context, you can dedicate resources to figuring out the next steps. For the threats, there are three potential, self-explanatory actions:
accept
mitigate
eliminate
The hardest part is to figure out what risks to accept vs. mitigate/eliminate since resources tend to be limited, especially at a startup. In high-growth scenarios, a resource used to mitigate a risk could be used for growth. I believe the most important goal here is to ensure that like growth, resources are used efficiently to mitigate or eliminate risks. A security expert can help figure out how to mitigate and eliminate risks efficiently. Here is the real benefit of having a security expert with a software engineering background — he/she can help figure out the technical level of effort because he/she can find a technical solution based on a deeper understanding of the architecture. With some depth, nuance, and creativity, a seemingly complex or large threat can become very manageable.
That’s it. That’s the first iteration of a threat model.
Of course, after this first pass, you can go deeper on the first iteration, but you can also break down the threat model further to “sub” threat models and repeat the process above. For example, to protect customer data, a company might develop a secrets management system to keep the keys that allow access to the customer data. This secrets management system itself might be a high-value asset, so you have to go through the threat model for this system. As a company gets more resources, it can continue to parallelize and create threat models for various systems and focus.
Exercise for the reader
Practice makes perfect! Here’s an exercise to practice threat modeling.
Let’s compare the threats for internal employee authentication vs. the threats for customer-facing authentication. Your company only has the resources to do one. Using the threat modeling guide above, under what circumstances does it make sense to prioritize one over the other?
This is the first of many series where I will talk about important security topics. I’m open to suggestions on topics, but I hope developers have a resource to better understand security.