How to become a better security engineer
Adapting an engineering mindset
Disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer or any other entities with which I am affiliated.
We’re still hiring across the board at Headway! If you’re interested in building a new mental healthcare system that everyone can access, please consider applying. On my team, I’m looking to hire a senior security engineer.
This week, I’m going to write about a slightly different topic from my usual discussion of security trends and products. In the past few weeks, I’ve written about “How to be a security person that engineers don’t hate” and “Only engineers can secure the cloud.” On top of that, my blog is heavily focused on security engineering with a strong emphasis on software engineering. Many people have asked me about resources to learn more and uplevel their skills. Consequently, this week, I thought I would write down my thoughts here along with resources that I regularly consult and “habits” that I’ve found to be helpful.
How do you adopt an “engineering” mindset?
I wrote in the subtitle that it’s important to adopt an engineering mentality. What do I mean by that? Most security people tend to be reactive. When there is a problem, they want to solve it immediately by adopting a process, buying a product, or maybe something operational like a training session to directly address the issue. However, this is tactical and only solves the issue in the short term. It rarely scales and usually leads to more operational work, which might be good for the team, but it doesn’t fundamentally solve problems.
For example, say there’s an over-provisioning issue with an application. The common response could be to restrict access, create a policy around this, train your IT team, inform the rest of the organization, etc. However, these only solve the problem in the short term and might reduce risk, but it feels like playing whack-a-mole that requires constant monitoring.
A security engineer might approach this in a different way. They might ask. What is the core issue here? It’s not that over-provisioning is insecure. It’s actually that people are provisioning because they need access to some information in the application. What is the information they need? Can we build something that gives them the information without the access? Maybe, we can build a system that provides temporary access?
The key here is that instead of thinking about how to solve the problem operationally, the security engineer asks “What can I build?” The engineer is always thinking of doing something scalable. To have this mentality requires an up-leveling of more fundamental engineering skills.
Finding the time
Before I talk about what I found to be helpful and what I do on a regular basis, there’s the question of time. I acknowledge not everyone has the privilege to have time during or outside their jobs to pursue professional development. There’s no easy solution to finding time, but it would be remiss of me not to say that it’s hard to do what I’m describing below without having dedicated time. However, up-leveling has a compounding effect. You won’t see results overnight, but even investing in the smallest improvements could yield results in the long term.
Where to start
As companies adopt more technology, security risk becomes primarily focused on technology. It’s hard to build something if you don’t understand how it’s built or what it does.
I acknowledge that security professionals come from all sorts of backgrounds, so this is not meant to be a “roadmap.” It’s a variety of tips, so feel free to cherry-pick whatever makes sense to you.
Learn how software and infrastructure are built
It’s hard to learn about complicated technologies without the fundamentals of how these technologies are built and used. Where this starts is usually around software engineering fundamentals. Here are some ideas:
Start with computer science fundamentals, e.g. data structures, basic algorithms, etc., and don’t rush into coding.
Coding is just another tool, but it’s important to learn fundamental skills that are widely transferable. In the past decade, much of the introductory computer science courses have been placed online. I personally like Harvard’s CS50 but also recommend Stanford’s CS106A
Pick up a programming language that’s widely used, such as Python and Terraform
Learn how things are built at your company from the minute someone writes a line of code up until it is deployed in production
Depending on the company, there’s likely a sophisticated chain of events that happens before the deployment.
You can also consider asking to work on a small task that shows how some parts of this deployment chain work. For example, maybe your WAF or some other security tools have configurations in Terraform. Consider making a change and see how the process of deploying production code works.
Build a simple end-to-end application and try to fill gaps in areas you don’t understand
Pick a simple application to build. It could be your own website. Learn how to use version control or store data in the database.
Don’t just build it. Understand the different parts and research how your company does it and how bigger companies do it
Understand high-level system design and architecture
This is a common interview question for software engineers, especially senior ones. They have to design a system given a basic prompt and discuss different design decisions. This is a common skill for software engineers, especially the more senior ones. If you want to speak their language, it’s good to learn.
I recommend the System Design Interview or this Github “repo”. If you don’t have a software background, a lot of this might go over your head. Some of the book explains what each part does, but there are plenty of resources online that explain how these components work
There’s definitely a lot to learn, but the compounding effect is real. Also, don’t be afraid to get hands-on and try a lot of the technologies yourself. There are resources online that will allow you to set up certain things like proxies locally. You can compare them to what happens in your company. Software engineers are typically happy to explain how something they built works!
Understanding security decisions/fundamentals
What I find often is that many times, security professionals know what should be done, but they don’t know why it’s done that way. Learning some software engineering fundamentals does help, but they aren’t security-specific.
One of the easiest ways is to take some security courses. It’s easy to do fundamental security tasks but forget why. For example, how do session tokens work?
I personally like my old advisor’s MIT secure systems class. I also recommend Stanford’s web security class. Finally, this is a great class on cryptography.
Keeping up to date on these fundamentals will help you reason about similar issues in the future
Understand security defaults in common systems
Look at common databases and understand what their security defaults are and why they exist
OWASP also has a cheat sheet series focused primarily on modern architectures, such as Kubernetes and Docker containers
Understand what your security tools actually do and how they work in the backend
Why is Snyk and Wiz deployed the way they are?
Look at what they are trying to detect and see if you can detect that yourself in a different way
Why are certain alerts the way they are in various tools?
Having fundamental security principles can help improve your understanding of the underlying technical mechanisms used to secure systems and how these mechanisms work together with that system.
Stay up-to-date on cool technologies
Many people want to jump here immediately. However, once you get the fundamentals, it’s easier to learn about new technologies because a lot of the fundamentals stay the same.
The resources here are endless. I personally like the following:
Hacker News, which is a classic
Join Slack communities related to your work
Go to meetups
Talk to your senior engineers to see what interests them and what they are working on
Being curious is an important part of staying up-to-date. Once you develop the fundamentals, it’s much easier to learn new technologies.
Takeaway
Being a good security engineer requires you to think about building solutions. You have to learn software fundamentals to understand how and what to build. There are a number of ways to learn, but it’s important to focus first on fundamentals and build your way up.