Hacker News Reader

Comment by ransom1538 | Hacker News Reader

This. The only person I would trust is a person that was a sysadmin for at least 10 years and decided to specialize in security for another 5 years. So you are looking at minimum of 15 years experience to be decent. Without deep sysadmin skills - I am at a loss of what they would contribute. You are going to update our firewall without understanding what CIDR notation is? You are going to create a VPC for the dev environment not knowing what a subnet mask is? You are going to monitor security with thousands of VMs with no cloud background? Security is a specialized specialized field. Not only that you need to be a bit of a bully. You are always fighting PMs for more time to vet things and patch things - all while being a cost center.

Why do we have so many security disasters? Because those people are rare unicorns, ridiculously expensive, with no way to show added value.

View on HN

If it takes 15 years to teach someone only to a decent level, the industry needs to start laying out plans for effective training and credentialing. The solution here is not higher years required, but more effective teaching. Learning on the job is practical, but not efficient, one example being you spend time dealing with org issues rather than learning something technical.

I don't agree that it takes 15 years though. I think you're setting the standards way too high for no good reason, especially for "decent".

> effective training and credentialing

By the time you're done creating the perfect "Information Security Certification" test everything will have changed. Even the most nebulous of security certifications (CISSP; which has a super generic test that doesn't cover much in the way of "practical security") still requires 5 years of experience before you can even take it.

It's just as bad as the JavaScript ecosystem. Maybe even worse, actually.

Information security is an ultra fast moving target. The only way for companies to effectively manage it is to hire people who constantly fuck around (with technology) and are always learning (the limits of) new things. It's incredibly hard to hire (and retain) people like that.

The "safest bet" for someone who really wants to be a great InfoSec professional is to get really good at Linux systems administration then start learning how to break into things. Because once you've broken into a system you need to know how to create/execute payloads. Otherwise you're going to get stuck on the first step every single time: Finding the vulnerability. You need to be able to exploit one host and then use that one to break into another system (pivot).

Learning Windows systems administration isn't as useful IMHO because there's fewer systems and they're all the same for the most part (monoculture). You can pick up everything you need to know about exploiting Windows in a short time and then exploit it limitlessly (haha) later. Whereas Linux sysadmin skills are applicable to a very wide array of systems from embedded stuff all the way to supercomputers.

Also, if you're going to get into hardware hacking or making physical devices that help you test the security of things Windows skills are basically useless. Nobody actually loads Windows 10/11 on to something like a Raspberry Pi in order to place a physical back door somewhere (or interface with SCADA systems, air conditioners, etc).

>By the time you're done creating the perfect "Information Security Certification" test everything will have changed.

>The only way for companies to effectively manage it is to hire people who constantly fuck around (with technology) and are always learning (the limits of) new things

Then make the curriculum about fucking around with technology, taught by people who fuck around with technology for a living. Then you get a nice certificate that says you fucked around with technology for a bit and showing that you're capable of fucking around with more technology.

You're right about all the previous certs, but the solution is simple: make the curriculum match how people actually learn in the industry.

>The "safest bet" for someone who really wants to be a great InfoSec professional is to get really good at Linux systems administration

Don't get really good: get pretty good then go learn programming. You talk about jumping between embedded and supercomputers later, but programming/AppSec is more important and way more useful (esp. if you don't already know how to program)

>Learning Windows systems administration isn't as useful IMHO because there's fewer systems

Oh no mate, AD is everywhere and those skills are immensely useful to a large amount of companies. Offensive Security even changed their OSCP exam to have an AD target set.

you are utterly missing a business dynamic here in America and elsewhere.. Companies that originate in, with strong-ties to, established finance, literally push skill down the pay stack, not up. What does that mean? If a certain engineering skill is rare, it will cost more money to pay someone, and harder to find. Therefore, commoditize and automate where you can, via cloud accounts and "best practices", outsource to another company where you can, and promote internally for ruthless cost-cutting, firing and aggressive contract manipulations. This is not extreme, this is normal and daily for decades.

The imaginary skilled professional you are describing clearly originates in the mind of an engineering worker.. a person gains skill through experience and is promoted. This is opposite of what management builds over time.. Management specifically and exactly destroys this career path because it costs them more money. As long as you can commoditize and outsource, you drive costs down, not up.

Meanwhile, it is "eternal September" in the job world, with streams of 20-somethings lining up to get into the markets. Add lower cost engineers, for example in Eastern Europe, South East Asia and South Asia. Rinse and repeat.

Thank you for this. This is the first post I've read on this thread that acknowledges the reality of what it's like for career-minded infosec folks.

I'm a 15-year infosec vet. I'm not nearly as technical as some of the HN crowd would like for infosec guys to be, in large part because high technical is not something employers generally want and are willing to pay for. If you want to maximize pay, the best path is to learn just enough to be regarded as competent, then move into management, sales, or PM work. There's barely room for the highly-skilled, highly-technical cyber guy in most large companies, let alone SMBs. Most companies chop this ideal infosec role into multiple parts too minimize cost and risk, just as you describe.

Agreed. And it's actually not an American phenomenon, but a global one (as long as it needs to be listed on US stock market). The model basically is to remove IT functionalities from branches and congregates all IT power (think DBA/DevOps/etc.) to HQ so that you only need to maintain one single big IT department. In the middle of this HQ will also try to replace custom solutions by one single solution that works for all branches/departments. The branches still need to maintain a small IT team but essentially they are just configuration pushers.

Then it outsources to Eastern Europe.

You could say the same about security folks without software engineering experience, too.

When I was working in infosec consulting, by far the best colleagues were those who had software engineering experience and could empathise with developers at the client in order to understand how systems would be built, where corners might be cut, which areas might be more ropey than others etc (and then use that understanding to help inform their thinking from an attacker's perspective).

You could tell at interview too - the folks with a Computer Science background and a side interest in security were much, much better than those who took the dedicated-cyber-security degree/masters route.

You absolutely need a real generalist for security. With that said, I don't think it's unreasonable to expect a developer to know about CIDR notation, networking and cloud systems though we're perhaps straying into more DevOps-y style roles.

This has been exactly my path. Can write data security guidelines and also read PCAP files fluently. However, you will not find very many of me.

Sysadmin/ops has too many offramps that drain talent before year 10. If you can integrate software/systems well, manage projects or do advanced troubleshooting; you will likely be pulled out of ops. Conversely there are an ocean of security certifications being issued to people who have very little operational/technical experience.

Data security in practice is being reduced to a policy and procedure checklist. It is frustrating for an engineering group to receive non-specific or contradictory policy guidelines written by non-technical people, but I have yet to see that change hiring or decision making. Businesses want someone who will agree to check the box. If that someone doesn't know all the details, that makes checking the box easier.

The future of cybersecurity is not skilled coordinator/PM but instead yet another non-technical management arm handing down mandates that are blind to technical reality. There isn't another option. There aren't enough people to fulfill demand, and the compensation for cybersecurity positions are often less than a senior infrastructure role. How many sysadmins really understand networking, programming, databases, etc; While also having the people skills to not alienate both management and highly technical development and operations teams? We will never have enough people at the intersection of that many skills.

I think the problem is sysadmin is NOT something you can learn using a personal account. Same for most DevOps things too, you simply don't have 1) the $$, and 2) the many services that you can play with. You HAVE to join a large corporation to learn the real stuff.

Plus nowadays more and more companies are going on cloud, so there are fewer sysadmins jobs anyway.

I'm someone who really wants to be in some admin jobs, be it DBA or sysadmin, problem is I first joined as a business analyst, and now I'm a DWH developer/Data engineer hybrid, every step towards a admin-ish job takes way too long and difficult for me :/

It's amazing that we can train someone to be a doctor and allow them to operate on your heart, brain, etc. within less time than you'd allow someone to touch your precious environment.

This points to two issues: education needs to be addressed with more input from industry, and expectations for hiring need to be realistic. 10 years before you're able to work on something security related is not realistic, nor is it sustainable.

I would not trust a surgeon to operate on me unless they have over 65 years of experience. I want them to have operated on patients since 'Nam. You went to a state medical school? Pfft. Go kill some other patient than me.

In all seriousness, your point brings up the idea of where does the responsibility for this immensely difficult task (securing networks) fall? If we could spread out the "required" 15 years of experience into each of the developers, would that have the same effect? Building software with security baked in would reduce the need for so much work after the fact.

General security awareness training in CS programs (not the 'don't get phished' type of security awareness) would certainly go a long way, in my opinion. Security being taught as a fundamental necessity of programming would, down the road, lessen the load everywhere else.

But there is also a fundamental disconnect between what schools are teaching and what industry is hiring for. The answer right now is "Go to school for cybersec, get your certs, then work for X years as a low-level help desk agent or call-center phone jockey".

Industry needs to tell educational institutions what candidates get from being a password-resetter that isn't taught in school, and work with those institutions to get those skills into the curriculum.

I have a lot more to say on the topic of cybersecurity and hiring, but I'm getting into rant territory.

Edit to add: You mentioned 'spreading out the 15 years of required experience'. I firmly do not believe it takes anywhere near 15 years of experience to become competent at cybersec.

Amusingly I would never hire a sysadmin to do security. Sysadmins were taught how to administer and apply this knowledge. With security the field is constantly shifting, you don’t want someone who knows how to apply you want someone with a deep understanding of the underlying principles able to design and explain new solutions. It’s really an engineer job.

Amusingly, this is a recent development.

Once upon a time there were programmers.

Then there were systems programmers and application programmers. Systems programmers wrote operating systems and utilities for them. App programmers wrote apps. There was a lot of crossover.

Then there were operators, systems programmers and application programmers. Operator was a junior position who did physical things (mount tapes, plug in cables) and ran commands to do things on the systems. They usually moved up to being…

Systems administrators, who did some programming in service to the systems, but not too much. The more senior a sysadmin was, the more time they spent programming and the less time they spent doing physical things… unless they wanted to do that.

Sysadmins started to specialize. People who configured switches and routers and talked to telephone companies became “network engineers”. People who spent time working on firewalls and security policies and thinking about that became “security engineers”. Junior people who read scripts to end users became the helpdesk. And so forth.

Then we noticed that a bunch of people were doing things manually when they should be automated. This was especially bad in places where there were no senior sysadmins or systems programmers. But we did have the internet, and senior sysadmins got together and started writing tools to make their lives easier: infrastructure automation.

Remind me, which kind of engineer?

That's very cute but that doesn't mirror my experience with reality at all. Sysadmins are people who are hired with specific knowledge regarding maintenance and management of the infrastructure. They are expected to have operational knowledge, are not asked to design novel solutions to unforseen problems and are paid accordingly. Most sysadmins I have worked with become unpleasant when they reach the limits of their expertise not curious. Good sysadmins tend to leave the field for better paid position in engineering or get hired as SRE by large companies which is aking to moving to an engineering position because SRE work like engineer.

The idea that senior sysadmins are behind the push towards automation is amusing. The biggest shift in the field in the past two decades came from Google when they decided to solve the tension between developers and sysadmins by more or less firing their sysadmins and hiring engineers to do the job instead.

> The idea that senior sysadmins are behind the push towards automation is amusing.

Many true things are amusing, including this one. I think you are operating with a remarkably narrow and historically ill-informed definition of "sysadmin", and your prophecies are self-fulfilling. If you have a tension between developers and sysadmins, it's a cultural problem.

Dang second thought, actually I kinda agree with you on the number of years though. Humans are strange animals that HAVE to learn from mistakes, and their OWN mistakes. Mistakes of other people rarely ring a bell loud enough.

But I do believe that for an entry level you don't need 15 years. Maybe 5 years of sysadmin or devops should be good enough.

Reply on news.ycombinator.com