1 karma · joined March 4, 2021
Fear Of Not Capitalizing? FONC
The bugs in all this complex software are out of scope of the conversation. It's a theoretical system, after all. The fact that the more complex and distributed a consensus-based system is, the more likely failure of the system is, isn't mentioned. Presumably either because it's implied that you should only ever use this when you have to, or because it's actually elementary.
The most stable highly available consensus-based systems are dependent on highly available hardware and networks. As soon as either become flaky, bugs start coming out of the woodwork like a log on a roaring fire (and this explains why Jepsen is so effective at finding bugs). Othe other hand, HA systems that don't depend on consensus can be smaller, and their software is typically easier to reason about, leading to fewer classes of bugs as well as a potentially smaller codebase which leads to less bugs.
I encourage all my teams to avoid built-in CI/CD features and plugins and just script what they want in a Docker container. It ends up being easier to maintain, breaks less often, and is more portable.
A company called QualiSystems made some software called CloudShell that was basically a web UI to creating architectural diagrams and hooking each component up to code. There's drivers pre-made for each piece. You connect the pieces, click "Deploy app", and it builds everything you need, deploys your app, runs tests, tears it all down after, etc. The devs literally assemble the whole system in a UI and then just "run" the whole thing. Ops is essentially no longer needed except to write drivers and define policy/governance. This is what we need more of.
Instead, we have Ops teams who are in charge of infrastructure, and spend weeks writing custom Terraform modules and CI/CD solutions just to get some basic infrastructure up. Oh, you need a change? Sure, it'll just be another day full of writing and testing code for me to make that one tiny change in infra. It's ridiculous. Devs should be able to do this work themselves as needed, without waiting for someone else to do it for them. The same idea applies to K8s.
Personally I think "deploying to K8s without YAML" is not a challenging problem for most teams as there are lots of different solutions to that. Where they actually need help is in gaining agency and independence, and to do that we need friendly user interfaces to automated guide-rails, so if the dev wants to deploy a new stack, nothing is getting in their way, but the end result is according to a corporate standard.
> Housing prices are high because so many people want to live in highly desirable areas with high paying jobs.
Young middle class white people can't find a house in San Francisco, sure, but that's not the major problem. Most of the housing problems are in places that are not desireable, and there are a myriad of different causes.
> There are plenty of less desirable places with cheap housing and ok paying jobs.
Not even remotely enough to fulfill all the low income housing needs in the US. Even if there were, people wouldn't be able to go there anyway, because they don't have disposable income, and there's no "let me uproot your life with $20,000 you don't have to pay back" fairies flying around.
Also see https://en.wikipedia.org/wiki/Affordable_housing and https://en.wikipedia.org/wiki/Affordable_housing_by_country#...
I think the best this company can do is to become a sort of SquareSpace for housing. But we've had pre-engineered houses before (remember the Sears Catalog homes?). And we have commoditized housing now, they're just called trailers. You could totally scale-up houses by stacking trailers (that's roughly how wooden homes are built today) but either nobody wants it, or nobody's been able to scale it and make it appealing/significantly less costly.
But I wish them good luck! The sooner we can get city councils to start allowing "unusual" home zoning, the better. I want my Tiny House permit, god damnit.
Get one cookbook of some cuisine you think you'll enjoy, and look for recipes that fit within a time frame you're comfortable with (keeping in mind that for many recipes, many steps can be done ahead of time or in parallel). For each recipe, make a list of ingredients and plan what to buy, when to buy it, when to prep it, when to cook it, and when to eat it (to anticipate a repeat cycle). If you follow this method you'll end up eating what you like, rather than what's convenient for a cabinet with ketchup, instant noodles, canned mushrooms, and dried basil. (Not knocking the poor man's pasta, just saying!)
Also, most recipe databases are awash with poor-quality imitations of traditional dishes, designed purely to cater to whatever your supermarket gives you, rather than good ingredients or healthy food. This doesn't mean you should buy $4 tomatoes, but even a different thickness of noodle can make a big difference. (Though Jersey tomatoes are worth the price; just don't refrigerate them!!)
I am an under-educated over-paid GED holding Glorified Sysadmin, and while a lot of the shit solutions I work on will eventually leak, I do spend a non-trivial amount of time looking for better solutions. I also try to keep my ears open for what other people are working on and read up on concepts I'm unfamiliar with, whether they crop up on HN or through work.
I learned about CDRTs several years ago at a former employer where a team mate was talking about how it was the future. However, we didn't end up using them. It turns out the more advanced your system gets, the more work goes into it, and the more of a pain in the ass it becomes to run [until it reaches a certain level of operational stability]. So I wouldn't say that using the most bleeding edge research is a good idea even if you know about it. Most people are still fucking up the basic stuff we were doing 10 years ago, and the neckbeards here probably have loads of stories about how much simpler and more stable distributed systems used to be.
And this should extend throughout the organization. If Architecture or Security or any other group is making your life miserable, they too should be DevOps'ing, working closely with you, caring about your frustrations that only they can fix. Sadly there are still so many silos left to break up.
We really need some new terminology.
It's also interesting just how many regulations deal with e-mail addresses. https://www.absolute.com/blog/are-email-addresses-confidenti...
#yolodevops: "We're not going to fix that broken cluster, it's going to be replaced next year with The Cool New Thing.", or, "We don't have time to fix these recurring alerts, because we have to build The Cool New Thing."
#fomodevops: "You're not running a dedicated Kubernetes cluster for your single instance Java app? Get with the times, old man.", or, "Make sure you scan your code with these 15 tools to identify bugs and security problems. No, you don't need to learn how to use them or write better code, just install 'em and churn out more features.", or, "You can't have reliable infrastructure without Terraform."
I mean, that's one of the many drawbacks. Executing random independent snippets of code in different stages based on multiple variables leads to an insane amount of untestable code paths in an undefined an environment. I hope nobody depends on these for production.
Where it does seem to shine is as a sort of verbose user interface. You could use these to compose user guides to fill out forms and perform the tasks of complex workflows (in theory). But I think the snippets should still be one piece of code so you can actually develop/maintain/use it according to modern best practices.
POD documentation was good for this back in the day. Here's an example of using POD to actually define your program's input: https://metacpan.org/pod/Getopt::Euclid
Now let's say the plumber works for a contractor. The contractor tells the plumber not to tell the customer about the mistake, because it would make the contractor look bad. The plumber can choose to own up to their mistake to the customer and right the wrong, or they can do what the contractor wants and charge the customer.
On the one hand, the plumber might decide to charge the customer. They keep their job, the contractor makes money. On the other hand, maybe the customer is poor and can't really afford the repair. If it doesn't get fixed, the customer'll have to deal with the brokenness themselves, even though the plumber knows they caused it. But then again, maybe the plumber is broke and really needs this subcontracting gig.
There is no simple answer there. But I think that in the context of software development, in most cases, the answer is simple. Most of us are fortunate enough to have the extra time and money to spend fixing our own bugs, regardless of what we're told to do during the regular 9-to-5. When we have the opportunity to take responsibility for our actions, we oughta.
It takes about 8 minutes to scan the whole internet for port 22. If there's an SSH 0day, and I have a botnet, I am going to make the entire botnet scan the entire internet on every port, because I know people move their SSH port. With 5,000 hosts, I can scan all 65,535 TCP ports across the entire internet in 104 minutes.
How fast can you patch your server after the 0day's been announced? What if you're asleep?
The time it takes to change the SSH port could also be used to harden the sshd config, which would eliminate several classes of previous exploits. A little more time and the entire system could be hardened, so that even if you are attacked, the attacker can't do much of anything.
If they're following industry standard practice (I'm hoping you mean "key-based auth only") then there is absolutely zero reason to move the port, other than to avoid filling up your logs, and two IPTables rules take care of that anyway.
Moving the port is like being at a bar and needing to go to the bathroom, but first leaving your cellphone on the bar, covered with a little napkin. It is literally obscuring a thing, hoping that it is secure.
Honestly, if people insist on security by obscurity, they should just do it already and stop apologizing for it.
You already have the right to copy all of someone's years of hard work, add a tiny patch, and pass it off as your own project with a new name. The very least you can do is give respect to the original author by allowing them to keep their original name.
Then there's potential for libel. Let's say you fork a project, keep the original name, and then pepper the project with Nazi propaganda. The original author is trying to get a job, and his resume has the name of the original project. A prospective employer searches the name and finds the new project (with the old name) full of hate speech. If he gave away the old name, a libel suit to change the new project's name may fail, and his reputation might be forever tarnished.
I'm pretty sure the dead simple solution is to either use SMTP or Signal and not a frankenstein's monster of both.
Second, security scanning is just part of an overall strategy for increased software quality, which helps the product made out of the software, which is the entire point of writing software. Who cares if your stack calcifies if the user has a better experience because your app crashes less, needs to be emergency-patched less often, and doesn't leak personal data like a fire hose?
I am an SRE, and not a little bit of a security nerd, and I wouldn't trust myself with getting security right.