twistedpair

Better than when Planet's DC actually exploded [1].

Restoration is hard when health and safety are in question. Good luck to these ops folks <3

[1] https://www.datacenterknowledge.com/archives/2008/06/01/expl...

"recession" talk provides cloud cover for many a company to pare back and prune products, processes, and people in a way that would normally cause more out cry. It's not that such changes aren't needed, but they can pile up until a forcing function triggers a hard look and it's acceptable to do them.

I prefer LastPass' featureset over 1Password, when I've trial them both, but I certainly don't prefer LastPass' opsec.

A good reason to give engineers PGP keys and turn on the "required code signing" feature on your org. Alas, security and productivity are perpetual odds.

I'm excited for FGPATs, but they're still in beta and still have a lot of shortcomings at this time.

IAM on GitHub needs so much <3. So broad, much ow.

For example, I trialed major security vendor's enterprise product. They required their app be granted Admin on the GitHub org. All they needed to do was create issues, PRs, and read source code for analysis. There are scopes for that.

I was eventually on a call with a principle engineer in this company, who kept saying they needed this permissions, and I kept showing him the API docs that showed that wasn't so. Eventually he said, "well, we won't _use_ all those permissions, so just give them to us anyway, because it's easier this way." Sure, I'll give you the ability to change all my code, add/remove users, drop repos... etc, and trust that some day, when you're hacked, someone will not use those over granted permissions maliciously?

Security is hard. Be careful what permissions you give your 3rd party GitHub integrations.

> found more credentials on the internal network ... giving admin access to everything

That's my hangup. The fact that admin/root level accounts can be accessed with "credentials" alone, rather than only via SSO/MFA/Yubikey. Were these service accounts, what happened to least privilege?

I'm still confused. Why did people have username/password logins to the AWS console? Either require SSO login, or require HW tokens to get in as an AWS user. Then it doesn't matter if someone finds the password file, it's useless.

Ghostery browser on Android is pretty easy to use.

Depends who the "you" is; someone just getting started with Cloud, or a savvy enterprise operator?

GCP has sided with an "easy out of the box experience". For example, a new project has a "default" network with some permissive firewall rules. A savvy operator wouldn't build things this way, but for a first time user, the cloud is daunting and a JustWorks™ experience gets them moving quickly (e.g. so they can SSH into their VMs easily).

Now, once you've gotten your feet under you, and want to build a solid cloud setup, you'll add Organization Policies [1] like "Skip default network creation", and all new Projects will be completely closed off from the web by default, at the cost of all networking being more complex. Once you're ready for this, turn it on.

So, how should a SaaS database work? Should you have to learn all the intricacies of sharding, partitioning, indexing, SELECT, FROM, HAVING, FULL/INNER/OUTER JOIN, WHERE, GROUP BY, LIMIT, before you write your first query? This is a long standing yin/yang question of product UX. What user persona and UX do you design for on the experience and complexity spectrum.

[1] https://cloud.google.com/resource-manager/docs/organization-...

Running your whole company cloud off a credit card is OK pre-seed round, but then you should get a proper business account setup w/ ACH payments/invocing.

Better still, after you grow a bit, use a GCP reseller (SADA and DoIT are great) or a usage commitment with your account manager for a nice discount. I've never heard of one of those accounts getting turned off in the middle of the night.

Sure, it sucks to hear of account lockouts like this, but productionizing your finances is part of the startup lifecycle.

FWIW, BigQuery tables can be configured to require a partition filter clause [0] in the SQL query, so that you cannot shoot yourself in the foot like that. Now if they'd just make an Organization Policy to let you turn it on by default for all new tables.

[0] https://cloud.google.com/bigquery/docs/querying-partitioned-...

This is why I switched to silk floss. Just search for it on your favorite Bezos emporium.

mabl | Boston, MA | Software Engineer | ONSITE/REMOTE | https://mabl.com/careers

We need your help to put our $40MM series C raise to work and keep scaling up. We’re bringing AI and cloud scale to software testing at mabl.

We’re a 100% serverless operation built on Google Cloud Platform that rapidly develops and deploys features on a CI/CD model. Five years in, we’re a recognized industry player and growing our engineering ranks to keep building out our platform.

Our open positions:

- Software Engineer (various levels and areas)

- DevOps Engineer / Sys Admin

- Support Engineer

- Inside Sales Engineer

Our stack is built with modern Java 11, TypeScript, and some Bash glue

Drop me (an engineer), any questions joe at-symbol mabl.com, and checkout our careers site [1]. We can’t wait to work with you.

[1] https://www.mabl.com/join-the-team

It's that easy for a lot of managed services.

Want GKE to run multi-zone, or Spanner to run multi-region, just check a box (and insert coin).

Anyone remember back when Glassdoor didn't use TLS for many years?

You're employer had a cakewalk seeing what you were posting if you did it from the office. :/

Just hit "#". That wasn't buried very deep.

Change for the sake of change.

I'm often a power user. I know all the features and shortcuts of the tools I use. All too often, these get removed. I recall being on a feedback call with a UX study, and complaining about a feature I use, and many friend use, that was recently deleted from the product. "Oh, no one uses that" was the response from the UX/PM.

There was a YT preso from several years back where the StackExchange founder explained how it ran off just ~10 servers, and could run on half that many if needed. He stressed the simplicity of their arch, and that their problem space was massively cachable, so the servers just had a few hundred GB of ram, and only had to do work to rerender pages, but could store them in cache most of the time. It was a C#.Net app.

So, I think there is a lot more in common than you think between HN and SO.

I'm not sure where the difference comes from, but it's there.

When I visit some public high schools in suburban CT, they look nicer than my college campus.

When I visit public schools in rural VA, they're a cluster of trailers and dilapidated structures that don't even have air conditioning, let less modern amenities.

So, maybe it's the taxes, or the highly active parents, or donations, but there's a visible difference, more than "$200" per student. Of course, how far and effective each dollar is WRT to outcomes is a different matter.

Here's some real data for you, in a NY college no less.

President of my undergrad college made $884K last year [1], but a medical school professor made nearly $2M.

1. https://nonprofitlight.com/ny/rochester/university-of-roches...

Who needs to buy 15 gallons of gasoline daily (unless in the transit industry or doing a long road trip)? Seems like a mismatch between frequency of earning (days), and car filling.

Most people fill their car weekly or less. Heck, since the pando, I do it less than once a month.

Ever seen Football coach, and college president salaries?

For state colleges, they're often some of the highest paid employees in the state. [1]

1. https://www.espn.com/espn/feature/story/_/id/28261213/dabo-s...

I live in a major US housing market. I want to get a bigger house for my family. Realistically, we're talking $1.2M to do so. That feels so crazy, given I grew up in a $150K house. Suddenly you feel compelled to do whatever it takes to make $400K/yr so you can buy a 3 bedroom house.

> It’s difficult for Enterprise Orgs to ensure only vetted actions are used.

You can already to this

1. Visit Settings > Actions > General Actions Permissions > Policies

2. Select "Allow non-org actions and reusable workflows"

You can then allow GitHub authored actions, verified publish actions, and OR provide an explicit allow list of actions.

This is why I love GCP Cloud Storage. The "colder" tiers are cheaper, and reads simply cost a lot more from there, but they don't slow them down and take days to restore. You pay with dollars, not time for restoring those GCS backups. e.g. Coldline [1] simply has reduced availability in exchange for being cheaper (99.9-99.95% availability, so 43min/mo, way less than "two days").

[1] https://cloud.google.com/storage/docs/storage-classes#coldli...

One assumes the backups will rotate out and be discarded after a fixed time, so everything is "truly deleted" for compliance reasons.

I know plenty of places (small/med startups) with "undelete" and "restore" account/data options built into their admin panels. Engineers shouldn't be flipping bits by hand, under duress.

I really wonder what these Altassian restore tools look like it if takes "hundreds of engineers across the company" to restore 400 accounts. Are backups siloed across many teams?

How do you think this happened? The "billable_time" database column was microseconds, not seconds? Seems like things are off by ~6 orders.

This is why I always advocate for all variables/dimensions to have the units in the name (e.g. `billable_seconds`).

> You might get bought-out or a new thing might come along that suits your model really well.

Once you get bought out, you'll likely need to integrate or replatform to the buyer's tech. Just look @ Google acquisitions. They spend the next few years paying down tech debt and scaling, so don't waste your time on that pre liquidity event. Spend that period of time on product improvements and market expansion.