twistedpair

GitHub simply isn't an agent code building platform. They're a RoR git host from 2008 running on an ancient MySQL setup that by some miracle continues to stay up, some of the time.

Their APIs are garbage from an agentic perspective. Try using Terraform to curate your GH org of a few hundred repos. Just running the `tf plan` will take 10min, because it's making thousands of API calls. Where's the bulk and batch APIs? No such thing. They are simply not agentic scale. Want to rerun the plan for your entire cloud vendor with thousands of VMs, IAM roles, buckets, etc... it'll run faster than checking on your repo labels.

GH needs rewritten from the ground up. The "repo is everything" arch has been creaking and falling over form many years now.

Lack of a local runner is my biggest peeve. Any novel GHA workflow creation results in a PR with 100 commits, until you can finally sort out all the non-obvious idiosyncrasies. For any modestly complex workflow, I move everything to a bash or TS file and call that, and then you can use _coding tools_ and a _local_ dev/eval loop.

How funny that GitHub Actions' lack of tools forces you to make a bunch of billed cloud runs with GitHub for workflow edits. I'm sure their PMs are very concerned about this trend.

My repos and PRs have gone from throwing 500 errors, to now all being 404. Not trending in the right direction here...

Since upgrading to Cloud Enterprise... why didn't my GH uptime get any better? Guess on-prem GHE server is the only way to do that.

They _are_ the LLM coding agent vendor, and are _owned_ by MS, the ~majority~ biggest shareholder of OpenAI. How can you NOT consider that 20x+ scaling in your capacity roadmap projections, where you are trying to get everyone to use these agents as part of your core OKRs?

Or, maybe your #1 IT priority was moving everything to Azure instead ;)

2,523 and counting

Ah, I see your problem there

> This requirement will be progressively enforced over time.

The last several years have been monotonically busier for me, year over year, and I've been a principal contributor in this game for many years. Sure, I can push out 1000 PRs a year by riding 4 concurrent agent hoards, but at some point our heads will simply explode. There's a human limit here, at some point.

I got a new boiler installed recently. $20K. I looked up the _retail_ price for the components/fittings/consumables, ~$6K. Even with overhead, that plumber made a good take for 2 days work. I'm only half kidding when I suggest the kids of today should be destined for the trades.

Honestly, Copilot is the worst of the AI tools at this point. IDK how they lost that lead so handily.

I don't think everyone will easily make the jump to coding at warp speed. Pushing 6 agentic sessions at once, while seeing a half dozen new features/fixes out to prod is more mental gymnastics. If you're the "add a button to a form" enterprise developer, this is going to feel like a dramatic shift in how you're used to working.

This is part of why we help defend Israel, to constrain wars to conventional means.

In the first Gulf War, we placed the Patriot batteries around Israel, as they said that if an Iraqi biological or chemical SCUD attack hit Tel Aviv, they would vitrify Baghdad.

Having nukes doesn't prevent _anyone_ from attacking you, but it does constrain those attacks to conventional means. And what if you pulled off a decapitation attack against Tel Aviv? Well their fleet of nuclear capable subs would make you pay.

> could the military use Anthropic’s Claude AI system to help shoot it down?

What a joke. I suggest folks read up on the very poor performance of US ICBM interceptor systems. They're barely a coin flip, in ideal conditions. How is Claude going to help with that? Push the launch interceptor button faster? Maybe Claude can help design a better system, but it's not turning our existing poor systems into super capable systems by simply adding AI.

I cannot approve PRs because the JSON API is returning HTML error pages. Something is really hosed over there.

In the age of Claude Code et al, my honest biggest bottleneck is GH downtime. I've got a dozen PRs I'm working on, but it's all frozen up, daily, with GH outages.

Are the other providers offering much better uptime GitLab, CircleCI, Harness? Saying this as someone that's been GH exclusive sicne 2010.

K8s absolutely reduced labor. I used to have a sysadmin who ensured all our AMI images were up to date and maintained, and who maintained a mountain of bespoke bash scripts to handle startup, teardown, and upgrade of our backeneds.

Enter K8s in 2017 and life became MUCH easier. I literally have clusters that have been running since then, with the underlying nodes patched and replaced automatically by the cloud vendor. Deployments also "JustWork", are no downtime, and nearly instant. How many sysadmins are needed (on my side) to achieve all of this, zero. Maybe you're thinking of more complex stateful cases like running DBs on K8s, but for the typical app server workload, it's a major win.

That one stumped me. Why not just encrypt with a hardcoded public key, then only the attacker can get the creds.

The simple B64 encoding didn't hide these creds from anyone, so every vendor out there's security team can collect them (e.g. thinking big clouds, GitHub, etc) and disable them.

If you did a simple encryption pass, no one but you would know what was stolen, or could abuse/sell it. My best guess is that calling node encryption libs might trigger code scanners, or EDRs, or maybe they just didn't care.

You're right. PNPM disables all install scripts by default. I was just noting one example.

I think it's a `pnpm-workspace.yaml` setting, for now, but PNPM has been pretty aggressive with expanding this feature set [1].

[1] https://pnpm.io/supply-chain-security

Good point, but until many popular packages stop requiring install.sh to operate, you'll still need to allowlist some of them. That is built into the PNPM tooling, luckily :)

I'm struggling to understand why Trusted Publishers is any better.

Let's say you have a limited life, package specific scoped, IP CIDR bound publishing key, running on a private GH workflow runner. That key only exists in a trusted clouds secret store (e.g. no one will have access it from their laptop).

Now let's say you're a "trusted" publisher, running on a specific GitHub workflow, and GitHub Org, that has been configured with OIDC on the NPM side. By virtue of simply existing in that workflow, you're now a NPM publisher (run any publish commands you like). No need to have a secret passed into your workflow scope.

If someone is taking over GitHub CI/CD workflows by running `npm i` at the start of their workflow, how does the "Trusted Publisher" find themselves any more secure than the secure, very limited scope token?

This is built in NPM. You can get an email on every pkg publishing.

Sure, it might be a little bit of noise, but if you get a notice @ 3am of an unexpected publishing, you can jump on unpublishing it.

ProTip: use PNPM, not NPM. PNPM 10.x shutdown a lot of these attack vectors.

1. Does not default to running post-install scripts (must manually approve each)

2. Let's you set a min age for new releases before `pnpm install` will pull them in - e.g. 4 days - so publishers have time to cleanup.

NPM is too insecure for production CLI usage.

And of course make a very limited scope publisher key, bind it to specific packages (e.g. workflow A can only publish pkg A), and IP bound it to your self hosted CI/CD runners. No one should have publish keys on their local, and even if they got the publish keys, they couldn't publish from local. (Granted, GHA fans can use OIDC Trusted Publishers as well, but tokens done well are just as secure)

Services like SES Inbound are only available in 2x US regions. AWS isn't great about making all services available in all regions :/

I just saw services that were up since 545AM ET go down around 12:30PM ET. Seems AWS has broken Lambda again in their efforts to fix things.

Wow, about 9 hours later and 21 of 24 Atlassian services are still showing up as impacted on their status page.

Even @ 9:30am ET this morning, after this supposedly was clearing up, my doctor's office's practice management software was still hosed. Quite the long tail here.

https://status.atlassian.com/

It feels like the root of the issue is the scoping design of JS itself, which makes tracking TDZ more costly for the interpreter, and the fact that JS is JIT rather than AOT compiled.

I laud the recent efforts to remove the JS from JS tools (Go in TS compiler, esbuild, etc), as you don't need 100% of your lang utils written in the same interpreted lang, especially slow/expensive tasks like compilation.

Identical, highly obfuscated (and thus suspicious looking) payload was inserted into 22+ packages from the same author (many dormant for a while) simultaneously and published.

What kind of crazy AI could possible have noticed that on the NPM side?

This is frustrating as someone that has built/published apps and extensions to other software providers for years and must wait days or weeks for a release to be approved while it's scanned and analyzed.

For all the security wares that MS and GitHub sell, NPM has seen practically no investment over the years (e.g. just go review the NPM security page... oh, wait, where?).

Mabl | US/India | Software Engineer | ONSITE / REMOTE

We need your help to scale up our cloud based, AI testing software startup and put our 40M Series C raise to work.

We’re a 100% serverless operation built on Google Cloud Platform that rapidly develops and deploys features on a CI/CD model. Seven years in, we’re a Gartner recognized industry player and growing our engineering ranks to keep building out our platform.

Our open positions:

- Software Engineer - Mobile Infrastructure (Onsite / Hybrid US)

- Software Engineer - Integrations Team (Remote India)

Our stack is built with modern Java 17, TypeScript, and Infrastructure as Code

Drop me (an engineer), any questions joe at-symbol mabl.com, and checkout our careers site [1]. We can’t wait to work with you.

[1] https://www.mabl.com/join-the-team

Should API access keys be stored in plain text such that they can easily be recovered from backups or clones to data warehouses?

Best practice would be to store such keys in an encrypted state, to prevent such breaches from non-production datasource access, or _even_ direct production database access.