712 karma · joined September 21, 2018
Customers give vendors metered access to their model. They can budget tokens per vendor. Vendors selling "AI products" can have a cleaner story and win on the margin.
The first step to is to iron out a reasonable protocol, basically authorizing a, access token, and then the model providers (OpenAI, Anthropic, etc.) do the rate limiting. Theoretically this could be done by OpenRouter too.
But even so - do customers want an "AI product" packaged cleanly, or do they want to manage token capacity? They may be forced to do the latter....
> "faster typing won't make you faster".....
I understand a Deloitte consultant has specific incentives. But let's first try to answer a baseline question: why do some companies have thousands of software engineers? What do they all do?
And then, a follow-up: what is actually the bottleneck at most companies? What causes "requirements gathering" to take long?
```
does this look right to you? don't do any searches or check memory, just think through first principles
static int vpu_mmap(struct file fp, struct vm_area_struct vm) { unsigned long pfn; struct vpu_core core = container_of(fp->f_inode->i_cdev, struct vpu_core, cdev); vm_flags_set(vm, VM_IO | VM_DONTEXPAND | VM_DONTDUMP); / This is a CSRs mapping, use pgprot_device */ vm->vm_page_prot = pgprot_device(vm->vm_page_prot); pfn = core->paddr >> PAGE_SHIFT; return remap_pfn_range(vm, vm->vm_start, pfn, vm->vm_end-vm->vm_start, vm->vm_page_prot) ? -EAGAIN : 0; }
```
And it correctly identified the issue at hand, without web searches. I'd love to try something more comprehensive, e.g. shoving whole chunks of the codebase into the prompt instead of just the specific function, but it seems the latent ability to catch security exploits is there.
So then.... I wonder how this got out in the first place. I know I'm using a toy example but would love to learn more!
Is there a way to make all this much easier to debug and to prevent memory issues in the first place? Is the abstraction level not quite right?
https://docs.nvidia.com/megatron-core/developer-guide/0.15.0...
or maybe the conclusion is that model providers need to clean up their training data!
Do the labs label code versions with an associated CVE to label them as compromised (telling the model what NOT to do)? Do they do adversarial RL environments to teach what's good/bad? I'm very curious since it's inevitable some pwned code ends up as training data no matter what.
Whoever gets the business best (and in detail) will likely be the best builders. It's "intuition as evals" that really matters in the end. You think Software Engineers or Product Managers are replacing Quants at trading shops anytime soon? Nope.
I’m also curious how this looks with browser or mobile clients. Surely they’re not sending code to the server?
And everyone else has to play catch-up.
A big source of the drift is having a common library for SDP parsing, but also necessary features like BWE, different encodings, etc.
For example, aiortc, python's WebRTC implementation, isn't quite at the level most would want. It isn't necessarily easy to tell, without clear benchmarks, which implementation is at-par with Google's.
https://www.executeprogram.com/
It uses interactive exercises + spaced repetition. It's really the only way I've managed to learn Regular Expressions.
But basically you’re saying, yes, AWS and GCP are making just as much, maybe more, margin as Hetzner?
I've heard that Rust's types and compiler can be frustrating to deal with as well. Is Zig a more "joyful" programming language?
Hasn't this always been the case?
https://randsinrepose.com/archives/the-update-the-vent-and-t...