fazkan

thanks

I also asked perplexity to analyze the data in all of these threads, this is what it came up with

https://docs.google.com/spreadsheets/d/1_ASxAG38uP3_vcVqVnhP...

clicky

https://news.ycombinator.com/item?id=39110194 (2024) https://news.ycombinator.com/item?id=34482433 (2023) https://news.ycombinator.com/item?id=29995152 (2022) https://news.ycombinator.com/item?id=29667095 (2021) https://news.ycombinator.com/item?id=24947167 (2020)

This is so weird, its not that hard to actually build an app builder. There are multiple open-source repos (bolt etc), they could have just paid their "AI engineer" to actually build an AI engineer.

Shameless plug, but we built (https://v1.slashml.com), in roughly 2 weeks. Granted its not as mature, but we don't have billions :)

In summary, uses KVM to run full Linux VMs with isolated virtual hardware on each box, and each microVM (cell) is tied to a Firecracker process in a 1:1 setup.

This reminds me of the husband-wife duo of Patrick and Radhia Cousot, who together created Abstract Interpretation [1]. Useful technique, learned about it in my formal verification class.

[1] https://en.wikipedia.org/wiki/Abstract_interpretation

tested with the old Rocm, created a PR with the latest.

haha, I was a bit confused, but I was referring to this one https://github.com/slashml/amd_inference. But the comment applies to other repos as well, do open issues in them, helps the maintainers prioritize features.

good feedback thanks, would you be able to open an issue

driver mismatch issues, we mostly use publicly available instances, so the drivers change as the instances change, according to their base image. Not saying it won't work, but it was more painful to figure out vllm, than to write a simple inference script and do it ourselves.

@Const-me according to this it should work, https://github.com/ROCm/ROCm/issues/2216

haven't tested it, but it should according to https://github.com/ROCm/ROCm/issues/2216

You just need to update the version check here

https://github.com/slashml/amd_inference/blob/4b9ec069c4b2ac...

feel free to open an issue, with the requirements and we will test it.

This is pretty cool. Is there a document that shows which AMD drivers are supported out of the box?

its just a big requirements file, and a dockerfile :) the rest are mostly helper scripts.

Thats our observation, which is why we wrote the scripts ourselves that way we can control the dependencies at least.

we have vllm in certin production instances, it is a pain for most non-nvidia related architectures. A bit of digging around and we realized that most of it is just a wrapper on top of pytorch function calls. If we can do away with batch processing with vllm supports, we can be good, this is what we did here.

you can mount a specific device to docker. If you read the script, we are mounting GPUs

https://github.com/slashml/amd_inference/blob/main/run-docke...

this uses vllm?

Just something that, we found helpful, support for new architectures is just a package update. This is more of a cookie cutter

yes, but handles all the dependencies for AMD architecture. So technically its just a requirements file :). Author of the repo above.

Kind of reminds of the cellular automaton, about how reducing the number of components can lead to complex worlds and rules. Maybe tangentially related. Conway game of life is another example.

Can someone help me make sense of this please. I know its a great team and all but sheesh.

Yeah, don't particularly like amplify, not enough control. Also, we use nextjs as as socket server, which does not play well with out of the box solutions.

haven't tried this, or rather didn't know about this, can you share some relevant resources. And your experience with them.

this seems to be too much complexity. have you ran this setup in production? curious about the maintenance overhead required?

flightcontrol looks cool, looking at the docs.

Although, your comment about it being popular, is a bit salesy. I would have checked it out nonetheless.

This, most of it, I think is to support on-prem, and cloud-flexibility. Also from the customers point of view, they can now sell the entire figma "box" to controlled industries for a premium.

Hey everyone, creator of the above repo there. Please open issues if there is anything you would like to see in this list. Or even better if you create a PR with the relevant stuff.

brainfuck, APL

I thought bazel was only needed for C++ builds? haven't seen it used in Python projects, is the utility of bazel for the C++ binaries that scientific libraries like numpy, pandas use?