sig_kill

Write up here: https://meshllm.cloud/docs/pages/exo-comparison/

Yes! I’ve worked on the settings interface between our runtime and llamacpp, these are documented and available via our config.toml file

This is actually how I develop and use the mesh at home. Rather than splitting models, I aggregate disparate compute behind one endpoint, without having separate inference providers on each host and a gateway like LiteLLM

There’s a breakdown on our docs page you can check out!

https://meshllm.cloud/docs/pages/exo-comparison/

Yeah, this is one area we’re struggling with due to the sheer volume of variations and conditions, but I’ve been thinking of collecting some real-time statistics around latency, prefill/decode, and model distribution… that way we can update some kind of live + aggregated performance numbers for interested parties.

We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)

It depends on the size / structure of the model… but generally you can check out the MeshLLM huggingface org for pre-computed model splits.

We have a job that will take popular models, and dice them up and post on HF.

When that specific model is served, the layers will be shovelled around the available nodes on the mesh.

Great question! We’ve had a lot of discussions about the direction we want to take this, and how to best generate some kind of incentive / fairness reward.

And we’ve found ourselves hesitating on a direction because, at least for now, the primary use case in a useable env is private hosts you own in a mesh; which makes any sort of reward/ incentive structure somewhat unnecessary (for this setup).

When the public mesh becomes large enough and we get around to extending the existing “mesh governance” features is likely when we’ll add something like you’ve described.

For now, the public mesh is totally open with no restrictions or limitations.