samspenc

I haven't played with it yet, but very impressive :) a full vector DB built on top of IndexedDB. That means it should run on all modern browsers, and should be able to handle large volumes of data (at least, proportional to disk space).

Yeah I was also going by their quarterly earnings reports since they're a public company. Though that only breaks down the revenue by continent, but was using it to approximate US revenue at 30% of worldwide.

True, but if we look at GDP ratios instead of population, it looks different.

Nigeria's GDP in 2022 was $472B, compared to US's at $25.2T, so Nigeria's total economy is roughly 2% the size of the US.

Meta / Facebook made $134B in revenue and $39B in net income (profit) last year. Assuming 1/3 of their business is in the US, that's roughly $43B revenue & $13B profit in the US last year.

If we assume their business is proportionate to GDP, they made $860M revenue & $260M profit in Nigeria last year.

So this fine is basically the company's entire profit in the country in a year.

Fascinating, despite the significantly better specs (and VRAM) on the AMD MI300x, the Nvidia H100 seems to match performance at lower batch sizes, and only loses out slightly at larger batches, I'm guessing the differentiator is mostly VRAM (192 GB in MI300 vs 80 GB in the Nvidia chip.)

Does anyone know if this is just due to ROCm vs CUDA implementations? Or something else?

Somewhat related to this topic, Move.AI https://www.move.ai/ does something very similar but specifically for 3D 'markerless' motion-capture (MoCap) data. You can use iPhones, smartphone or recently any inexpensive network-connectable camera to get synchronized video from multiple sources, and then use that sync'd footage and their service to get accurate MoCap animation without needing expensive MoCap suits or hardware.

Plask.ai and Rokoko Vision do something similar, but only support 1 camera afaik, so their camera-based MoCap is less accurate than Move.ai's multi-camera solution.

Agreed, this works great for a single or small set of users (2-3), but OP is asking about self-hosting an open-source / open-weight LLM locally to serve a larger set of users (100s per post title). I suspect they may need a larger server farm with a lot of GPUs to handle the load.

Thanks for making this open source! As someone who is a total noob in this space, I see WebXR allows one to create VR applications that run in browser across a variety of VR headsets https://en.wikipedia.org/wiki/WebXR

"WebXR Device API is a Web application programming interface (API)[1][2] that describes support for accessing augmented reality and virtual reality devices, such as the HTC Vive, Oculus Rift, Oculus Quest, Google Cardboard, HoloLens, Apple Vision Pro, Magic Leap or Open Source Virtual Reality (OSVR), in a web browser."

In your experience, does the promise hold true, is WebXR really "write once run anywhere" for the VR space?

This is par for the course for any text-to-speech or speech-to-text service these days, check out ElevenLabs' other service pricing and it is similar - they have monthly pricing but the character usage is capped at each level.

Actually other major cloud providers, including AWS, Azure and GCP, have similar character / token / word count based pricing as well.

There are a few services that do this already, but they are all somewhat lacking, hopefully Meta's paper / solution brings some significant improvements in this space.

The existing ones:

- Meshy https://www.meshy.ai/ one of the first movers in this space, though it's quality isn't that great

- Rodin https://hyperhuman.deemos.com/rodin newer but folks are saying this is better

- Luma Labs has a 3D generator https://lumalabs.ai/genie but doesn't seem that popular

From the article: "Microsoft making very similar arguments about how antitrust was going to stifle innovation" ... "when Microsoft lost its cases’ innovation didn’t end. What did decline after ... was Microsoft’s browser market share"

Yup, this is what Apple is afraid of as well, and why they are fighting it so hard with very similar arguments.

Very cool, looks like React frontend and NestJS backend, so Typescript / Javascript for both. Curious why you chose NestJS over other options?

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality:

  const session = await window.ai.createTextSession();
  const response = await session.prompt(prompt);

Bloomberg has posted the interview video on their channel now https://www.youtube.com/watch?v=zh6REnqwXe4

Was curious about this as well, seems like the closest comparison today is to Groq, wondering how it holds up.

> Sohu is built for one thing and one thing only: transformers

Thanks for clarifying this. Could you clarify whether your chip supports the transformer architecture in general, or only specific models for e.g. Llama 70B? In case of the latter, would your ASIC have to be reprogrammed for each model?

Entertaining, you won't be disappointed - but it's been a while since I've seen 1990s themed webpages with colorful wallpaper backgrounds and iframes, and this is definitely one of those. I didn't realize how hard those are to read these days.

Pretty similar, except I was thinking of the VC firm ... and I was wondering, does it really matter if Sequoia Capital is more partial towards Macs?

I think it will bring about change, but it will be a slow and long road. I don't think we are going to see instant results or changes in society, but rather something similar to the Internet and mobile, where it takes a few years to be adopted by more and more people and the technology improves over time, albeit a bit more slowly than early adopters are hoping for.

I wonder if the open-source LLM community understands what just happened here - we finally got a truly large LLM (a whopping 340B!) but it costs ... $15K per A100 x 16 GPUs = a minimum of $240K to just get started. Probably closer to $500K or half a million dollars once you factor in space, power, cooling, infrastructure etc.

First I've heard of this restriction in the SD license. Curious why they would add something like this - wouldn't they want to encourage a network of self-hosted SD solutions providing the service to someone who couldn't host or run it themselves?

Finance influencers and online forums have been abuzz with this news the past few days but now that it is happened ... looks like nothing changed? Oil prices, the US dollar and the markets seem stable?

Is this one of those events that turn out to have been "much ado about nothing"? I'm not an expert in this area, curious if anyone has more insights into this.

OK this was a bit funny:

  Top HW failure modes: 
  * GPU falling off the bus

I honestly thought "do they mean GPUs falling off a bus entering the data center" and then realized its actually the connectivity, as they mention in the next line

  GPUs falling off: In this case, GPUs are not detected by the host on PCIe.

I think fb.com is their internal domain and they never really bothered to change it, I think employees used to have a @fb.com e-mail, at least this was true a few years ago, not sure if that has changed.

Should have (2017) in the title to indicate when this was published.

Sounds like Suno AI will soon have this feature as well https://x.com/suno_ai_/status/1794408506407428215

The context for this is this tweet, basically ffmpeg (performance) was featured in an Intel keynote at Computex 2024 https://twitter.com/unevenprankster/status/17978292906016688...

More on the Intel announcements for anyone interested https://finance.yahoo.com/video/intel-flexes-chips-computex-...

I mean, this is to be expected right, the LLMs are operating at a scale of billions (trillions) of parameters, so it's impossible to explain how every single step or response works.

Overall, people can grasp what an LLM does at a high level. But to debug exactly why an LLM gives a specific response to a specific query, one would have to debug hundreds of billions of parameters.

And TIL that K-9 Mail is actually owned by the Mozilla Foundation.

I had to look up the history of this and found the following: "On 13 June 2022, it was announced that K-9 Mail had been taken over by MZLA Technologies Corporation, a subsidiary of the Mozilla Foundation with current maintainer Christian Ketterer joining the team, and plans for K-9 Mail to be rebranded as Thunderbird for Android following the completion of a feature roadmap" https://en.wikipedia.org/wiki/K-9_Mail

Good for them I guess? Mozilla does have deep pockets, I do hope they use it to improve K-9 Mail for everyone.

Nvidia's 'Chat with RTX' can do this as well https://www.nvidia.com/en-us/ai-on-rtx/chatrtx/

You do need a beefy GPU to run the local LLM, but I think it's a similar requirement for running any LLM on your machine.

Technically Recall (Windows) and Rewind (Mac) are also both self-hosted and don't run on the cloud, but this one is open-source, I think that would be a bigger selling point.