brrrrrm

2,144 karma · joined February 9, 2021

http://twitter.com/bwasti

http://github.com/bwasti

[all posted thoughts and comments are my own]

they should make it accessible at https://imagen.google like how meta did with https://imagine.meta.com

mixture of experts gates on the feed forward network only. the shared weights are the KQV projections for the attention mechanism of each layer.

brrrrrm··on Gemini AI

I once met a Google PM whose job was to manage “Easter eggs” in the Google home assistant. I wonder how many engineers effectively “hard coded” features into this demo. (“What the quack” seems like one)

brrrrrm··on MLX: An array framework for Apple Silicon

factorization methods are somewhat uncommonly used in deep learning (the likely target of this framework) and have compute properties (such as approximate outputs, non-deterministic number of iterations) that make them unlike the BLAS++ standard APIs.

einsum seems like a reasonable thing to request, but it's hard to be performant across the entire surface exposed by the operation.

brrrrrm··on We have reached an agreement in principle for Sam to return to OpenAI as CEO

> 1. They can get equivalent position and pay at the new Microsoft startup during that time, so their jobs are not at risk.

citation?

brrrrrm··on Nvidia introduces the H200 an AI-crunching monster GPU that may speed up ChatGPT

this was announced last week. The major changes are memory capacity and bandwidth. That means smaller batch sizes (i.e. inference) will be faster (1.4-1.7x) and larger batch sizes (i.e. training) will be possible. Current large batch sizes, however, won't be faster.

brrrrrm··on Music ControlNet: Multiple Time-Varying Controls for Music Generation

can I try this out somewhere?

brrrrrm··on The diamond world takes radical steps to stop a pricing plunge

Sustainability is a luxury belief. The market will change when such values disseminate more broadly.

brrrrrm··on Fast and Portable Llama2 Inference on the Heterogeneous Edge

ML has extremely predictable and heavily optimized routines. Languages that can target hardware ISA all tend to have comparable perf and there’s no reason to think Rust would offer much.

brrrrrm··on Is AI the next crypto? Insights from HN comments

Genuine question: have macro trends ever, in the history of humanity, been consistently and correctly analyzed by fine-grained time series data?

I was always a bit sketched out by the macro economics classes I took in college.

brrrrrm··on Immanuel Kant – What can we know?

Philosophy is a good tool to find very poetic and often quite succinct descriptions of what you might otherwise come to on your own. It also provides a shared language among people, although often at the expense of inclusivity

brrrrrm··on AI and Open Source in 2023

Why do you say “prompts” is the canonical trade secret?

brrrrrm··on Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

Out of curiosity, why do you think the answer would be so simple and also completely untested?

brrrrrm··on 12ft is down, vercel banned me

Boo hoo? A predictable situation and an entitled/cocky developer. Sounds like he’ll get his other stuff back in due time

brrrrrm··on ExecuTorch: Run PyTorch programs on mobile and edge devices

Any plan for web support? :)

brrrrrm··on Tiny Language Models Come of Age

mind boggling that 7B is now considered a small model. I think it's valid, given the preeminence of 70B+ sized models. But wow, the community really just leap frogged over single digit billion parameter sizes.

brrrrrm··on Humans are biased. Generative AI is even worse

Any recommendations on modern philosophy/literature that goes deeply into this stuff? "Bias" is of course a woefully under-defined and weaponized word.

brrrrrm··on Efficient streaming language models with attention sinks

Hierarchical attention gets you close https://arxiv.org/abs/2210.05529

brrrrrm··on Stable LM 3B: Bringing Sustainable, High-Performance LMs to Smart Devices

how much faster is 3b in practice? Seems like an uncommon size, so it would make sense to have the title of "best 3b" lol

brrrrrm··on Stable LM 3B: Bringing Sustainable, High-Performance LMs to Smart Devices

https://mistral.ai/news/announcing-mistral-7b/

looking at the 3b results (here https://github.com/Stability-AI/StableLM#stablelm-alpha-v2 ?), it looks like Mistral (which outperforms Llama-2 13b) is far more powerful

brrrrrm··on Stable LM 3B: Bringing Sustainable, High-Performance LMs to Smart Devices

Curious how this compares to Mistral's release? 3b->7b doesn't seem like a huge leap and Mistral has shown phenomenal results for its size

brrrrrm··on DALL-E 3 is now publicly available inside Bing

Probably. I’ve had limited success getting LLMs (trained on chats/instruct) to output special codes indicating they’re communicating with a separate system (e.g. google, stable diffusion) and then taking that and feeding it back to the user

brrrrrm··on First Impressions with GPT-4V(ision)

Audio to text solves written word articulation, right? Besides this post is about vision, which also solves it.

brrrrrm··on Intel’s Ponte Vecchio: Chiplets Gone Crazy

> This is likely a compiler issue where the v0 += acc * v0 sequence couldn’t be converted into a FMA instruction.

Err, is the ISA undocumented/impossible to inspect in the execution pipeline? Seems like an important thing to verify/fix for a hardware benchmark...

brrrrrm··on How Mojo gets a speedup over Python – Part 2

I just want to see real un-hyped benchmarks. Comparing random Python native code makes no sense and seems dishonest, deterring me from actually trying out the tool.

I want a Python that can statically plan underlying GPU allocations, avoids CUDA kernel dispatch overhead and enables a multi-GPU API that isn't some multiprocessing abomination.

brrrrrm··on Jeff Bezos on AI (1998) [video]

I notice that a lot of really successful people speak quite quickly early in their careers. It's almost as if their own voices can't keep up with the excitement they have about their product/idea and they haven't quite honed in the outward presentation of it.

brrrrrm··on New York Times considers legal action against OpenAI as copyright tensions swirl

I think attempts to tease out this distinction is going to make these laws unwieldy to use in practice

brrrrrm··on New York Times considers legal action against OpenAI as copyright tensions swirl

What’s going to be the name used for the laws that attempt to tackle machine paraphrasing?

brrrrrm··on Candle: Torch Replacement in Rust

https://github.com/huggingface/candle/blob/main/candle-kerne...

That’s the C++ variant of cuda, not Rust. They’re just binding it as one might in Python. At which point, what’s the value add?

brrrrrm··on Candle: Torch Replacement in Rust

At least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for new operations, just for data loading and high-latency network stuff.

People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.

← PreviousPage 6 of 15Next →