(I guess the training these could be somewhat more efficient if it was codepoints instead)
382 karma · joined July 22, 2010
(I guess the training these could be somewhat more efficient if it was codepoints instead)
The article's approach just requires some indivisible base "atoms" to start the building the larger "tokens" from. (see: the DNA example)
It could be raw bits, but that's pointless, since dividing bytes into bits does not give the model any extra expressive power and requires 8× more computation. (yes, there's some structure at the bit level eg. lowercase/uppercase is represented as a bit in ASCII, but that's trivial to learn from just bytes at the scale the model operates)
Bytes are the ~best point on the expressivity/computation curve for this approach (for text).
(something else to ponder - why do bytes have 8 bits?)
R0 = R1 + R2
[1]: https://en.wikipedia.org/wiki/Blackfin#Core_featuresThis loses all non-persistent UI state such as scroll positions, half-entered text etc. etc.
Not really a workable architecture for a real app. React was invented because of this problem (how to make functional-style view building actually work...)
@dataclass @final class CaseA[T]:
field: T
type MyEnum[T] = CaseA[T] | CaseB
This also gets exhaustiveness checking in `match` statements (depending on the type checker). Overall, enums have a similar style to Scala and Java >=21 (mixing OO + ADT). I guess syntactically it can be slightly verbose depending on the exact situation...
I wouldn't call them second class, but yeah, not a lot of existing libraries use this style.Typed dicts (with the latest PEPs) are an improvement over any other language besides TypeScript. Yes, in TS, certainly `Partial` / `Pick` / `Omit` and intersection types make modeling the web API swamp easier, and that's one place TS is superior to anything else.
Python does have the Type Manipulation PEP 827 [1] out to give it similar powers to TS, but I feel that's unlikely to be implemented soon / as-is.
Having an effect system for exceptions would be nice indeed. That's actually something I'm looking into (having an "allowed exceptions" annotation for functions and then checking it at test-time via failure injection and possibly in type checkers).
This is before the DOM Canvas 2D API existed, so effects like digging are implemented by overlaying multiple 1px-high black GIF <img> tags over the background.
wccftech summary: https://wccftech.com/fujitsus-monaka-chip-3d-stacks-2nm-cpu-...
2026: https://global.fujitsu/-/media/Project/Fujitsu/Fujitsu-HQ/te...
2023: https://global.fujitsu/-/media/Project/Fujitsu/Fujitsu-HQ/te...
FugakuNEXT, the supercomputer it will be used in, also some details about the next-gen Monaka-X: https://global.fujitsu/-/media/Project/Fujitsu/Fujitsu-HQ/te...
For a single CPU: 844 GB/s memory bandwidth (12 channels of DDR5 RDIMM, 8800MT/s). 4.3-6 TFLOPS.
Basically comparable to a modern (although not top-end) GPU, so good for HPC & AI workloads. (though only 2 CPUs per node, with GPUs 4-8 is more common)
But if I'm wrong, dare say, how would you write a production memory-safe JIT compiler today?
For CT scans, which use X-rays, the visible structure is very likely due to the metallic (copper) interconnect, clock- and power-delivery layers, which are generally quite different from the underlying transistors.
These layers are larger-scale, and copper also absorbs X-rays much more strongly than silicon.
It can be applied to tracing, metrics, logging ("sampling") and profiling.
Everywhere else the percentages are quite a bit lower still (if judged by the same standards).
Search keyword: "Adaptive sampling"
Trick: it's easy to check this - just try eg. a specific short almost-nonsense high-entropy phrase such as "Scarf Color Plump 蘼撅" via OpenRouter using the playground for different providers/models, and observe the shape and language of the reasoning and the response, which tend to be quite different (MiMo and GLM are quite similar, but still identifiably different enough).
Then the problem is the language, as the grandparent observes.
Forbidding compile/build-time shenaningans is trivially bypassable and has already been bypassed in the NPM ecosystem by just making the library code itself (not the build scripts) malicious - eg. do the bad thing when the code is loaded/tested, assuming the language has static constructors.
If you do AI fixes, for the greater good, at least publish them (or even better, try to submit upstream, although expect to be ignored if the upstream does not like slop...)! :)
Seems like it's far from production-ready and there hasn't been much activity this year?
WASM is not memory safe in the same sense, in particular, heap corruption vulnerabilities still exist, which could eg. allow bypassing auth, leaking private information etc.
C compiled to WASM is still vulnerable to something like OpenSSL Heartbleed. Fil-C (and CHERI) is not.
Solvable by turning the usual C library allocation pattern inside-out:
Only the C side allocates, Rust gets views of that memory.
Also well established - calling WebAssembly components requires exactly the same pattern, as the only way to malloc is to call into the WASM side.
It's about "planning" (ie. searching an action space) which does not generally have a heuristic, so it needs to do "exploration" (which is exactly the opposite of what you want for optimal path finding) to counteract getting stuck in local optima due to greedy search...
The point of the 4×2 tiles is exactly to avoid the blockiness artifacts of vector quantization (visible in the article).
Larger tiles increase the average error and entropy because of the way VQ works - it tries to find blocks that look like each other and then encode all similar blocks by referencing a single tile (usually choosing the "average" or "mode" the blocks).
As tiles grow larger, the chance to find highly similar blocks obviously decreases (and deviation from the representative tile increases).
Reducing one of the dimensions is probably better positioned on the quality-speed frontier.
re: the greeting, I think introverts around the world do not necessarily appreciate being randomly greeted by strangers, they just get used to it if the culture is like that. From that perspective, I don't think just a random greeting from a stranger always makes people feel more warm and the days more bearable either. It's more of a correlation rather than causation.
People have different personalities and different associations.
It's not. Insincerity and mindless, automatic smiles might be a little bit frowned upon. Genuine, but subtle smiles (as appropriate for strangers) are appreciated, at least for the younger generation. The older generation might still be somewhat "traumatized" by the Soviet occupation and its cultural effects.
(jj of course has its own warts as well)