I second this (most excellent) recommendation and suggest, in particular, the band "Windhand".
For example, Windhand's song "Three Sisters":
898 karma · joined December 2, 2020
https://NN-512.com
Jonathan Aylard's NN-512 ResNet50 example:
https://github.com/jonatron/test_nn512
For example, Windhand's song "Three Sisters":
Filtered through (muffled by) ear plugs, heavy metal is a good accompaniment to software work.
All I meant was: it's not hard to write your weights to a file as floats and fread() them into a struct. I view this as the simplest possible universal interface.
And you're right: NN-512 supports only the classical ConvNet operations. It does that one thing, and it does it well. If you need LSTMs or transformers, or training, etc., another tool is needed.
Have a nice day.
The fastest way to sort integers is a radix-256 increasing-significance radix sort, and that has been a well-known fact since the 1960's. In other words: a stable bytewise counting sort, sorting from least-significant byte to most-significant byte.
A comparison sort, no matter how clever, will not outperform a radix sort, except for very small arrays or very large (many byte) integers.
Most competent engineers will be able to dump their network weights to a file as described in the headers. Do you understand your network or not? The weights, etc., are literally a C struct in the header file. Use fread() to load the floats. If that's too complicated, NN-512 is not for you.
As clearly stated on its brief webpage (which you might care to read before commenting) NN-512 does every kind of fusion you can imagine, fusing convolutions, fusing elementwise operations, fusing batch normalization, etc. See https://nn-512.com/example/28 for an example.
If you're using a classical ConvNet (e.g., a DenseNet-121), as many people are, and you want to do inference on AVX-512 CPUs, nothing will outperform NN-512. And your inference code will be completely stand-alone. No dependencies.
Generate fully specialized, stand-alone, human-readable C code for AVX-512 neural net inference:
Far outperforms TensorFlow, PyTorch, etc.
Code generator is also stand-alone (no dependencies) and is written in Go.
The AI companies are re-encoding large corpora of human creative work as a kind of compressed, conceptually-indexed representation. This allows that work to be mixed and the patterns regurgitated in very flexible ways.
Everybody's output (our creative work) is assimilated by the AI, and becomes the AI's output.
For example, OpenAI's Codex is trained on about 54 million public GitHub repositories.
This allows Microsoft's CoPilot to regurgitate pieces of that code without attribution. Code from many sources is blended together and regurgitated for Microsoft's customers (without any acknowledgement of the source).
It is perhaps the greatest theft of intellectual property in the history of Man.
The peanut gallery loves to complain about superficial aspects of Go. Typically these are people with little or no actual experience using the language and tools. They fixate on imagined problems that don't matter in practice.
But anyone who has used Go full-time for a few years is likely to deeply respect and appreciate it.
I use Vim on a concatenation. It's a simple technique that allows me to search, read, annotate, and understand how everything fits together in a medium sized C++ codebase full of templates (i.e., a pile of garbage).
Then use Vim to read the concatenation and (regexp) search.
The effective mantissa is like FP16 but it's padded out to be the same size as FP32.
In other words, there's 1 sign bit, 8 exponent bits, 10 mantissa bits that are USED, and 13 mantissa bits that are IGNORED.
1 + 8 + 10 + 13 = 32
The 13 ignored mantissa bits are part of the memory image: they pad the number out to 32-bit alignment.
Fast CPU convolutions: https://NN-512.com
Both are completely stand-alone (no external dependencies).
Every few weeks they are attacked: 12 hours of attempted logins, port scanning, etc.
It's perfectly normal and nothing new.
The bits1 and bits2 arguments to pcg32.New() should each be 64 bits of "noise".
They may be drawn from /dev/urandom or whatever. You can use crypto/rand.Read() from Go's standard library, for example. Just read 16 bytes using crypto/rand.Read() and use encoding/binary.LittleEndian.Uint64() to produce bits1 and bits2.
Or you can use a hash to produce bits1 and bits2. If you want to use a constant seed, take 32 hex digits from an arbitrary git commit id. Like this: bits1=0xd240853866f20fc3 bits2=0xe536cb3bca86c86c
It's an orthogonal issue.
Here is a stand-alone implementation of PCG32 in Go:
package pcg32
type Src [2]uint64
func New(bits1, bits2 uint64) *Src {
return &Src{bits1, bits2 | 1}
}
func (s *Src) Uint32() uint32 {
var (
x = s[0]
y = uint32(x >> 59)
z = uint32((x>>18 ^ x) >> 27)
)
s[0] = s[1] + x*6364136223846793005
return z>>y | z<<(-y&31)
}
func (s *Src) LessThan(n uint32) uint32 {
for min := -n % n; ; {
r := s.Uint32()
if r >= min {
return r % n
}
}
}A great example is this .map().filter().reduce() joke we're looking at today.
Well-written Go is absolutely sublime.
Just like with the Thai cave rescue.
Free and beautiful.
For personal projects (including NN-512) I use Mercurial for version control, not git. But in any case I don't want the compiler (or the website it generates) to depend on git or any other version control software.
That same Nanode has been running the back-end of an Android app for almost 5 years, on another port.
For many years, I have run all my websites and services (back-ends for apps, etc.) on a few $5/month Nanodes.
For temporary/experimental stuff I sometimes use Vultr and Digital Ocean, but Linode's service and reliability is superior (especially versus Vultr). Linode is just top-notch.