GPU memory bandwidth is the limiting factor, not PCIe bandwidth.
The memory bandwidth is critical because the models rely on getting all the parameters from memory to do computation, and there is a low amount of computation per parameter, so memory tends to be the bottleneck.
Agreed, 12 GB does not seem useful. For coding LLM, it seems 128 GB is needed to be even close to the frontier models.
For generative image processing (not video), it looks like one can get started with 16GB.
The school shooter type seems to want to be present locally, to go out with a bang, and to do it somewhere they are connected to.
They are not aiming for maximal scalability nor chance to get away.
Of course, other threat actors might, like a Unabomber type or groups designed to destabilize society (perhaps foreign sponsored) by doing repeated actions etc.
You are helping the companies train better LLMs... Both by just paying for their expenses, but also they will use the training data.
That may or may not be something one considers a worthwhile contribution.
Certainly it is less valuable than helping a person grow their intellectual capacity.
War is not amusing. In this war several hundred thousands have been killed, over a million are injured for life, and many millions are terrorized by frequent random bombings of civilian targets.
For knowledge workers in western companies, 20 USD/month for ChatGPT style tool, seems like it is likely worth it for employers to front.
I think over time we will see this included in "standard office tools" like email/messaging/videoconf/documents. Which would make Microsoft and Google very well positioned.
But probably everyone will adopt it - so it may not give a competitive advantage to any given company.
OpenAI is not at all profitable right now. Which is their plan. They have from the start been focusing on hypergrowth to become one of the very biggest players. The monitization phase comes later. Probably still a few years from now. Of course it remains to be seen if they are actually able to capture a significant amount of the value when the time comes. There are many other strong players - Google and Microsoft have a massive advantages wrt distribution and data collection. Meta as well. And they have existing profitable businesses, so they can afford to give away "AI" for a long time. Cost to serve users right now is quite high, that will likely go down by a factor 10x over the next 10 years (assuming the energy prices don't go haywire).
Duck typing is great, for example to support a range of numeric inputs - say fixed-precison integers, floats, dynamic precision integers, and numpy arrays, pandas series, tensorflow/pytorch tensor.
That duck typing can support functions with unbounded types, dos not mean that it is necessary, not generally sensible, for a particular function to support unbounded types.
Happens all the time. Problems come quickly when the datasets used for evaluation are not clean, or the evaluation is incorrect - data leakage, problematic imbalance between groups, distribution shifts vs the actual production data. Or people just checking the average performance, but not typical or worst case. Have seen many people run in circles chasing metrics that are meaningless to the task they are supposed to be solving.
Yes. And the same for DS/ML people also, please. The amount of ML people that can meaningfully drill down and actually understand the data is surprisingly low sometimes. Even worse for being able to understand a phenomena _using data_.
Seems like you have identified a potential business need! If some component could simplify the auth similarly, that would presumably be very valuable. Could be an open source project, or a startup (or both).
No they will specify something like dB SPL, A-weighted with Fast time integration. Sometimes referred to as dB(A) SPL.
And measured with a particular method, which specifies things like distance from sources, environmental conditions, etc.
When referring to sound in the physical world, "decibel" mean dB SPL (sound pressure level). Which is defined as the ratio to the smallest perceivable sound pressure level. Unfortunately that is still a bit underspecified, it may be measured with a frequency weighting like A weighting. And then there is the integration time or other temporal aggregation, but that is separate from decibel/log.
When in analog audio, it usually means dbV, relative to a reference voltage.
And in digital audio, usually dBFS - relative to the maximum amplitude that can be represented.
Property based testing is incredibly useful for state-free systems/modules. Especially those that have a wide/complex input space. Simple examples would be a general (de)serialization library for something like JSON.
In Norway, you can get 25% tax break on R&D costs. It can be combined with other softfunding up to 70% for small companies doing research, and 50% for larger companies doing development.
Not necessarily correct the answer! Sometimes it just changes it, insists that it is correct - but it might still be wrong (perhaps now in a new way)...
Strong endorsement! I would like to add that MicroPython can also be extended in C by writing small (or large) modules in C that one can call from Python. There is a module variant that can be installed at runtime (via package manager) called dynamic native modules. This enables extending with native code without having to rebuild or reflash the firmware.
MicroPython can be used either as the top-level entry point, with your C/C++ code being modules (callable from Python). Or you can embed the interpreter into a C/C++ program - see the "embed" port for an example.
I love decision trees. Conceptually simple, computational efficient and giving very good results for a lot of tasks. I especially use them on microcontroller grade system, via emlearn - which converts scikit-learn models to embedded friendly C code.
These articles are a good and pretty comprehensive introduction. I would have loved to have even more examples around the bias/variance trade off for forests, it is a key concept that not all practitioners have integrated.