I love that this article includes a test program at the bottom to allow you to verify its claims.
405 karma · joined August 27, 2013
> Interviews can be scheduled (and rescheduled) interviews around the clock, 7 days a week.
I believe the second `interviews` should be omitted.
First, as a definition, below the noise floor means that the power of my signal at any given time is smaller than the power of the ambient noise in my channel, and usually this implies that you're only interested in a particular segment of frequency spectrum (e.g. within the 10MHz band centered at 1.8GHz). If we were doing a simple frequency-shift keying or amplitude-modulated signal, once the noise power exceeds the signal power, there is basically no hope of recovering anything useful, as those are both demodulation schemes that rely upon obtaining instantaneous estimates of the frequency or amplitude of the signal of interest.
However, spread-spectrum methods make a time/frequency tradeoff, where the signal of interest is "spread" across multiple points in time and frequency. A very simple example of this is to say "if I want to transmit a 1, instead of transmitting one cycle of a sinusoid at 18.GHz, I will transmit 10 cycles". Then, at the decoder stage, you average across 10 cycles of your carrier in order to detect whether a signal was sent or not. By doing this averaging across time, you get a 10x gain versus the noise which is expected to cancel itself out as often as not.
True spread-spectrum techniques are more advanced than this, they actually use wave shapes that are more complicated than just a sinusoid to make it easier to detect when they start and stop (whereas with a sinusoid there's a fair amount of ambiguity if you shift one period to the left or right) but the fundamental idea of averaging across time is the same.
Through this mechanism we are able to rescue out signals from far below the noise floor, although it reduces your maximum transmission rate. When dealing with digital radio systems we can even rescue out signals from below our quantization floor, although not too much lower, as eventually you lose the ability to average out a signal that is fluctuating by significantly less than a single bit.
Whenever I talk about making tradeoffs in transmission speed to aid in reception, I am reminded of the ELF systems in submarines [0]. While they did not use spread-spectrum techniques, (they just jumped between two frequencies, 76Hz and 80Hz) they still correlated across time to boost up their effective SNR. [0] https://en.wikipedia.org/wiki/Communication_with_submarines#...
My professors in grad school explicitly discouraged use of that word anywhere in technical writing, as they felt it would immediately give the reader the impression that the actions taken in the research were not thought through. Example: “This new technique enables arbitrary manipulations of data” should instead be replaced by something like “this technique enables a wide range of manipulations of data”.
julia> using LoopVectorization, BenchmarkTools, Test
function AmulB!(C,A,B)
@turbo for n = indices((C,B),2), m = indices((C,A),1)
Cmn = zero(eltype(C))
for k = indices((A,B),(2,1))
Cmn += A[m,k]*B[k,n]
end
C[m,n]=Cmn
end
end
M = K = N = 144; A = rand(Float32, M,K); B = rand(Float32, K,N); C0 = A*B; C1 = similar(C0);
AmulB!(C1,A,B)
@test C1 ≈ C0
2e-9*M*K\*N/@belapsed(AmulB!($C1,$A,$B))
96.12825754527164
Perhaps they have more extensive examples coming that showcase the capabilities further. I understand it's difficult to show all strengths of the entire system in a short demonstration video. :)
EDIT: As expected, there are significantly better benchmarks shown at https://www.modular.com/blog/the-worlds-fastest-unified-matr... so perhaps this whole discussion truly is just a matter of the demo not showcasing the true power of the system. Hopefully achieving those high performance numbers for sgemm is doable without too much ugly code.
If you have an example of this, I'd be interested in tracking it down. I didn't see this in Yuri's blog post, nor am I aware of any egregious examples of this right now.
I run three 1080p monitors off of my M1Pro, and I don’t notice any heavy CPU load from whatever software load the driver may be adding.
You can see some Julia code that generates the sandbox config rules here: https://github.com/JuliaCI/sandboxed-buildkite-agent/blob/ma...
This is supported by taking the code found in the gist [2] linked from my SO answer and running it on my M1 Pro. Compiling it, we get `dgesv_accelerate` which uses Accelerate to solve a medium-size linear algebra problem, that typically takes ~8s to finish on my M1 Pro. While running, `htop` reports that the process is pegging two cores (analogous to the result in my original SO answer on the M1 pegging one core; this supports the idea that the M1 Pro contains two AMX co-processors). If we run two `dgesv_accelerate` processes in parallel, we see that they take ~15 seconds to finish. So there is some small speedup, but it's very small. And if we run four processes in parallel, we see that they take ~32 seconds to finish.
All in all, the kind of linear scaling shown in the article doesn't map well to the limited number of AMX co-processors available in Apple hardware, as we would expect the M1 Max to contain maybe 8 co-processors at most. This means we should see parallelism step up in 8 steps, rather than 20 steps as was shown in the graph.
Everything I just said is true assuming that a single processor, running well-optimized code can completely saturate an AMX co-processor. That is consistent with the tests that I've run, and I'm assuming that the CFD solver he's running is well-written and making good use of the hardware (it does seem to be doing so from the shape of his graphs!). If this were not the case, one could argue that increasing the number of threads could allow multiple threads to more effectively share the underlying AMX coprocessor and we could get the kind of scaling seen in the article. However, in my experiments, I have found that Accelerate very nicely saturates the AMX resources and there is none left over for future sharing (as shown in the dgesv examples).
Finally, as a last note on performance, we have found that using OpenBLAS to run numerical workloads directly on the Performance cores (and not using the AMX instructions at all) is competitive on larger linear algebra workloads. So it's not too crazy to assume that these results are independent of the AMX's abilities!
[0] https://stackoverflow.com/a/67590869/230778 [1] https://stackoverflow.com/a/69459361/230778 [2] https://gist.github.com/staticfloat/2ca67593a92f77b1568c03ea...
When using these JLL packages from Julia, it will automatically download and load in dependencies, but if you're using it from some other system, you'll probably need to manually check out the `Project.toml` file and see what other JLL packages are listed as dependencies. As an example, `LLVM_full_jll` requires `Zlib_jll` [3], since we build with support for compressed ELF sections. As you may have guessed, you can get `Zlib_jll` from [4], and it thankfully does not have any transitive dependencies.
In the Julia world, we're typically concerned with dynamic linking, (we `dlopen()` and `dlsym()` our way into all our binary dependencies) so this may not meet all your needs, but I figured I'd give it a shout out as it is one of the easier ways to get some binaries; just `curl -L $url | tar -zxv` and you're done. Some larger packages like GTK need to have environment variables set to get them to work from strange locations like the user's home directory. We set those in Julia code when the package is loaded [5], so if you try to use a dependency like one of those, you're on your own to set whatever environment variables/configuration options are needed in order to make something work at an unusual location on disk. Luckily, LLVM (at least the way we use it, via `libLLVM.so`) doesn't require any such shenanigans.
[0] https://github.com/JuliaBinaryWrappers/ [1] https://github.com/JuliaBinaryWrappers/libLLVM_jll.jl/releas... [2] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/rele... [3] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/blob... [4] https://github.com/JuliaBinaryWrappers/Zlib_jll.jl/releases [5] https://github.com/JuliaGraphics/Gtk.jl/blob/0ff744723c32c3f...
I use the Dell D6000 and I run three (1080p) external monitors in addition to the built in monitor.
[0] https://www.displaylink.com/products/universal-docking-stati...
The downside is that generating system images can be quite slow, so we're still working on ways to generate them incrementally. In any case, if you're inspired to work on this kind of stuff, it's definitely something the entire community is interested in!
For the package ecosystem as a whole, we will be slowly increasing the number of third-party packages that are built for aarch64-darwin, but this is a major undertaking, so I don't expect it to be truly "finished" for 3-6 months. This is due to both technical issues (packages may not build cleanly on aarch64-darwin and may need some patching/updating especially since some of our compilers like gfortran are prerelease testing builds, building for aarch64-darwin means that the packages must be marked as compatible with Julia 1.6+ only--due to a limitation in Julia 1.5-, etc...) as well as practical (Our packaging team is primarily volunteers and they only have so much bandwidth to help fix compilation issues).
To answer the GP, the minimum amount of work necessary is to write a number for each element in the array, which scales by n^2 since there are n^2 elements. It is of course only possible to beat this if you can make assumptions like “the diagonal of my matrix is nonzero and everything else is zero”, but that’s not general matrix multiplication anymore, that’s a specialized sparse matrix routine. For a dense, “normal” matrix, O(n^2) is the absolute best we could hope for.
In my experiments with transmitting realtime audio signals using Opus, 10ms frame sizes are acceptable for 1-way synchronicity (e.g. if you want the user to perform an action and hear the result as simultaneous with the action) but it's definitely the upper bound. From what I remember my signal processing professor say, the threshold to try and hit is 7ms total latency for truly undetectable processing, but I can't find a reference for that, so take it with a grain of salt.
[0] https://www.opus-codec.org/docs/html_api/group__opusencoder....
The abstractions that are built up within that codebase feel very clean, like they get out of your way so that you can be productive, without drowning you in syntax.
I didn't take it as such; there are legitimate shortcomings to any tool, I just wanted to provide pointers to other readers that the devs are aware of it, and that there is ongoing development to address it. :)
> Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things.
As someone who, before Julia, was firmly entrenched in C/C++/Python land, I suppose I am discovering many of these "obvious" things for the first time. :)
That being said, packages such as SIMD.jl [0], and LoopVectorization.jl [1] are making fantastic progress, to the point that LoopVectorization forms the basis of a legitimate BLAS contender, in pure Julia [2]. It's not totally there yet, but it's close enough that real work is being done in LV at OpenBLAS-like speeds.
As an aside, I find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible enough to allow for direct manipulation of the LLVM intrinsics being emitted by user code.
[0] https://github.com/eschnett/SIMD.jl [1] https://github.com/chriselrod/LoopVectorization.jl [2] https://github.com/MasonProtter/Gaius.jl