bronxbomber92

I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.

A question for the author(s) since they seem to be very responsive to this thread :).

1. How fine grain is each task? In a traditional matrix multiplication kernel, for example, each thread block is responsible for a small output tile of the resulting matrix. In Mirage's mega kernel, would there correspondingly be a task for each small output tile?

2. How does the Mirage compiler form the task graph? Does it have domain knowledge of every operator's data flow at the granularity of individual elements? Again taking matmul as an example: a given output output tile requires the correspond M_BLOCK rows of the A matrix. If the A matrix was itself an output of a prior matmul (+ nonlinearity), the dependees would be all of output tile tasks corresponding to those M_BLOCK rows of the operator that produced A?

An example of “AST via reflection”

https://docs.scala-lang.org/scala3/reference/metaprogramming...

What are the hero use cases for AI agents? Reading the website, I don't see a good demonstration of what "AI agents" are and what they're good for.

I believe this post is referring to device-scoped memory barriers - also sometimes called fences - as opposed to execution barriers.

The former being a mechanism to ensure memory accesses follow a well defined order (e.g. it'd be bad if the memory accesses executed inside a critical section could be reordered before or after the lock and unlock calls).

The latter being a mechanism that ensures all threads (within some scope, perhaps all threads running on the "device") reach the same point in the program before any are allowed to proceed.

Threads are also a tool for concurrency.

The author states it wasn't actually work-life balance that was making him happy and tired. Rather, he discovered:

> By mid-2021 I was tired all the time. I know I wasn’t alone, because it was an ongoing meme inside Google2. It’s only now that I realize what was wrong: I missed the satisfaction of building things and finishing projects.

This is the same for any specialized software engineering role. Compilers, GPGPU, embedded systems, computer graphics, image processing, etc. In an interview panel for any of these roles, you will be expected to be a competent software engineer and have domain knowledge about the sub-field.

Do you have references to the incident(s) or published work(s)?

> The players are traded freely among teams Yes, this happens more often than in the past, but there are still many players and staff that stay with an organization for an extended period of time (e.g. Tom Brady and Bill Belichick).

> all teams are owned by the same business What do you mean? Each team has a different owner.

> So for example the San Francisco 49ers have very little to do with San Francisco, except for the name' In many geographical regions, you grow up watching the team that is closest in proximity. e.g. I grew up in rural western New York State and I grew up watching the Buffalo Bills because they were the closest franchise despite being 100 miles away.

The name is just a semantic. Would you be happier if the 49ers were called the "Northern California 49ers"?

My impression was that the iTunes stunt was not received well because they had already lost popularity. The iTunes stunt didn't help, but I doubt it materially hurt their popularity either.

There is roughly equal levels of complexity in any modern, high performance, general purpose/programmable chip.

How much would the trade offs change if GPUs shared the same main memory as CPU?

> It occurs to me that for this particular algorithm there might be a chance to rescue it. Instead of simply spinning, a waiting workgroup might make a small amount of progress recomputing the aggregate for the partition it's waiting on. After a finite number of spins (the partition size divided by this grain of progress), it would have the aggregate for the partition so would be able to move on to the next partition. Thus the correctness concern becomes a performance concern, where a very high probability of the spin yielding early is likely "good enough."

Yes, I've seen this workaround perform well in practice, sorry for not mentioning it >.<. It does blow up any asymptotic efficiency guarantees though, which may or may not be acceptable for the application at hand.

> I'll not make any promises about another blog - I worry that subgroup size tuning is too much in the weeds other than for a very specialized audience. But I certainly do hope to blog more about piet-gpu and will see if I can touch on the topic then.

Looking forward to any future piet-gpu posts you may write :-).

On 1, you are right that most HW most of the time will make the requisite forward progress after the atomic bump. But Vulkan does not guarantee it and most HW doesn't either (you're just getting lucky with a high probability).

The property you're describing is discussed in https://johnwickerson.github.io/papers/forwardprogress_concu..., page 1:10:

"After probing our zoo of GPUs with litmus tests, we determined that all GPUs had the following property: once a work-group begins executing a kernel (i.e. the work-group becomes occupant on a hardware resource), it will continue executing until it reaches the end of the kernel. We call this execution model the occupancy bound execution model, because the number of work-groups for which relative forward progress is guaranteed is bound by the hardware resources available for executing work-groups; i.e. the hardware resources determine how many work-groups can be occupant."

However, I know even that property is not true of all GPUs and thus cannot be assumed by Vulkan spec compliant code. (I actually doubt it's true of any GPU's pre-Volta - it's just that the progress needed is often a natural side-effect of switching between threads for latency hiding).

Volta does guarantee this property + more. All starvation-free algorithms are supported by the architecture. See https://devblogs.nvidia.com/inside-volta/. In particular, this line:

"Independent thread scheduling in Volta ensures that even if a thread T0 currently holds the lock for node A, another thread T1 in the same warp can successfully wait for the lock to become available without impeding the progress of thread T0."

Independent thread scheduling is often thought about as providing a deadlock freedom guarantee between threads in the same warp, but the same guarantees must also extend to coarse execution scopes (e.g. a workgroup waiting on another workgroup).

So, as far as Vulkan is concerned, I do not consider this a defect in the spec. I have seen this exact algorithm deployed in production and deadlock despite never having deadlocked at the developer's desk.

On 3 - sounds like another good blog post topic. It'd be interesting to hear your experience trying to (and being unable to) write shaders that are subgroup agnostic but use subgroup functionality. I should Brian's blog post on the matrix transpose as well, maybe some of that experience is documented there.

A couple of small comments.

1. The single pass prefix sum the author implements is not legal Vulkan code. The algorithm requires spinning on conditions to be satisfied by concurrently running threads. This is allowed to deadlock in Vulkan and is mentioned explicitly in the "Limited Forward Progress Guarantees" section of the Vulkan Memory Model blog post linked to by the author. The only GPUs on which is it technically safe to run this algorithm are the Volta and newer NVIDIA architectures.

2. The mask argument of the CUDA shuffle intrinsics is there, not because it is a more powerful abstraction, but because compiler technology of today is unable to prevent illegal code motion of shuffle intrinsics. There are threads on the LLVM mailing list that discuss this in some detail.

3. I am happy to see this topic written about. Portability of compute algorithms and where APIs differ or lack is not well understood and CUDA tends to dominate the discussions. Regarding subgroup portability - I do wonder if asking developers to write their code for specific subgroup sizes and allowing them to request subgroup sizes is the correct long term abstraction. In Metal you can get a long way writing code that uses subgroup ('simdgroup' in Metal speak) functionality without actually making any assumptions about the subgroup size. It is a balance - there will always be performance to be had if you hard code such assumptions, but the more we can close the gap, the less it makes sense to do so.

I know very little about the actual technology, but I feel like the blurb on their front page explains this succinctly:

“ Firecracker enables you to deploy workloads in lightweight virtual machines, called microVMs, which provide enhanced security and workload isolation over traditional VMs, while enabling the speed and resource efficiency of containers”

Security of a VM + efficiency of a container.

Does “1M for 4 years stock” mean a 1M grant per year that each vest over 4 years or 4 years worth of grants that total 1M?

We have a similar culture on my team. When I was a junior engineer, being able to poke my head into any open office that sounded like it had an interesting conversation going on was really an invaluable way of learning.

Metal 1.0 reflected the capabilities of mobile GPUs at the time. Apple is not in a race with Khronos.

You're understanding of the history of Mantle, Metal, and Vulkan is very poor.

1. Mantle was not originally being proposed as a "open" API as you are implying. A reading of the [original white paper](https://www.amd.com/Documents/Mantle_White_Paper.pdf) makes apparent that AMD was trying to leverage their game developer mind-share from the consoles by bringing it to desktops. Mantle was being proposed as an AMD GPU only API.

2. Metal was released 10 months after Mantle was announced (and 3 months after API docs surfaced). That's an extremely short timeline for a big company to go from zero to deciding to release a new graphics API, spec that API and it's shading language, write drivers and compilers for it, write developer tools, and author documentation. The timeline you're proposing simply doesn't make sense. Apple was clearly working on Metal long beforehand.

3. Vulkan, as an initiative, was reactionary to Apple's release of Metal, being that - up to that point - iOS and macOS were the largest OpenGL[ES] markets and Apple the only company commercially invested in OpenGL (OK, maybe not the only, but clearly Apple was the most influential Khronos member). By the summer of 2014, it was clear that Mantle - as an AMD only API - was not commercially viable. Thus, it was "donated" (so to speak) to Microsoft and Khronos to bootstrap DX12 and Vulkan development. (My understanding is that DX12 was already under development, but it's clear that the API shifted drastically in response to Mantle as evidence by the DX12 version 1.0 docs containing passages verbatim taken from the Mantle programming guide).

Please stop spreading false narratives in order to bolster your agenda.

And macOS. And iOS.

Hmmm, I think I know where the OP saw this link posted originally :-).

It's a simplified form of Hungarian notation: https://en.wikipedia.org/wiki/Hungarian_notation

On a related topic, what are other's favorite learning resources for applied distributed systems?

I assume you mean suburbs of New York City, as the suburbs in the rest of the state are not nearly expensive. Rent for a 1 bedroom apartment will be much closer to $800 (and I'm not so sure the job prospects in a NYC suburb is substantially better than a Rochester/Buffalo/Syracuse/Albany suburb given how expansive, and thus difficult to commute into, the city is?).

If the Metal debug layer doesn’t warn you, then it is a bug (either in the driver or in the debug layer depending on the intended behaviour).

What is the nature of your work within SWE?

What did your day to day look like?

"Cloud" can mean a lot of things, so what does it mean to you in this context?

Edit: Put another way - day to day, what are people in such roles working on exactly?