Triton Fork for Windows Support

Triton's team doesn't support Windows, and, worse yet, does not accept community PRs to enable any sort of support.

Here's the github issue: https://github.com/triton-lang/triton/issues/1640

And here's the performance comparison of Flex Attention with and without torch.compile (tldr it's 3x slower than a standard MHA when not compiled): https://github.com/rasbt/LLMs-from-scratch/blob/76e9a9ec02a1...

EDIT: after taking a look at the repo, the only thing changed in the "46 commits ahead of [official triton]" is the README. Somewhat sketchy.

> This is the development repository of Triton, a language and compiler for writing highly efficient custom Deep-Learning primitives. The aim of Triton is to provide an open-source environment to write fast code at higher productivity than CUDA, but also with higher flexibility than other existing DSLs.

So if you clicked in expecting the illumos-based virtualization platform, this isn't that. Though

> This is the basis for torchao, which crucially changes some large models from "can't run" to "can run" on consumer GPUs. That's easier than supporting them in other quantization frameworks, or letting the consumers use Linux or WSL

Does sound neat on its own merits.