Deep Learning in JavaScript

I know operator overloading is controversial, but the way it allows automatic differentiation to work transparently through regular arithmetic expressions is very helpful. Without it these libraries will never feel like PyTorch.

JavaScript is so much faster than Python, and the package management experience is so much better, that I really think a JS library would have a chance to take market share from PyTorch in the long term if it had those features.

Are you open to pull requests? If I have the time I'd love to contribute. I'm sure others would as well.

You should write up a short article on this, even something really simple like one of the examples in the README but with some commentary and examples of output and then post it to a few places like https://dev.to/ or maybe https://hashnode.com/ or even Medium (even though I'm not a big fan). There aren't many newer implementations of PyTorch in JS and I've been looking for one to learn from for some time so I'm sure there are a lot of other JS/TS developers out there that would be interested. Getting to the front page of HN certainly helps but having an article somewhere will help everyone after this week find it through a Google search.

Again, thanks so much for doing this work! It's really helpful to have everything spelled out in JS for those of us who haven't used Python much (I'm sure Python devs can relate when they think about JS projects).

In this project, I tried to reimplement most of PyTorch in Javascript. It is implemented from scratch in a well-documented, unit tested, and interpretable way, so it can help other JavaScript learners get into Machine Learning!

Hope you enjoy!

See: https://github.com/xenova/transformers.js/blob/8804c36591d11...

This is really old, but added as part of the shape of the vector as well: https://github.com/nicolaspanel/numjs/blob/master/src/dtypes...

So yours is SO MUCH MORE fleshed out than mine, but I have a similar (again very fledgling) one going here that uses Typescript:

https://github.com/Marviel/lab-grad

I think there are some really neat opportunities to increase the approachability of ML algorithms when you take advantage of the Typescript templating system to explicitly ensure you're not accidentally composing tensors of incorrect shapes.

Unfortunately, my above lib above doesn't yet handle tensors, it's still at the level of individual neurons. Hopefully someday I can bring it further along, use WASM or WebGPU, etc. :)

Edit: formatting

Making JavaScript look like Python in this case could potentially bite you in the ass.

From the example:

    const torch = require("js-pytorch");

    // Instantiate Tensors:
    x = torch.randn([8,4,5]);
    w = torch.randn([8,5,4], requires_grad = true);
    b = torch.tensor([0.2, 0.5, 0.1, 0.0], requires_grad = true);

    class Transformer extends nn.Module {
      constructor(vocab_size, hidden_size, n_timesteps, n_heads, p) {
        //.....
        this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p=p);

    /**
     * All of the arguments for this function are positional
     * and cannot be provided in a different order than defined
     */
    function performOperation(values, arg1 = false, arg2 = -1) {
        //....
    }

    /**
     * This works, but only because of the order
     */
    const result = performOperation([1, 2, 3], arg1 = true, arg2 = 10);

    /**
     * This does not work
     */
    const result = performOperation([1, 2, 3], arg2 = 10, arg1 = true);

    /**
     \* What is actually happening
     \*/
    arg1 = true; // global variable
    arg2 = 10;   // global variable

    const result = performOperation([1, 2, 3], 10, true);

https://github.com/tensorflow/tfjs

I'd love to see PyTorch in JS, but I think unless you get it running on the GPU it won't be able to do much.

Needs more layers to be really useful though.

I always thought examples using browser extensions would be neat since their built in JS and you only need to download the model once.

I was looking for something like this.

Now I just need a guide that tackles the principles of this, but from a typescript/javascript perspective.

PS: TyTorch (from Typescript) sounds nice!