Hacker News Reader

Comment by GaggiX | Hacker News Reader

Hopefully the library will support in the future the shapes of the tensors in the type system, that would be great to prevent many errors.

View on HN

After a few years, I am coming to an opposite opinion on this. Yeah, shape checking is nice and all and you should be able to run them before-hand (might be a linter pass). But inside type-system while most people have LSP running in the background and bark at you on any type mismatch? It is not going to be a pleasant development experience.

I cannot really see how it can be a worse experience than waiting for the model to run, fix the error found, wait a second time, fix another error; when you can just check instantly if the model architecture is sound.

Because that's not my workflow looks like?

* Fire up Jupyter notebook, create some PyTorch nn.Module, validate the output is expected.

* Built out the model, validate the output is expected.

* Check the data shape, do a eval() run with the data.

* Move out of Jupyter notebook (or not), to actually run the trainer.

Each of the "build out model" phase, shape inference at compile time would be kind of annoying especially if you have a LSP constantly running compilation.

That's also my workflow but the "Check the data shape, do a eval() run with the data." is the annoying process I was describing, the iterative "waiting for the model to run", "see that it doesn't work", "print random tensor shape trying the debug the code" etc... There is nothing annoying about having an oracle that can check it instantly unless you think that suffering should be part of the experience.

I am not arguing against having an oracle. Just saying in the static type-system (like the ones in Rust / C++ / Swift) would be too frequent to invoke to inhibit development. I think having a linter pass or similar is great. Much like TypeScript actually, where the type hint is the addon rather than intrinsic to the development process, making various things easier at REPL time.

Is this just an issue with the way the editor is presenting the LSP feedback to you?

Reply on news.ycombinator.com