How are sum-types handled in deep learning?
E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
Rust to the rescue, as it always is for safety. (I don't know if this directly translates for tensors, I used it for manipulating point clouds and iirc it allows for arbitrary dimension containers)
Just a word of caution for anyone interested in using it.
Things like windowing, slicing, views are all “supported” but have many limitations and are non intuitive. In particular, I needed an “axis shift” fn and I ended up spending hours scouring docs just to find a way to shift along an axis. When I mean by shift is a roll in numpy, ala: https://github.com/rust-ndarray/ndarray/issues/281
Answer: you can’t with this crate. I implemented a dynamic n-dim solution myself but it uses views of integer indices that get copied to a new array, which have indexes to another flattened array in order to avoid duplication of possibly massive amounts of n-dimensional data; using the crate alone, copying all the array data would be unavoidable.
Ultimately I’ve had to make my own axis shifting and windowing mechanisms. But the crate is still a useful lib and continuing effort.
While I don’t mind getting into the weeds, these kinds of side efforts can really impact context focus so it’s just something to be aware of.
But really, the whitepaper should include this sort of detail, and you should be following the white paper.