And how does one compare the results in a way that is easy to parse? 7 models producing 1 PR each is one way, but it doesn’t feel very easy to compare such.
272 karma · joined February 5, 2022
And how does one compare the results in a way that is easy to parse? 7 models producing 1 PR each is one way, but it doesn’t feel very easy to compare such.
People will go "what is this?", "huh, I’m not gonna make a user for this, can’t tell what it is". Those are my 2 cents.
Thank you for a lovely post!
I think you can afford the extra characters to show the whole page in portrait mode. (iPhone 16 pro Safari)
But if you are just interested in learning a new language and trying it in data science OR are not currently looking to enter the data science job market, then by all means: Julia is great and in many ways superior to Python for data science.
It’s just that «everyone» is doing data science in Python, and if you’re new to DS, then you should also know Python (but by all means learn Julia too!).
As someone who has had to use geopandas a lot, having something which is up to an order of magnitude faster is a real dream come true.
On your case about swapping between image representations: let’s say you’re doing a FFT to transform between real and reciprocal representations of an image - you probably have to do that transformation in order to do the the work you need doing on reciprocal space. There’s no getting around it. Or am I misunderstanding?
Please don’t take my response as criticism, I’m genuinely interested here, and enjoying the discussion.
My response is: these conversions are unlikely to be the slow step in your code, don’t worry about it.
I do agree though, that it would be nice if the compiler could simplify the math to remove the conversions between units. I don’t know of any languages that can do that.
It makes for great APIs (dot-chaining from one type to another), well-defined types (parse, don’t validate) and keeps the code associated with the type.
There’s a very active and helpful community on discord too.
I’m also confused about the perf2 performance. For the threads example it starts around 70_000 reqs/sec, while the processes example runs at 3_500 reqs/sec. That’s a 20 times difference that isn’t mentioned in the text.
One of the brilliant ideas hyperspy incorporates is that we consider datasets to have a navigation dimension and a signal dimension (think, you measure a spectrum at each point on an image), and you can easily transpose between them. This means that you can «move around» on the image and see what the spectrum looks like, or transpose and see what the image looks like as a function of the spectrum.
In particular I think the model building, where you can fit components to your dataset, is really useful.
It works best with the Jedi LSP - pyright doesn’t support the way we added lazy loading / extensions to the base hyperspy package.