HNHacker News
TopNewBestAskShowJobs

ihnorton

1,471 karma · joined October 23, 2011

Software engineer at TileDB

isaiah plus hnhiring -at- tiledb -dot- com

submissionscomments
ihnorton··on Faster CPython Ideas – Issue Tracker for Faster CPython Project
CFFI [1] is a step in the right direction, inspired by LuaJIT's CFFI. It originated from the PyPy folks and is supported in PyPy [2]; it's also supported to some degree by Numba [3]. I don't know what level of C call optimization is available when using the JITs, so I can't speak to the performance, but I've used it casually via CPython and was impressed by the API. That said, it has been around for a while and the traction seems somewhat limited -- I would guess because most people who have this kind of problem also need more than "just" FFI.

[1] https://cffi.readthedocs.io/en/latest/index.html [2] https://doc.pypy.org/en/latest/extending.html#cffi [3] https://numba.readthedocs.io/en/stable/reference/pysupported...

ihnorton··on We were promised Strong AI, but instead we got metadata analysis
Even for programming topics, sites like gitmemory routinely rank higher than the original GitHub repos and StackOverflow answers that they are "mirroring", or the mirror page gets full result billing while the original StackOverflow answer only gets a single title line in the "other results from site" format.
ihnorton··on You can handle The Diamond with CMake
Many (if not most) libraries don’t work correctly with add_subdirectory.
ihnorton··on Ask HN: What are some “10x” software product innovations you have experienced?
The rr debugger [1]. I desperately missed it after switching to a mac laptop -- to the point that I now have an older linux system which exists primarily to run rr.

(I've heard that Pernosco [2] - partly built on rr, AFAIK - is even more revolutionary, but haven't yet tried it myself)

[1] https://rr-project.org/ [2] https://pernos.co/

ihnorton··on Symbolics.jl: A Modern Computer Algebra System for a Modern Language
I won’t comment on the relative numbers because there are top-notch developers in many language communities.

I think the more important point is that Julia has attracted enough first-rate people to self-sustainably build out an ecosystem — and even more keep joining. Several aspects of Julia’s design and core tooling interact to provide compounding leverage to this group. I think it’s a similar situation to the development of the NumPy ecosystem where standardizing on a common array data structure and API led to an explosion of interoperable libraries. Julia arguably takes that a step further by allowing any code to interoperate with high performance and fluent APIs. Julia’s performance characteristics also reduce the barrier to entry because people can make deep contributions throughout the ecosystem without needing to develop low-level programming expertise on top of their domain-specific knowledge.

ihnorton··on Using JAX to Accelerate Research
> can Theano drive JAX?

Yes, that is in active development: https://pymc-devs.medium.com/the-future-of-pymc3-or-theano-i...

ihnorton··on Pyston v2: Faster Python
10x reflects both experience and expert recommendations. You may recognize the author [1]:

> Nowadays, my rule of thumb for pandas is that you should have 5 to 10 times as much RAM as the size of your dataset

[1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/

ihnorton··on Bazel Build System Support for LLVM
It’s been a couple years since I looked, but I recall LLVM’s CMake being among the cleanest I’ve ever seen.
ihnorton··on Pyston v2: Faster Python
In the context of pandas, 3 GB of (raw, uncompressed) data could easily require 30 GB of RAM, and that kind of overhead adds up quickly.
ihnorton··on Pyston v2: Faster Python
> A very-low-overhead JIT using DynASM

Interesting. DynASM [1] is the template assembler used in LuaJIT, so it sounds like they might be JIT'ing CPython bytecode. IIRC this is also what the first version of Pyston did. I'm curious how this is working out, both implementation and performance-wise compared to LLVM (used in Pyston v1). That could mean there is a lot of performance still on the table, at least for some kinds of code, but also a big complexity jump to get further gains.

[1] https://luajit.org/dynasm.html

ihnorton··on Ninja: A simple way to do builds
A small warning for code with many CMake ExternalProject dependencies: the defaults for cmake+ninja will recursively run `ninja -jN` for each level of the dependency tree, which can spawn a very large number of compile jobs.

There is a proposal to add jobserver support for global process count management [1], but it has not been merged for philosophical reasons (IIRC). The CMake folks maintain a soft fork with this patch applied, available in the `ninja` package on PyPI and probably elsewhere.

[1] https://github.com/ninja-build/ninja/issues/1139

ihnorton··on Attempts to make Python fast
The principal developer of Pyston commented on the JavaScript comparison recently [1]:

> This is a common view but I've never heard it from someone who has tried to optimize Python. Personally I think that Python is as much more dynamic than JavaScript as JavaScript is than C.

[1] https://news.ycombinator.com/item?id=23247618

ihnorton··on Shasqi (YC W15) aims to make chemotherapy more powerful and less toxic
If you are interested in medical devices, at least in the US there are hospital-based postdoc positions which will hire some engineering PhDs even w/out specifically medical thesis focus (as you are coming from ferroelectrics: potentially MRI or robotics research). In such a position there is a lot of opportunity to attend medical lectures, and also ideally to shadow MDs in clinic, observe procedures, and sit in on case reviews.

There are also various 1-2 year MS degree programs, often called "MS in biomedical sciences", which can function as a bridge into medical careers. Some are designed to prepare for MD or other professional school, while others have a focus on broader background (variety of subjects like anatomy, physiology, and pharmacology).

ihnorton··on What Working At Stripe Has Been Like
I've heard several times about the Stripe "Library" and "written culture", both in previous patio11 posts [1] and elsewhere. I understand part of that comes from the top and screening for writers. But I'm also curious about the basic mechanics of the library; wordpress? big ol' mediawiki? git repo of markdown docs? custom software?

How has the library process (if any) itself changed over time?

I've worked at non-startup BigCo/healthcare with formalized procedural docs, (technical) writers, change management, etc., and often they were minimal, boring, impersonal, frequently outdated -- and rarely read. I'm imagining (for better or worse) this is something a little more like a shared internal blog archive?

[1] https://www.kalzumeus.com/2019/3/18/two-years-at-stripe/

ihnorton··on Array Programming with NumPy
NumPy’s direct predecessor Numeric was released in 1995.
ihnorton··on Draft of OCaml Scientific Computing book
> Indexing, slicing, and broadcasting are three fundamental functions to manipulate multidimensional arrays.

...

> Indexing and slicing is arguably the most important function in any numerical library.

These statements are undoubtedly true. The first question any practitioner familiar with other systems will ask is: what does basic arithmetic, array manipulation, and linear algebra look like?

But from what I can tell on a very quick skim, that question isn't really answered until the section starting with these sentences, on page 123. I've noticed this situation every time I look at the Owl docs webpage too, FWIW (have not looked recently though).

I understand the need to be perceived as fully-capable for modern tasks -- and that's fine for a 2-4 page set of teaser examples up front -- but I think this book would become much more approachable if the basic mechanics of doing math were presented first.

ihnorton··on TileDB 2.0 and the Future of Data Science
Fixed in the latest image update (tiledb/tiledb:2.0.1). Thanks for the report!
ihnorton··on TileDB 2.0 and the Future of Data Science
There seems to be a problem running the example under python 3.6 used in the docker container. Investigating. Please ping us (tiledb.com for contacts) and I will let you know when the docker image is updated.
ihnorton··on TileDB 2.0 and the Future of Data Science
Here is a quick demo of dataframe i/o: reading to/from a pandas dataframe, and then creating and slicing a sparse array with heterogeneous dimensions (int and string):

https://gist.github.com/ihnorton/1544bb169b4ec8cd385e9c3e756...

ihnorton··on Zig cc: A drop-in replacement for GCC/Clang
There is a semi-maintained (perhaps more accurately “occasionally resurrected”) C backend for LLVM: https://github.com/JuliaComputing/llvm-cbe
ihnorton··on FDA clears ‘world’s first’ portable, low-cost MRI
The MGH Martinos Center has a very nice open source MRI page with hardware and software instructions to build a functional, low-resolution tabletop MRI (I think this is primarily targeting instructional use at a university level, so requires a fair amountof background):

- https://www.opensourceimaging.org/project/instructional-tabl... - https://tabletop.martinos.org/index.php/Main_Page

ihnorton··on Femtolisp: A lightweight, robust, scheme-like Lisp implementation
https://github.com/JuliaLang/julia/pull/17108 only implements the REPL part but it’s a start.

“Downloadable Windows LA REPL” was a criteria Julia met within ~6-8 months of public release, and it was one of the reasons I got involved even though I only ever use(d) windows at work (...mostly for scientific instrumentation which shipped with and only supported windows).

This is slowly changing, but IMO windows support is still table stakes for a scientific computing environment, and seeing windows support meant both that I could use it where I most needed it, and signaled that Julia had potential for real traction (not abandoning >70% of users out of the gate). I think windows support is also one of the reasons Python/NumPy got to be where it is — it wasn’t always perfect, but they at least cared about windows, more so than most other scripting languages in the mid-2000s. Ditto CMake.

ihnorton··on Femtolisp: A lightweight, robust, scheme-like Lisp implementation
Is there a Lisp I can download, for Windows, that provides a linear algebra problem solving environment* out of the box or from a package manager? Ideally linked to a reasonably optimized BLAS implementation.

* this is intentionally under-specified, but to me it means some nontrivial subset of matlab/numpy functionality

ihnorton··on Ask HN: Which VM is the easiest to target(for a compiler)?
Have a look at Luje: https://github.com/davidgiven/luje
ihnorton··on Bazel Release 1.0
I hope that this 1.0 status will prompt the Qt people to take a closer look at Bazel, but that's probably strongly conditioned on the quality of Windows support. Qt Co. have indicated that CMake is the most likely replacement for qmake in Qt 6, which would be a lateral change at best.

Is anyone using Bazel to ship cross-platform GUI applications?

ihnorton··on Ask HN: How do you handle/maintain local Python environments?
> What do these do. Why are they adding an alternative gettime implementation for MacOS? Why are they patching a bunch of test files to make them pass?

The gettime patch is to support macOS SDK < 10.12, as it says in the patch itself. From a quick look, some of the test patches appear to be for Python 2 compatibility. Every linux distribution and system integrator has a patch list just like that for complicated software. If a particular patch concerns you, I would suggest opening an issue on their bug tracker to ask for an explanation.

> It's unfortunately one we ran into recently while migrating a legacy system off conda, but it's not the first and I doubt it will be the last.

Unclear what that has to do with conda. The same package is available on PyPI (source only), and the line you highlighted almost certainly exists for some other build target than conda, where the compilers will automatically find numpy includes in CONDA_PREFIX without any extra effort.

ihnorton··on Ask HN: How do you handle/maintain local Python environments?
> Not a fan of Conda, the cli is terrible, their unofficial builds are opaque and in my experience the very few packages that are not available on pypi are rarely worth installing (usually the worst kind of "research code"[1]).

The CLI used to be slow, but since 4.6 the performance is reasonable -- still slower than some non-Python package managers, but in my experience no longer so slow as to be aggravating (< 10s install now, whereas previously >30-60s in the past just solving the environment)

> their unofficial builds are opaque

I'm not entirely sure what builds you are referring to, but conda-forge [1] is completely open source and community-driven, and Anaconda recipes are in a github organization called AnacondaRecipes [2]. If you are talking specifically about the Python interpreter builds for the Anaconda distribution, I believe the build recipes are here [3] (although I don't know that for a fact).

> my experience the very few packages that are not available on pypi are rarely worth installing

Depending on your field, platform, and dependencies, that is certainly believable. However, several projects such as Apache Arrow and Rapids have dropped support for PyPI wheels [4] because the engineering cost of wheel-building on linux is prohibitive, and conda is preferable on windows for a variety of reasons.

> usually the worst kind of "research code"[1]

That's an oddly-specific complaint. Anyway, given the variety of environments where people run `setup.py`, that kind of thing is usually explained by "someone put in a PR to make it build somewhere." It might be surprising if you've never built or packaged python extensions with native dependencies before, but I'm not sure it's a particular indicator of quality.

[1] https://conda-forge.org/ [2] https://github.com/AnacondaRecipes [3] https://github.com/AnacondaRecipes/python-feedstock [4] https://medium.com/rapids-ai/rapids-0-7-release-drops-pip-pa...

ihnorton··on Ask HN: JIT Compiler Landscape of 2019?
- if you are willing to do LLVM, you might look at Julia as (in some sense) a Lispy LLVM frontend. See: https://www.youtube.com/watch?v=BBUrQId0HhQ

- re LuaJIT, take a look at the RaptorJIT fork: https://github.com/raptorjit/raptorjit ... They have (for now) stripped out all targets except x86_64. lukego has built some neat tooling for understanding traces, and folks are working on a straight C interpreter to reduce the porting effort for new architectures.

- also re: LuaJIT, http://terralang.org/ might be of interest

- JVM is still around! Has some very interesting development around the Graal/Truffle combination.

ihnorton··on Ask HN: Why is Google in love with purposely-impaired languages like Go, Dart?
We have an entire profession built around the often-vain attempt to clarify and resolve ambiguities in the use of those 70k+ words: law.
ihnorton··on On me, and the Media Lab
How does anyone, really?

https://www.nytco.com/board-of-directors/

← PreviousPage 2 of 15Next →