HNHacker News
TopNewBestAskShowJobs

zanellia

160 karma · joined September 22, 2020

PhD in numerical optimization and control

https://github.com/zanellia

submissionscomments
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
That's quite cool. Maybe the whole thing can be rewritten in Julia too at some point. I just know too little about Julia to judge.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Python to ASM would actually be really cool and would guarantee performance gains for small matrices, but it would require quite some implementation effort. Not sure about WASM.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
I'd say that's possible in principle - definitely not there at the moment though (and not even planned).
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
For what it counts, I have developed code for this kind of applications exclusively in C for ~5 years (let's say 20% of my working time). I still think that debugging a segfault that you could have avoided is not very productive and that motivated me to look into possible alternatives.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Not sure how easy it would be to make the code generated by Pythran standalone, i.e., no dependency on the Python runtime library. Any Pythran expert? :)
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Thanks for the suggestion. Until now it's been a lot of discussion with friends and colleagues and much less actual collaboration on code writing - I might have drifted into bad practices.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
I would disagree on "easier" :) Ever spent half a day debugging a segfault?
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Fair enough - it's blurred I'd say. I see C as a lower-level, and yet still high-level, language, if compared to Python :)
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Personally, I think Julia is great - just don't know it well enough to write a package that takes Julia ASTs and generate C code from them :) There could totally be a Julia implementation of the main idea behind prometeo (Julia per se does not solve the problem that prometeo aims at solving).
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
hahaha thanks!
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
fair enough - could not cram "embedded" into it :)
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
"A program that translates between high-level languages is usually called a source-to-source compiler or transpiler" from https://en.wikipedia.org/wiki/Compiler.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
I think many people who have at least once first prototyped a numerical algorithm in a high-level language (say Python, Julia, MATLAB?) and then implemented it in C, can relate to the experience of transitioning from error messages of the type: "dimension mismatch for XYZ" to "segmentation fault". That's in my opinion a strong motivation to build tools that can automate certain parts of the development process.

Writing C code directly is as a good option, as long as your code is not too complex to develop, maintain and extend.

And, again, Python here is intended to be the host language for an embedded domain specific language that gets compiled into C. It does not need to be efficient it needs to be expressive and easy to analyse and transpile.

zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
yep, that's right.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
The point of prometeo is not to obtain a "performant Python". Python is used merely as a host language for an embedded domain specific language. You could do the same thing with any other language with a mature library for AST analysis :)
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
I think most HPC people would disagree with this statement. State-of-the-art HPC code is still written in ASM (see e.g., https://github.com/xianyi/OpenBLAS) [that's what Intel is doing too]
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
fair enough :p I might change it in the future.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Unfortunately, I think that writing a transpiler for general Python programs might be rather difficult without resorting to approaches used, e.g., in Cython/Nuitka. Among other things, computing the worst-case heap usage could be quite cumbersome/computationally heavy for a general program without "constraints". I'd be happy to hear what others think about the topic though.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
> Soo... it takes Python syntax and produces a C program, with no links back to Python - is that right? It uses a strict subset of Python, so that Prometeo programs are valid Python, but not necessarily the opposite. Is that fair?

yep

> Do you envisage this being a conduit for tight loop optimisation in Python? Or is it rather "you'd like a C program but can't write C good"?

There are already plenty of options for calling high-performance libraries from Python. Now 1) interpreting Python programs that use, e.g., NumPy, can be slow. 2) Compiling these programs using, e.g., Cython or Nuitka, can speed up the code across calls to high-performance libraries, but the resulting code will still rely on the Python runtime library, which can be slow/unreliable in an embedded context.

Coming to the second part of the question, writing C code directly is definitely an option, but, after doing a bit of that, I realized how tedious/error prone it is to develop/maintain/extend relatively complex code bases for embedded scientific computing (e.g. this one https://github.com/acados/acados). Or, to put it as Bjarne Stroustroup once said "fiddling with machine addresses and memory is rather unpleasant and not very productive". The good news seemed to be that many of the code structures necessary to write that type of code are rather repetitive and can hopefully be generated automatically to some extent.

> And if the former, how do you compare to Nuitka and Cython? I read your README but couldn't quite make sense of it :)

This table (from the README) shows some computation times for Nuitka, prometeo, Python and PyPy.

CPU times in [s]:

Python 3.7 (CPython) : 11.787 Nuitka : 10.039 PyPy: 1.78 prometeo : 0.657

Other than performance, the main difference is, again, the runtime library dependency.

zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Right, that's indeed the main reason I could not simply use Cython or Nuitka (or Julia?). The Python runtime library will do all kinds of non real-time/embedded friendly operations such as garbage collections, memory allocation/de-allocation and so on, in the background.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
For the time being, it targets single core/thread applications only.
zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
That's correct. I'd say one of the fundamental differences between the two lies in the fact that the code generated by prometeo does not depend on a runtime library (which is somewhat fundamental for embedded applications, e.g., embedded optimization). From prometeo's README:

Finally, although it does not use Python as source language, we should mention that Julia too is just-in-time (and partially ahead-of-time) compiled into LLVM code. The emitted LLVM code relies however on the Julia runtime library such that considerations similar to the one made for Cython and Nuitka apply.

zanellia··on Show HN: Prometeo – a Python-to-C transpiler for high-performance computing
Hi all,

prometeo is an experimental modeling tool for embedded high-performance computing. prometeo provides a domain specific language (DSL) based on a subset of the Python language that allows one to conveniently write scientific computing programs in a high-level language (Python itself) that can be transpiled to high-performance self-contained C code easily deployable on embedded devices.

The package is still rather experimental, but I hope this concept could help making the development of software for high-performance computing (especially for embedded applications) a little easier.

What do you think of it? Looking forward to receiving comments/suggestions/criticism :)

zanellia··on Attempts to make Python fast
In my opinion there is some potential there. Especially exploiting the increasing integration of typing-oriented features (i.e. type annotations) and the interest in using those to carry out static analysis (e.g. in mypy, but also Facebook's Pyre and Microsoft's Pyright and many other), it might be possible to speed up execution times a bit. This is especially true if we restrict the attention to a restricted subset of Python as, e.g., within domain specific languages. It might not make sense to entirely reverse engineer a language that was designed to be duck-typed into a statically typed one. However, for some domain specific applications I find performance oriented static analysis an interesting tool.

To make it more concrete, here is an experimental DSL for embedded high-performance computing that uses static analysis and source-to-source (Python-to-C, actually) code transformation: https://github.com/zanellia/prometeo.

zanellia··on Haskell language features and standard libraries in pure Python
That's quite interesting. I guess one could try to do something even cleaner than this https://github.com/dry-python/classes using type hints only and code transformation based on Python ASTs (see e.g. https://github.com/berkerpeksag/astor). I have done something similar for (typed) Python-to-C transpilation for fun (https://github.com/zanellia/prometeo), but, now that I think about it, it could be used to implement typed polymorphism for Python classes quite easily.
zanellia··on Haskell language features and standard libraries in pure Python
Indeed, I guess much of the type annotation stuff would be much easier with type hints from Python 3.6+ (https://docs.python.org/3/library/typing.html), plus it might add some level of compatibility with recently developed static analysis tools like mypy, pyright, pyre.
← PreviousPage 2 of 2