Hacker News Reader

Comment by old-gregg | Hacker News Reader

Sure... and it means that Swift and Clojure are just as useless as Julia for numerous use cases where start-up times matter, like piping CLI commands together, in Julia's case you're looking at pathetic 3 execs per second.

Moreover, even in some server-side applications it's super neat to have the luxury to spawn a new process to service certain requests, not having to worry about memory leaks. It's a perfect "API" which allows multiple languages to interact together.

It always bothered me when people dismiss the start-up time by adding "just" in front of it. We're not talking about web frameworks, these are _general purpose_ programming languages, and horrendous start-up time automatically disqualifies them from being general purpose and places them into a niche category, in my humble opinion.

View on HN

> it means that Swift and Clojure are just as useless as Julia for numerous use cases where start-up times matter

Or that different languages require different approaches, and you can't translate 1:1 between technologies. My Mac starts up 27x slower than my C=64, but that doesn't make it useless for any task that requires turning it on.

> We're not talking about web frameworks, these are _general purpose_ programming languages

Are we? I've never written a line of Julia in my life but the impression that I get is that this isn't intended to be a general-purpose language:

> Of course, one might argue that Julia is not intended to be a general-purpose programming language, but a language for numerical computing.

> As has been pointed out elsewhere “Base APIs outside of the niche Julia targets often don’t make sense” and the general-purpose APIs are somewhat limited.

The Julia webpage has 6 tabs listing features, and 5 of them are about numerics. The 6th says "General Purpose", but it's mostly about FFI.

Julia is intended to be (and already is) a general purpose programming language. What it does not try to be is a language that tries to be good for every purpose. Python for example is a general purpose programming language but you wouldn't use it to write kernel modules, and likewise you wouldn't use C to write your simple website (even though you obviously can). A language that tries to be everything to everyone will either be too bloated and complex, or lack the included batteries for pretty much anything.

Even if we don't use technicalities, Julia is a very powerful language that allows people to extend it for purposes it was not built for. If it didn't support JSON, you could write a JSON type as integrated and fast as what the stdlib offers, it's metaprogramming/multiple dispatch paradigm allows for easily creating frameworks for many purposes and while numerical processing gets special attention the language will still outperform in terms of speed most dynamic languages in any domain.

And I do think short scripts are within Julia's main targets and are merely victims of the fact that the language is still too young and the battles the devs chose to fight within that limited time (and either AoT options or an interpreter that runs code while it's compiling could solve it really well for example, but both would require a lot of time and work which could be used for other features).

Are you sure?

https://micropython.org/

https://facil.io/

The poster said "(even though you obviously can)".

People will try all sorts of weird language experiments, so it's no surprise those libraries exist. But do you really expect a significant number of web developers to shift over to C, just because it's technically possible?

How do think many IoT devices serve their Admin UI?

I have no idea, and I don't really see how relevant this is to the original remark that you responded to.

They are written in C unless they are powerfull enough to hold something else, like Cisco does with Chicken Scheme on their routers.

https://www.micrium.com/rtos/tcpip/

Quoting the relevant quote "you wouldn't use C to write your simple website", except that is exacly what EE does when on device memory is measured in KB.

C++ would be safer, but the C89 culture reigns in such domains.

I did generalize above, but when I said "simple website" I really meant the usual simple website one the web, not firmware/embedded stuff. I kinda said that because I already wrote one in C including a basic CGI library, as well as a mid-sized one with Rails, and the difference clearly falls into "C doesn't try to be good at website creation, but you can obviously do it" (and in some occasions you just have to). I'd also add that Roller Coaster Tycoon was written in Assembly, but it's not a language that is particularly good for game dev in general (but still a completely general purpose programming language).

Micropython is not really python (which by definition is defined by PEP and CPython implementation), it's a language largely similar to python 3 since you can't just pick a python program or library and run on it at all times, but it's not really a relevant discussion (and it's nice to have multiple variants of a language you like that are good for multiple purposes, which solves the bloat problem as long as you use only one of them at a time).

Regardless, I hope Julia gets to the point that you can target as many places as some of those languages even if it's not nearly the best in each domain (like WASM, embedded, OS stuff, shared libraries).

Actually Assembly is how most 8 and 16 bit titles were written on, higher level languages were the Unity from 80-90's gamedev scene, Roller Coster Tycoon isn't alone.

I get your point, though.

Not necessarily. It's entirely possible to keep a daemon running with the loaded runtime to speed up start time. I'm not arguing if and whether these languages are suitable for scripting rather startup time alone need not be a disqualifying factor. If you evaluate holistically and arrive at a language you'll always have one thing or the other that is not as what you want.

A few years ago, as a hobby project, I designed a programming language and wrote an interpreter for it in Java. (I've never released it or shared it with anyone; I was never quite happy with it and eventually moved on to other things.) As you can imagine, an interpreted language with the interpreter written in Java is going to be rather slow to start.

So, I implemented exactly the solution you describe here. I made my interpreter run as a daemon and listen for requests on a Unix domain socket (with a custom binary protocol). I then wrote a client in C, which opened that Unix domain socket and sent it a script to run. The C program also redirected stdin/stdout/stderr to/from the interpreter through the Unix domain socket, if connected to a tty it could switch to/from raw mode, it notified the server if certain signals occurred, etc. Using the interpreter this way, startup time was a lot faster. This approach can be applied to any language which supports runtime evaluation of code.

A long-running background interpreter carries significant security risks, though. A process that lacks most linux capabilities could use it to regain access to things it shouldn't have access to. A process could interpose itself and get access to data and code it shouldn't. If there are multiple users involved, one could bypass ACLs and similar. Essentially, you're removing a significant proportion of the security guarantees the OS provides around isolated processes.

Some of the concerns you raise can be addressed. For example, make the daemon and socket per-user and give the socket 600 permissions. Also, the SCM_CREDENTIALS message (on Linux) or LOCAL_PEERCRED/getpeereid (macOS/*BSD) can be used to validate the caller has expected UID.

The issue with missing-normal-capability processes is more difficult to address. One possibility, at least on Linux, would be to get the pid from SCM_CREDENTIALS message, and then read /proc/$PID/status to check capability bits. It could default to denying access to processes with less capabilities than it itself has.

(SCM_SECURITY can be used to pass the SELinux security label from client to server, which could also be used as a security measure; maybe the server could refuse access to processes running with a different label, or have a whitelist of allowed labels; if SELinux is being used to sandbox a process, that would prevent it from accessing the background interpreter unless that was explicitly allowed by whitelisting.)

> like piping CLI commands together, in Julia's case you're looking at pathetic 3 execs per second.

If you want to build CLIs without the JIT overhead, you can compile your Julia code: https://github.com/JuliaLang/PackageCompiler.jl

How big are the resulting binaries these days? It’s been some time since I tried this, but I decided that the size of the compiled code didn’t fit my use case (a serverless function IIRC)

Horrendous is the answer :)

It is still crazy big, because it still has to include the LLVM compiler in it.

It doesn't work very well. I got SEGFAULTs every time I tried; filed a bug report; no resolution. So, while it does exist, it's not a high priority for the Julia team, so I wouldn't rely on it.

I had the exact same experience when I tried over a year ago. The package was eventually closed IIRC, but a recent news letter claimed to have fixed it.

It's getting better. See how I managed to compile a bunch of data science packages https://discourse.julialang.org/t/201908-compiling-a-data-sc...

> it means that Swift and Clojure are just as useless as Julia for numerous use cases where start-up times matter

This is not true for Julia, you can simply precompile the code: https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4...

Julia's default model is that script functions are only JIT compiled when run (just like the JVM, etc.), but you can just as easily force the compile and stick it in an image.

> you can just as easily force the compiler

Have you actually tried the instructions in there? I did, to get a better time for julia in the n-body programming languages shootout https://benchmarksgame-team.pages.debian.net/benchmarksgame/... (I am the current leader among julia implementations). Suffice it to say, it did not work. I mean i'm not a brilliant programmer or anything, but I would hardly say it's "just as easily".

I am a Julia noob and managed to use PackageCompiler.jl for a package I wrote and it just worked out of the box.

I think the only non-stdlib package I used was StaticArrays, but I was able to use the happy path described in the PackageCompiler docs and got it working in an hour or so max. This was probably in March or April, so relatively recent.

As you might be able to see I ran it on 25 Jun 2019

I didn't have any non-stdlib packages at all!!

I'll give it another shot.

Totally, this is why have the right tool for the job approach. I would not start up a Spark cluster for processing 100MB of data I also would not start to write a CLI tool in Clojure (even though it is one of my favorite languages).

I would write a CLI in Clojure if it were a reasonable option. Something like Gambit or Chicken but for Clojure would be a dream. I guess there’s Ferret, so maybe I should give that a try.

Also checkout Joker language. It's an interpreter that implements a subset of Clojure.

You got me thinking about that word, CLI. I know that usually means a Unix shell, but when I just run my bespoke file conversion tool from CIDER REPL, that interface is also a command line one, what else is it?

What a CLI wants to be when it grows up.

Check the REPL experience from Lisp Machines and Xerox PARC Lisp/Mesa/Smalltalk workstations.

Think of Swift Playgrounds or Jupiter notebooks, done in late 70's hardware.

> Sure... and it means that Swift and Clojure are just as useless as Julia for numerous use cases where start-up times matter, like piping CLI commands together, in Julia's case you're looking at pathetic 3 execs per second.

The REPL is a tad slow but compiled Swift binaries have virtually no startup time overhead.

Eh? Swift surely doesn't fall in this category, as it's meant to be used compiled, not from a REPL, and doesn't have a massive heavy runtime like Clojure. I refuse to believe that a Swift CLI compiled to native code as intended has a startup time problem.

Depends on how long the piped commands are going to run for... if they run for minutes doing stuff, who cares about the extra startup time?

>Clojure are just as useless as Julia for numerous use cases where start-up times matter,

Clojure can be targeted for graal afaik. Not sure how well it works though.

I’ve used it a few times in the past experimentally. It definitely improved startup time. It was hard to set up though, so I generally didn't find it useful.

I’ve become disenchanted with clojure over the last couple years, so I haven’t tried anything recently.

> I’ve become disenchanted with clojure over the last couple years

I've been trying to get into it last 6 months :p. Now I am Wondering why you say this.

try the following

`date 1>&2 | sleep 1 | date 1>&2 | sleep 1 | date 1>&2`

you'll actually see the three date commands output immediately so the three julia processes would actually launch in parallel if they were in a pipe

Reply on news.ycombinator.com