However, those isolated examples rarely if ever translated to high performance in real-world projects. The same issue has an experience report of the travails of IBM's San Francisco project. Performance was a huge issue that to the best of my knowledge they never fully resolved.
More in my article "Jitterdämmerung": http://blog.metaobject.com/2015/10/jitterdammerung.html
So he debugged some more and figured out that he misunderstood some of the inner workings of how vectors copy data and also that he did not understand the threading library he was using properly. He then fixed those two things. After this further exercise he reduced the difference to factor of 4. However he was never able to work out why my code was 4 times as fast as his C++ and abandoned it.
I know for sure that with appropriate expertise the C++ could probably be made to go perhaps twice as fast as my Groovy code. But the point is, none of the supposed benefits come automatically regardless what language you are using. And unless you flip over to GPU or FPGA accelerated methods, the final outcome is well and truly in the same ballpark anyway.
But all this is to say that "rarely translated" might be true at the for applications that are completely in the high performance domain. But for all the applications where the high performance code is in niches at the edge and there simply aren't resources or expertise to fully tune the native implementation ... I think it's translated all the time.
Dynamic languages can be fast by being well designed and simple (Wren) or highly optimized (JS) or both (LuaJIT). There’s also the experimental GraalVM, but this is definitely the exception and not the rule.
Writing performant C++ is actually not hard if you have rudimentary C++ knowledge. That said, rudimentary C++ knowledge is a lot more expensive than rudimentary knowledge of other languages. But the options aren’t just dynamic languages vs C++; there’s a ton of middle ground with VM languages like Java and C# and native languages like Go. The first two aren’t much harder than dynamic languages and I find Go easier than any dynamic language I’ve used to date (I’m a professional python developer). But all of these languages are on the order of half of C’s performance and 100X the speed of CPython or Ruby and 10X of JS.
Oh well this VM meme is getting old,
https://www.ptc.com/en/products/developer-tools/perc
https://docs.microsoft.com/en-us/dotnet/framework/net-native...
> But for all the applications where the high performance code is in niches at the edge and there simply aren't resources or expertise to fully tune the native implementation
It's interesting you say this, because in my experience it's the JVM which requires absurd amounts of tuning and native programs which are much more consistent. The proper and easier way that native programs are written lends itself to fairly respectable performance, mostly because the object and stack model of say C or C++ is so much friendlier to the CPU than in most dynamic languages.
In general, for all that I hear statements along this line, I've only twice seen code to back it up, and the C was so de-optimized from the OCaml version that I suspect it was intentional - the author (same for each) was a consultant for functional languages, and in one case switched the C inner loop to use indirect calls for every iteration and in the other switched the hash function between the C and functional comparison.
Also worth calling out Go, which is rather unique in that it has stack semantics but it also has a garbage collector, so it’s kind of the best of both worlds in terms of ease of writing correct, performant code.
It's a rigid term:
https://en.wikipedia.org/wiki/Tracing_garbage_collection#Pre...
perf shows how much time does GC eat, and that's quite a lot. Thus in the majority of benchmarks go lags behind java or on par with it at best.
>there is so much less garbage than in other GC languages
That is not true since strings and interfaces are heap allocated thus the only stack allocated objects are numbers and very simple structs (i.e. which contains only numbers), so you would have a lot of garbage unless you are doing a number crunching, which could be easily optimized by inlining and register allocation anyway.
Ah, neat! I learned something. :)
You’re mistaken about only numbers and simple structs being stack allocated. All structs are stack allocated unless they escape, regardless of their contents. Further, arrays and constant-sized slices may also be stack allocated. I’m also pretty sure interfaces are only heap allocated if they escape; in other words, if you put a value in an interface and it doesn’t escape, there shouldn’t be an allocation at all.
Structure could be stack allocated, but any of it's fields would not if there is anything but a number.
A trivial example:
https://segment.com/blog/allocation-efficiency-in-high-perfo...
func main() {
x := 42
fmt.Println(x)
}
./main.go:7: x escapes to heap
BenchmarkEscapeInterface-4 50000000 33.3 ns/op 8 B/op 1 allocs/op
BenchmarkEscapeConcreteValue-4 200000000 9.45 ns/op 0 B/op 0 allocs/op
BenchmarkEscapeConcretePointer-4 100000000 10.0 ns/op 0 B/op 0 allocs/op
BenchmarkEscapeArray-4 50000000 21.3 ns/op 0 B/op 0 allocs/op
BenchmarkEscapeStruct-4 100000000 12.8 ns/op 0 B/op 0 allocs/op
type Inner struct {
Slice []int
String string
Int int
}
type Struct struct {
Int int
String string
Nested Inner
}
func (s Struct) AddThings() int {
return s.Int + len(s.String) + len(s.Nested.Slice) + len(s.Nested.String) +
s.Nested.Int
}
func BenchmarkEscapeStruct(b *testing.B) {
for i := 0; i < b.N; i++ {
s := Struct{
Int: 42,
String: "Hello",
Nested: Inner{
Slice: []int{0, 1, 2},
String: "World!",
Int: 42,
},
}
_ = s.AddThings()
}
}I think this matter is sufficiently resolved. Go trades GC throughput for latency and it doesn't need compaction to get good cache properties because it generates much less garbage than traditional GC-based language implementations.
It does. Any language could do static allocation, go is not different from java here, the problem is that in any real code nearly all your strings and arrays would be dynamic, thus heap allocated, as well as interfaces. Consider also that allocations in Go are much more expensive than in java or haskell.
> Consider also that allocations in Go are much more expensive than in java or haskell.
This is true, but unrelated to cache performance, and it's also not a big deal for the same reason--allocations are rarer in Go.
EDIT:
Consider `[]struct{nested []struct{i int}}`. In Go, this is at most 1 allocation for the outer array and one allocation for each nested array. In Python, C#, Haskell, etc, that's something like one allocation for the outer array, one allocation for each object in the array, one allocation for each nested array in each object, and one allocation for each object in each nested array. This is what I mean when I say Go generates less garbage.
A typical example, yeah. I've said about structs of ints already, it's not a common type unfortunately anywhere beyond number crunching, in which go sucks anyway.
In haskell you could have unboxed array with unboxed records. Check Vector.Unboxed.
Yeah, but you were wrong (you said other kinds of structs would escape to the heap). The innermost struct could have a string member and a `*HeapData` member; it wouldn't matter. The difference in number of allocations between Go and others would remain the same. The difference isn't driven by the leaves, it's driven by number of nodes in the object graph; the deeper or wider the tree, the better Go performs relative to other GC languages.
> In haskell you could have unboxed array with unboxed records. Check Vector.Unboxed.
For sure, but in Go "unboxed" is the default (i.e., common, idiomatic); in Haskell it's an optimization.
https://days2011.scala-lang.org/sites/days2011/files/ws3-1-H...
If you add in cases where simd, software prefetching, or memory access batching help, the difference will only grow.
Then I moved to Google, working on the indexing system and properly learned C++. I've since worked on several projects for several companies where the annual cost for compute time is easily 200 times my salary. These are systems where it pays to extensively profile both CPU and I/O and retain C++ optimization experts.
I still see the appeal of platform-independent binaries and dynamic optimization, but you really need the startup time and optimization cost advantages of ahead-of-time compilation. Hopefully the platform independence and dynamic optimization features would also be decoupled from the garbage collector.
HP Research had project dynamo that was basically a tracing JIT emulator for PARisc CPUs that ran on PARisc. It showed that binaries compiled with -O2 could get performace comparable to -O4 through dynamic recompilation.
Ideally, we'd distribute programs in a compact and compressed CPU-independent SSA representation similar to SafeTSA or LLVM bitcode. Installation would AoT-compile the binaries and keep around the SSA form for use by an HP Dynamo-like runtime optimizer. The AoT could compile functions and methods to lists strait-line extended basic blocks with a call to a trampoline loop or other techniques for lightweight instrumentation/tracing of native code. The dynamic optimizer wouldn't need to work with arbitrary machine code, only AoT compiler output. Also, it would never have to disassemble native code, but could always start by gluing together the SSA for each extended basic block in the trace.
A few CPU features could make native tracing and dynamic recompilation extremely light weight. Unfortunately, Intel and ARM are disincentivised to do so, as it would make it easier to migrate off of their intellectual property.
Edge cases. But obviously it's not comparable to something ahead-of-time compiled and hand profiled with PGO and stuff. Although in theory you could put that through JIT too, but it would probably just add overhead and only slow things down.
Pretty cool that it can run faster than AOT compiled C, any benchmarks or posts with more details on this?
They found that most programs ran slightly faster in the JIT. I'm still not sure why this technique didn't catch on.
which is what java touts as being able to do. Unfortunately, i think this sort of speedup is really dependent on the application (and the developer using common idioms that is recognized by the JIT).