But I don't put genetic algorithms in the same bucket. Genetic algorithms are a different breed of optimization algorithm compared to neural nets and gradient descent which is what the modern crop of AI is basically all about.
1 karma · joined December 4, 2019
But I don't put genetic algorithms in the same bucket. Genetic algorithms are a different breed of optimization algorithm compared to neural nets and gradient descent which is what the modern crop of AI is basically all about.
People see patterns where there are none. I think this is fundamentally why humans fail at statistics. If every fiber of your being wants to see patterns then you will see patterns. Probably why people hallucinate when in sensory deprivation tanks as well. The brain will make up patterns just so it can continue to see them.
The paragraph right after follows up with the statistical failure that pattern seeking leads to
> They most often ranked no interview last. In other words, a majority felt they would rather base their predictions on an interview they knew to be random than to have to base their predictions on background information alone.
So people would rather do busy work in order to continue to satisfy established pattern seeking habits than figure out a better way.
For an example of the kind of logical problem optimizers solve you can take a look at: https://github.com/google/souper.
So I don't see how you can take the current neural nets and get them to design a more efficient CPU architecture or a better JIT.
The fact that you don't know if "collatz(n)" is always 1 or not means you are limited by the compiler's termination limit for inling and simplifying certain expressions. So the fact that the compiler terminates indeed has a lot of bearing on what kinds of optimizations are allowed and derivable in the absence of an expressive enough type system.
Some people might object to that example but the fact remains that logic and type systems are intimately tied to program analysis and optimization. In the domain of pointer alias analysis there is something known as separation logic which allows for statically analyzing aliasing relations. Now if this information is encoded as part of the type system then certain mechanical derivations about aliasing become much simpler which leads to more optimization opportunities. Rust's type system I believe is more related to affine logic but similar arguments still apply. You can not express the invariants that can be expressed with Rust's type system in C++. Actually I take that back. You could in theory encode affine or separation logic with the template system and then encode your program using those templates but that's basically the same work as re-implementing Rust in C++. This means Rust's type system allows for program optimizations that would not be possible otherwise. Similar to how knowing "collatz(n)" always being 1 allows you to elide that condition entirely. No amount of inlining and simplifying will allow you to perform that same optimization.
Oh, nm. I can edit.
Information for optimization passes doesn't come for free and as terminating programs compilers can only fill-in so many holes before the programmer must fill-in the rest or barring that pay a performance penalty which is what happens in dynamic languages even with a JIT.
Instead you made a bunch of disparaging remarks, trivialized large swaths of CS theory, and failed to see the actual point of my comment in connecting type annotations as mechanically verifiable invariants/assumptions that can be used in optimization pipelines.
Seems like all I need to do is bone up on alias analysis, JITing, and inlining and I'm good to go.
Guess I can also solve all pointer aliasing problems by just adding "if p != q {...}".
Compilers, optimizations, type systems, and proof systems are all very closely related and if you want correct optimizations then you must annotate your code with types that the compiler can verify and use as assumptions in an optimization pipeline. These things are not easy to engineer. Saying just JIT it doesn't make much sense.
So it's more than just sufficient cleverness. It is also about the language providing you the tools to express certain invariants that the compiler can mechanically verify and then use in an optimization pipeline.
Edit: This is mostly out of context now and can be ignored.
I'm not condoning what this guy did but the reason he was able to do all of that is not because he was being shady (although from a specific perspective he was being shady I think it is a matter of degrees when it comes to this stuff). It is because the system is set up to enable exactly this kind of behavior and he just happened to overstep a few too many boundaries whereas the people that have been playing this game for a while have a much more measured approach.
So instead of jumping on the bandwagon and burning this guy maybe we should reflect on how the ecosystem operates as a whole and whether the rules are slightly rigged. He'll have his day in court but if the rest of us don't learn from the experience then not really sure if all the wasted bits on the matter are worth it in the end.
Why is it OK when VCs do it and not OK when a rank-and-file engineer does it? I think you too are focusing on the wrong things. The startup and VC ecosystem is all sorts of retarded. This case blowing up just gives you a small glimpse into the startup shell game.
First to know what smart looks like you have to be smart yourself before you can hire more smart people. Second you have to be really cognizant to not hire more people like yourself because not all smart looks the same way. Third whatever position you're hiring for you need to understand well enough to recognize the same kind of skill in others but this kinda conflicts with the second point because expertise always introduces bias. Fourth you have to be ruthless when it comes to firing people because every so often you will make a mistake and hire a bozo that looked smart and people really don't like firing people.
Basically all the cards are stacked against you and what I've observed in most software organizations is that you'll be lucky if you manage to make 10% of your workforce satisfy the "smart" metric.
> The first principle is that you must not fool yourself — and you are the easiest person to fool.
I'm tired of these state of affairs. All of the above can be done in real programming languages, with real syntax. There is no need for yet another external DSL when an embedded DSL in the form of a library will suffice.
Cell division is an unstable positive loop that requires dampening with negative feedback loops at higher organization levels. When those negative feedback loops break down you get an unstable positive loop in the form of cancer.
So you see how having cell division in a multi-cellular context can not be "cured" of cancer? You'd need to stop cell division and somehow figure out a way around entropy. Cells divide to renew/reset and shed accumulated environmental damage. So no matter what you do as long as you have a system organized around unstable processes like this you will always have a control problem when the negative control mechanisms stop working.