HNHacker News
TopNewBestAskShowJobs

havercosine

202 karma · joined May 25, 2022

games, maths, data science, machine learning
submissionscomments
havercosine··on What scientists must know about hardware to write fast code (2020)
Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.
havercosine··on The SR-71 Blackbird Astro-Nav System worked by tracking the stars
The most badass story related to 'using basic tools and a steady hand in a pinch' is Buzz Aldrin pulling out a sextant and sliderule to do some calculations in Gemini XII. The description in the article reads "Aldrin pulled out a sextant and his slide rule and put his MIT doctoral research to work."

More details on the link here: https://www.nasa.gov/feature/gemini-xii-crew-masters-the-cha...

havercosine··on A non-mathematical introduction to Kalman filters for programmers
This is a great post. Another aha! moment I had in understanding Kalman Filter came from a short note by John D. Cook (link: https://www.johndcook.com/blog/applied-kalman-filtering/) . A traditional calculus/differential equations based modelling of systems takes a view that there is no uncertainty in the data and everything is captured by system model. OTOH, data driven estimator will say that "it is all in the data", completely ignoring the model of the physical process that is generating the data. Beauty of Kalman filters is that it combines these two approaches.
havercosine··on Rust vs. Julia in scientific computing
I agree with the observation (most people do not contribute to libraries), but I think the arrow of causality is little different. Python is used as a glue code in scientific computing on top of libraries written in C/C++. The development ergonomics of these languages are intimidating for many people, with both languages having enough footguns. The net result is no-one wants to peek under the hood and see how things are working.

A fresher take on scientific computing like Julia, if it is mainstream, might enable more contributions and in general understandability of the black box algorithms.

havercosine··on Rust vs. Julia in scientific computing
This is spot on. I love Julia, whatever little I have dabbled into it. History of deep learning would have been slightly different if Julia had taken off as the implementation language. Might have encouraged more hackability and understandability. Wrapping our heads around Pytorch source or Jax source is tough.

But this introspection gave me a reality check. In my own work, as much as I love Julia, I fallback to quickly using Python libs. In industry, most of the people's focus is on using someone's already written code. The byproduct is inheriting Python and C/C++ interface mess around dependency management. People needing to write their own algorithms are majority academics and researchers where Julia has flourished.

The incentives of corporate work are setup to build on top of Python's ecosystem which means (sadly) Julia will stay in its niche.

havercosine··on Computerraria: A fully compliant RISC-V computer inside Terraria
This is a super impressive achievement.

- What would you say is your biggest learning from this experiment? - What were the Aha! moments you experienced while learning like this? For example, something you vaguely understood before but is now super clear to you.

havercosine··on The tiny corp raised $5.1M
I don't mind bit of skepticism. But let's see: Comma is actually used by people in their cars to achieve some degree of self driving. He actually shipped something that works! The idea is ambitious as well as commercially interesting: car manufacturers don't reinvent wheel for every component but rely on OEMs to help them out. This was a similar bet. That way it is wildly more successful than hundreds of other undifferentiated yet another social/lending/flavour of the season apps.

It takes guts to put out such bold bets in writing. We've seen many (senior!) tech people sneer at George's at times naive optimism. I actually find the "how hard could be" attitude refreshing against "no no it is complicated you can't do that" gatekeeping. Because otherwise we will end up using big-tech for lack of alternative. It is not the critic who counts and all that..

havercosine··on Why the original transformer figure is wrong, and some other tidbits about LLMs
This is a good intuition. At times it reminds me of old school hand rolled feature engineering used in time series modelling: assuming that the signal is made up of a stationary component and a sine wave. Though haven't managed to mathematically figure out if the two are equivalent.
havercosine··on Writing Python like it's Rust
Wholeheartedly agree. I'm a self taught programmer who programs to get things done rather than just doing programming for fun (I started in ML/data science field). That way, my code often resembles the caricature of these hacky codebases alluded above. And I'm well aware of it.

I often have arguments about utility of Python with my more technically solid engineer friends who keep telling me "Python doesn't scale" etc. And I often come back to the same point you highlighted: is the particular hell worth the result? For 85-90% of the cases the answer is resounding yes.

For nimble teams, startup projects, internal tools : most of the code is often thrown away eventually. Python's beauty is that the language quickly gets out of the way. Everyone then is forced to focus on complexity of the business domain, "does the code solve the business problem". The architecture/scale/speed problems will eventually arrive with Python. But most purist engineers overestimate how soon it will come. At most of the failed startup attempts I was part of, the PMF problem was more pressing to solve than software constraints.

Early days of YouTube, Dropbox, Instagram (and maybe OpenAI too) are big testament to this. I've made my peace these days by not fighting to prove Python is the best language etc. If someone tells me "Rust beats Python" kind of argument: I say I agree, wish them luck and focus on shipping things.

tl:dr; Python is still the best choice to "quickly deliver value and test your business hypothesis".

havercosine··on 100K Context Windows
Has anyone actually tried to talk with Anthropic team for obtaining commercial usage permissions? I'm slightly vary of anything that says "talk to our sales team and explore partnerships" to be a time waste. But happy to change my mind if anyone has gone through the pain and found it worth.
havercosine··on Re-implementing LangChain in 100 lines of code
Personal experience: was using LangChain and its output parsers for getting structured data. It was having a very high error rates (probably prompt was becoming too long and confusing). But it is a just prompt + some parsing logic. Replaced it with straight asking openAI GPT for json that matches some Rust struct / Python data-class. The errors went down and got one extra dependency out from the project. Tried to use its self hosted embeddings but the implementation (strangely) seemed tied to something called Run-house.

Not to belittle the library, but most of it is a very thin wrapper classes that reek of premature abstraction, couple with hit-n-miss docs. At this point, given the hype, it is primarily optimized for cooking up demoes quickly. But not sure if the valuations or production use is justified.

havercosine··on Sold a Story: How Teaching Kids to Read Went So Wrong
Hello :) . Pleasantly surprised and happy to see folks who know about Engelmann and DISTAR here. Random tidbit: I too quit software work to launch experiment in ed-tech two years back. Randomly stumbled on Englemann through a rabbit hole. I'm obsessed with his ideas around sequencing the lessons correctly. Beyond reading, I was interested in using it to improve how we teach Maths. Unsuccessfully tried that mixed with video games. But I agree that Engelmanns research on theory of instruction remains a treasure trove, as it came from his practice and not writing a PhD thesis.

I taught my 4 year old to read English through the book. She's far ahead in reading amongst her peer group (English is not our native language). I know sounds cheesy, but please count me in for exchanging ideas or fixing education using his methods.

havercosine··on Launch HN: Pynecone (YC W23) – Web Apps in Pure Python
I hear your sentiment. As a backend / ML guy who tried to ship projects as a solo dev: I do understand JS/HTML. I understand the programming model of web and still build things the conventional way: templating on server which returns HTML. But lot of time still goes in CSS, to get things exactly working the way we want.

The catch is trying to keep up with Javascript complexity. Javascript/react world is entirely different beast. Spent some time understanding class based React. But then things moved to hooks. I sort of started grokking hooks. But now there is new clamour (not yet mainstream) about something new like server side components or signals or hydration or what not. Same is the case with Svelte. I tried the basic tutorials, understood it. But Svelte kit started talking about 100 different ideas about server pre-rendering components etc. I thought why bother? Jinja2 templating and bit of inline JS looked a better option.

Python devs want to stick to Python because it is still second best language to get something out there, quickly. Except for UI. Currently I'm looking into using HTMX as much as possible. If this thing solves for UI side of it then I'll definitely give it a try.

havercosine··on The Little Learner: A Straight Line to Deep Learning
I second this. Given your background (lisper) maybe do the Little Learner book once it is out + Karpathy's video series. Follow it up by building a slightly complicated application in your favourite domain (text, images, videos, time series).

Also word of advise from my experience (I'm not an expert in DL either): Think of DL field as a game of lego blocks. The ideas in this book / Karpathy's videos are the basic lego blocks: parameterised linear functions, non-linearities, auto-grad, cross-entropy / KL divergence loss and gradient descent. Then there is entire body of more complex legos discovered simply by practice (alchemy!): transformer blocks, layer norm, max-pooling etc. It is impossible to understand how the second kind were obtained from first principles. The trick is to not beat yourself up about the advanced blocks too much but just play around, read up things in papers. Just focus on fundamental blocks.

havercosine··on The Little Learner: A Straight Line to Deep Learning
Well, hard to say if it is a better resource. But it will appeal to those who are in a hurry (and yes, it is okay to be in a hurry). For example, fast.ai first chapter will have you build a dogs vs cats classifier. But how do they build it? By doing `from fastai.vision import *`. They justify it by claiming to use Teaching the whole game approach. If you are a somewhat experienced engineer who already knows Python, what is Jupyter then this approach will get you started quickly. For someone who feels did I spend my time/money but nothing exciting is happening yet, this is a good start. But for some it is crazy and makes them feel even more scared (what is happening with all this imports, how does it work).

My criticism with fast.ai, (I am part time educator), is that this approach is an information overload and poor sequencing. Their comparison with Teach whole game approach is flawed because a game of, say Football, is essentially simple. So you can say just start kicking around. But we don't teach chess this way. It is accepted that you have to spend some time upfront to learn the rules before you can play even simple game. Sure one need not learn castling or en-passant upfront. But you get the drift.

This book (looking at the preview chapters) is going to follow the lego blocks approach or bottoms up approach to build it. For me, this is correct way to teach supervised ML focussed on neural networks and deep learning. We have a problem of too many library plumbers in the ML field currently. People who can piece together library function calls without knowing why it is working. But this house of cards is not sustainable strategy to build AI based application over long term.

Long story short, the book will need patience but that patience will be worth it!

havercosine··on The maze is in the mouse: what ails Google
Hard agree on the interview focus. I think Google's interview culture has optimized heavily for missing forest for the trees, where the person will be extremely good at solving hard but well defined problems. But general customer empathy, releasing things that matter, long term maintenance are not favoured at all. I never worked at Google. But had a senior product manager who joined a startup where I worked. We were building an SMB focussed product for developing world. At one point of time a pissed off customer was on the call and the account manager begged the xoogler to join the call. The guy was astonished at the suggestion and blurted out that "we don't talk with customers, we build". The product doomed, I moved on. But that one incidence has had lasting impression on me.
havercosine··on The Little Learner: A Straight Line to Deep Learning
'Little..' series books are famous for their pedagogical style: small bit of concepts, presented as a dialogue, making you think at each step, in a well thought out order. It will appeal to anyone who wants to build it out themselves, understand the 'big' ideas in the field. Skim through the second chapter to see how this will be done: they define line function in inverted way where slope and intercept are taken as parameters later. Feels weird but exactly the mental model shift we should be doing when thinking about ML.

Read the preface. A very high praise coming from Guy Steele Jr and Peter Norvig.

At the same time, a warning! Scheme based introductions don't appeal to everyone. Some people feel way out of comfort zone in coding with it (which is sad because it is much simpler). Also, the utilitarian appeal is low: it won't right away see a step change in your Pytorch knowledge or whatever. The appeal of these books is to think deeply about fundamental ideas by implementing them in simplest language without too much help.

In short, YMMV. But if you have a long term view it might help you a lot than sort of currently fashionable trends. (Though I must admit that fast.ai is not just a flavor of the season resources but much better!)

havercosine··on ChatGPT is a blurry JPEG of the web
The blur is addictive because it feeds a feedback loop: rather than tiring out your brain on understanding one thing in detail, you can watch two summaries and have a vague sense of understanding. It allows to jump to next novelty, always feeding the system 1 of the brain but system 2 is rarely brought in picture.

I wonder if this will lead to a stratification of work in the society: lot of jobs can operate on the blur. "Just give me enough to get my job done". But fewer (critical and hopefully highly paid) people will be engaged in a profession where understanding the details is the job and there's no way around it.

In Asimov's foundation novel this is a recurring theme: they can't find people who can work on designing or maintaining nuclear power. This eventually leads to stagnation. AI tools can prevent this stagnation only if mankind uses the freeing of mental burden with the help of AI to work on higher set of problems. But if the tools are used merely as butlers then the pessimistic outcome is more likely.

The general tendency to lack of details can also give edge in some cases. Imagine if everyone is using similar AI tools to understand company annual reports which gives a nice, tiktok style summary. Then an investor doing the dirty work to go through the details may find things that are missed by the 'algo'.

havercosine··on Fundamental Math for Game Developers
Gustavo has created superb courses around Maths and Physics of game development.

I did 2 of his courses and RoI in terms of learning is huge. The crucial point is: he starts with an empty file and codes up the stuff with you at an understandable pace, line by line. A lot of concepts I just understood in hand-waving sense became much more clearer after going through this routine.

I'm planning to next take his Physics course but maybe code it up in Rust so that I'm not re-inventing some wheels (like dynamic arrays). But if anyone wants to brush up on most important pieces of maths: vectors, matrices, calculus I will heartily recommend his courses.

havercosine··on Google calls in help from Larry Page and Sergey Brin for A.I. fight
The fear is a lot deeper than quality of chat GPT's results for now.

Google is afraid that ads driven business models might get dislodged for "give me a service and I will pay you money for that" model. That will be a nail in the coffin. Imagine someone brings a marginally better conversational search engine with a low enough subscription price that some people migrate to use it rather than ad infested, SEO driven web results.

Also, with benefit of hindsight, Microsoft seems to be planning to attack Google from the flanks.. competition on a different business model than a better ad driven engine.

Sundar isn't exceptionally promising is spot on. We have so many cases of small business people being locked out of their accounts etc. They seemed to have preferred being a cool engineering company, not a B2B product company that other businesses can trust.

It is probably incorrect question to ask, but do some Google employees anonymously feel that lot of culture wars waged within company (Damore episode, the fights of ethics AI team with leadership) have caused the company to loose focus? I'm not disregarding moralities of the positions, just thinking if it was the case that many senior executives have to spend time firefighting company image in media due to these episodes.

havercosine··on Ask HN: Math books that made you significantly better at math?
I so so much agree. Just applying the D (differential operator) twice reveals what are we doing much more than the conventional d2y/dx2 notation.

By and large mathematics education has missed the point of invention of computers. It is only occasionally used to make a point. We should be teaching mathematics with a programming first approach: get your function code to compile, ponder on its signature, write some test cases to really understand what is going on.

havercosine··on Ask HN: Math books that made you significantly better at math?
My cultural reading of this notation problem is this: mathematical notation was formalised when scientists were writing papers and letters to each scientists in the era of scarce paper and no internet. So focus was on succinctness at the expense of explainability. It is akin to why commands on Unix machine were short, you were talking to an actual serial terminal to machine somewhere away. So saving a letter or two helped. Persisting with cryptic mathematical notation today, we are stuck with an idea well passed its sell by date. Mathematical notation, mostly, is not precise.

Sussman (who wrote the famous SICP book) wrote another book structure and interpretation of classical mechanics. Tough book to go through. But they start with the same premise: mathematical notation is confusing (and hand wavy at times). A better symbolic notation should reveal enough details to be able to code up the mathematics in a program. I found this approach to be bang on target, but could never get enough time to actually go through the book.

And I realised why the 'let us build it up from scratch' books work. They force you to think about the function signatures and shape of objects passed to each function. This approach reveals gaps in our understanding much better. For example, F=ma is looks like an algebraic statement, hiding the fact that `a` on the right is about time evolution of the system through the derivative.

Steven Strogatz made a funny quote in his infinite powers book. (I'm paraphrasing), if Newton was doing this in today's era he might create a flipbook animation to make this point and not symbols.

havercosine··on NeevaAI
Ignoring all the flak Stephen Wolfram gets, this is something that Wolfram Alpha does much better. Asking `escape velocity on Enceladus` to ChatGPT, it gave correct formula and still wrong result. All the hard-work of generating computable verified results for the Wolfram Engine database seems invaluable in comparison.

The deeper fact that Stephen Wolfram talks about in comparing ChatGPT vs Symbolic AI is true too. A layer deeper from ChatGPT is just a bunch of floats. So it is a lot harder to debug why the system arrived at the wrong result. For symbolic computations there might be a way to resolve how the expressions were transformed by the compiler and where it went off track, a debugging in true sense.

It is my gut feeling that we might see a next AI winter in a decade where we will finally start combining symbolic and connectionist approaches.

havercosine··on Hypermedia Systems
This book coming out is an amazing news. The HTMX website already does a fine job of explaining the philosophy behind hypermedia applications. But taking a 101/102 application like contacts app and building it with hypermedia is enlightening. Wishing you best luck for this book. I also liked the fact that rather than preaching one way to do things, the book shows alternatives and their downsides too (for example, the scripting chapter covers vanilla javascript, alpine.js and hyperscript). Kudos!
havercosine··on The cloudy layers of modern-day programming
I agree that most of the routine software work feels like, umm, filling forms? The way I've made peace with it is to accept that the work that pays the bills is going to be boring. After considering ourselves to be special as software developers for many years, maybe we accept that it is not so special after all, like flipping burgers.

Two ways to out from the existential dread: one startup on a problem statement that feels exciting to you, so that you get to pick your tools, processes and what not. But this is not a realistic/affordable option for most people. Also, if your startup achieves even a moderate level of success, you'll be back to solving boring problems.

The other is to find a pockets of software development outside work, where you can still solve interesting problems, like developing small games. Joseph White, who created the famed programmable fantasy game console Pico-8 alludes to this (https://youtu.be/87jfTIWosBw?t=1080) ! It was an explicit goal to build a tool to solve cute, interesting problems because routine software development feels like glueing things together. The whole youtube talk linked above is worth watching. But the trick here is to forgo commercial motivations. People make fun, cute little games on Pico 8, distribute them as cartridges. Occasionally, there's a hit like Celeste. But otherwise its a small community of people solving problems and building things just for fun.

havercosine··on Ask HN: Is it still possible to live in a terminal?
I was doing this experiment few years back with `emacs -nw`, tons of other packages in Emacs and custom elisp code. Used `ein` in emacs instead of Jupyter notebooks, browsed web in text browser etc.

But as I grew older I realised I need to pick my battles wisely. I still use emacs for some simple edit tasks and tramp for remote server file edits. But for normal day to day development that pays the bills, I've switched to vscode.

GUI based apps are offering better experience. For example, I can still configure Emacs to work well with Golang or Python. But the out of the box setup you get by clicking couple of buttons in VSCode is far superior.

It is separate matter if terminal world is getting abandoned is right thing or not. Maybe it is relic of a different era: when computers were time shared machines and connecting to them with terminal was the only option. But I increasingly realised that isolating myself out of better GUI based options is not the right battle to pick.

havercosine··on Google is shutting down Stadia
I think this is a spot on assessment. Google has always been a 'nerds trying to build a cool technology' company. Yes, one of it is search which is infinitely useful for users. But other than that, they have been in their shells nerding out things. Accidentally some of the technologies become useful and popular. But there is no grounds up thought about 'how can we make it useful for customers', in relative comparison to products from companies like Amazon and Slack.

Most user loved or impact making products like YouTube and Android have been acquisitions. I tried setting up Google Analytics for a business and touched it after many years. And the sheer complexity of the documentation and puzzling talk of UA to GA4 migration left me with a sour taste. I instantly switched to setting up plausible.io instead.

Notwithstanding the popularity of their tech ecosystem (because they are coming from Google), it is often painful to use. For all the flak Meta gets, the developer oriented technologies released by them: Thrift, PyTorch are fun to use as compared to Protobuf, Tensorflow.

But as someone who loves small companies taking on giants, it is heartening sign. Hopefully with ineffective leadership and employees focused on activism instead of actually building things, in next five years, un-bundling of Google will happen. Vertically focused players will bleed them by thousand cuts.

havercosine··on Ask HN: Boring but important tech no one is working on?
Shameless plug. I'm building this, a fun but curricular platform for games focused on mathematics grades 8 onwards. Please check https://pledu.co/trigo-shooter (no login needed). The game levels are designed to gradually expose you to one concept at a time. Currently live with 3 games on Trigonometry, Vectors, co-ordinate geometry. I am all ears for feedback!
havercosine··on Is Haskell really the language of geniuses and academia? (2019)
'Tried to it right notwithstanding what masses think' and commercial success are not exclusive. Two examples. Clojure's maintainers very carefully vet what goes into the core language. It has been a source of frustration for some contributors too. But the net result has been more positive than negative and the language has been successful (relative comparison with Haskell). Second example is Lua. There was a comment by its creator (I'm paraphrasing from memory) 'Lua grows by answering why, not by answering why not'. But it still has found (again, relatively more) success, despite being conservative about language feature-set.
havercosine··on Is Haskell really the language of geniuses and academia? (2019)
Thanks for writing about that experience, makes sense. My first to forays in the language ended the same way. I tried to do mainstream programming stuff in it but realised I wasn't making much progress in first few days and eventually moved on.
← PreviousPage 2 of 3Next →