Indeed, there have been so many new activation functions that I have stopped following the literature after I retired. I am glad to see that people are trying out new things.
I'm surprised that the article doesn't mention that one of the key factors that enabled deep learning was the use of RELU as the activation function in the early 2010s. RELU behaves a lot better than the logistic sigmoid that we used until then.
I worked summers in a lab in the 1980s that had Silicon Graphics, Apollo and Sun workstations. The Sun was the easiest to program by far so it got the most use.
Yeah - it was pretty chaotic when I was there too. We had to use the internal purchase order system to buy certificates from a CA. It was cumbersome as it required several levels of approval. I have long since quit IBM thankfully and had forgotten all about it until I saw this post today :-)
When it comes to certificates, mess-ups happen frequently in big companies. When I worked at IBM, I initially had the responsibility for obtaining and renewing the code signing certificate for a Java applet that we used in our product. I handed off the responsibility to another employee when I left that group.
I was on vacation in the middle of nowhere when I got a panicked call from the product manager asking me to renew the certificate as the employee had quit. I had to spend half a day walking someone through the renewal process - ruined my day.
Article like this are why I keep coming back to HN: I was using an LDR (Light Dependent Resistor) for a similar task but this LED idea is fascinating. It would have never occurred to me!
I will never buy a car without CarPlay - it's that simple. I will likely buy an EV at some point but I will never buy a Tesla, Rivian or GM EV since they lack this essential feature. I like that all of my phone's relevant data is accessible and I don't have to do any sync. I listed to podcasts with the Overcast app on my iPhone - the CarPlay interface is excellent. I like having access to a choice of maps on the display.
That was an excellent write up! I am really surprised that the built-in navigation system does not enable you to search for charging + activity. That seems like it ought to be essential!
My point is that while Cisco was an early winner, Amazon and Google started growing a few years later and were much larger winners in the end. I think there is a possibility that history may repeat with Nvidia -but I have no idea what new companies are going to rise in the AI era.
Nvidia of today reminds me a lot of Cisco of thirty years ago. Back then when the Internet was starting to take off, Cisco was one of the few companies actually making any money from the Internet. The frenzy to buy Cisco was intense. Thirty years later, Cisco still exists but absolutely nobody is excited about them.
I worked in tech for thirty years and was able to retire comfortably. I was well paid but never had a spectacular salary - what worked was steady saving and investing.
Indeed, one of my last big projects I worked on before I retired was using an LSTM model for named entity recognition. It was many times faster than anything you can do with an LLM.
I spent the summer of 1989 working on an Meiko Computing Surface which was a parallel computer built from T800 Transputers. I learnt Occam and implemented a few parallel algorithms on it. As I recall, the code running on the Transputers did not have access to the file system so getting data into and out of the mesh networks was difficult. It appeared at first glance that back propagation might be a good fit on the Transputer but I never did get it running. Instead, I used a Connection Machine CM2 to train my neural networks. However, getting time on the CM2 was hard so I wound up just using a collection of IBM RS6000 workstations that IBM donated to my university.
After working in AI for a big chunk of my career, I have switched gears in retirement and am having a great time learning to program small controllers like Arduinos. It's like going back in time - minimal OS, simple tools and direct access to the hardware. Just for the heck of it, I am building a simple robot from scratch right now and its been great getting back to the basics.
How does this kind of constraint modeling compare with logic based approaches like Answer Set Programming (https://potassco.org)? Years ago, I got excited by the possibilities of ASP but it does not seem to have progressed much
The RP2040s can multitask. This is because the Arduino compatibility layer is actually built on top of MbedOS which support preemptive multitasking. You have access to the full API set of MbedOS in addition to the Arduino APIs when the code is running on an Arduino RP2040.
It has Bluetooth, Wi-Fi and an IMU built in which is very useful for a robotics project I'm currently working on. Plus it has a huge amount of RAM and flash compared to an ATMEGA Arduino