HNHacker News
TopNewBestAskShowJobs

abhi9u

1,070 karma · joined July 12, 2019

I am a software engineer (unemployed at the moment). I am building https://drpawd.com and trying to build a writing career at https://codeconfessions.substack.com/
submissionscomments
abhi9u··on What every developer should know about GPU computing
Yes, that's a good feedback. Few other people also mentioned this. I was trying to save space, to spend time more on the "how things work" part. But I see your point that some motivation behind parallelization would make it a more interesting read.
abhi9u··on What every developer should know about GPU computing
Thank you for reading! :)
abhi9u··on What every developer should know about GPU computing
True. This is probably what first few lectures of a first course on GPU computing would cover. The nitty gritty of how to write parallel algorithms for maximum throughput on the GPU is where things become too specific. :)
abhi9u··on What every developer should know about GPU computing
True. It's becoming a lost art. Although, I think anyone who takes a CS program probably learns enough about CPUs in a Computer Organization or Computer Architecture course. Even if they never have to use that knowledge ever again. :)

On the other hand I agree that people who take the non-CS route may not be that intimately familiar with all the details.

abhi9u··on What every developer should know about GPU computing
Thank you :)
abhi9u··on What every developer should know about GPU computing
Thank you for sharing!
abhi9u··on An Analysis of DeepMind's 'Language Modeling Is Compression' Paper
Thank you for sharing this. I am the author, happy to answer any questions about the article.
abhi9u··on Ask HN: Should I use LLMs for NLP such as sentiment analysis or classification?
I'm not sure what is the state-of-the-art in sentiment analysis. However, GPT-3.5 and GPT-4 work great for it with few shot learning. You give a representative set of examples in the prompt and it works pretty well.

On the topic of classification, large language models such as BERT are the state-of-the-art. But, there was a paper recently at the ACL [1] which showed that a simple approach of using gzip compression + k-nearest neighbour achieved similar level of performance. Later on, their KNN evaluation methodology came under question because instead of doing KNN with k=2 they reported accuracy based on top-2. Even then, what it shows is that we don't need LLMs for simple tasks such as classification, where traditional techniques still work well and are much cheaper to run. I'm sure some people might say that we should just discard that paper because it had that error. But Sebastian Raschka ran it on the IMDB movie reviews dataset and reported 71% accuracy [3]. I wrote an explainer article on it [2].

[1] https://aclanthology.org/2023.findings-acl.426.pdf [PDF]

[2] https://codeconfessions.substack.com/p/decoding-the-acl-pape...

[3] https://magazine.sebastianraschka.com/p/large-language-model...

abhi9u··on Understanding Immortal Objects in Python 3.12: A Deep Dive into Python Internals
Thank you for submitting this. I am the author, feel free to ask any questions!
abhi9u··on Probabilistic Machine Learning: Advanced Topics
Yeah. Kevin Murphy just announced on Twitter that this is the final official version of the "Advanced Topics" book.
abhi9u··on LPython: Novel, Fast, Retargetable Python Compiler
That project has received only 20 commits since it was open sourced, with bare minimum readme. Doesn't look like they are interested in exposing it too much for outside use.
abhi9u··on LPython: Novel, Fast, Retargetable Python Compiler
Short summary - https://twitter.com/abhi9u/status/1684962764115308545
abhi9u··on Decoding the ACL Paper: Gzip and KNN Rival Bert in Text Classification
I take your point that it could have been more strongly worded. The reason I say it "devliers unexpectedly well" is because the whole concept of using gzip for classification is unintuitive, and even after fixing the flaw it still manages to get decent accuracy (given that it is no more beating state-of-the-art models).
abhi9u··on Decoding the ACL Paper: Gzip and KNN Rival Bert in Text Classification
Yes,that's true. This approach will not work as well as techniques which build a semantic model of the data, such as embeddings.

For example if you want to do positive vs negative sentiment classification of movie reviews, this will perform just as well flipping a coin. (I tried this)

It works well for multiclsss classification. For example, let's say I have an article about soccer, I am a bound to have words like goals and scores. And I will find a good amount of texts in the corpus from the same class with same vocabulary. Thus their ncd will be lower, as opposed to texts from other classes. But this isn't really doing anything related to semantics.

I am going to be writing another article on this, where I will try to explain why gzip is helping it do this.

abhi9u··on Decoding the ACL Paper: Gzip and KNN Rival Bert in Text Classification
Thank you for reading :-)

I mentioned it towards the end, in the 2nd last paragraph. Those issues in the evaluation do bring its accuracy down a bit, even then it performs better than expected, considering it is doing knn on compressed data.

abhi9u··on Ask HN: What tech newsletters are you currently subscribing to?
I started one newsletter/blog couple of months back. I write blog style posts where I write detailed articles on technical topics for example explaining the AlphaDev results from DeepMind. And I also write a links digest style post every week as well where I share some interesting articles and resources every week.

https://codeconfessions.substack.com

abhi9u··on Ask HN: Could you share your personal blog here?
I just started 2 months back. Here it is: https://codeconfessions.substack.com/

Here are some of my posts which have been well received:

- https://codeconfessions.substack.com/p/creating-chatgpt-plug... : It takes you through a tutorial showing how to use the function call feature to build your own ChatGPT with plugins

- https://codeconfessions.substack.com/p/exploring-deepminds-a... - This explains the results of the AlphaDev paper from DeepMind

- https://codeconfessions.substack.com/p/mojo-the-future-of-ai... - This gives an introduction to the Mojo programming language

- https://codeconfessions.substack.com/p/will-ai-replace-progr... - This one is my personal take on whether programming jobs are in danger becaue of AI or not.

abhi9u··on Tell HN: AWS Is Down
On a Friday! Feels for Engineers handling this across companies. :-|
abhi9u··on Ask HN: Which Vector Database do you recommend for LLM applications?
I have been using FAISS for running https://drpawd.com. It is pretty solid, the only downside is that it's all in-memory so as your dataset grows you need to throw more memory at it.
abhi9u··on Ask HN: A Good AI Newsletter?
What are you looking for? Industry News, papers or more hands on code oriented newsletter?
abhi9u··on Ask HN: Is Anyone Else Having Indexing Issues with Google Search Console?
I don't think it's going to be any different. I guess if the page response time is fast it helps with your overall SEO but I don't think it will have impact as far as indexing is concerned. Probably your guess is right that this might be Google's way of handling AI generated content.
abhi9u··on Ask HN: Is Anyone Else Having Indexing Issues with Google Search Console?
Yes, I have experienced the same. It takes a long while (several days) after submitting sitemap for the pages to actually get indexed. Even after it starts indexing, it will do few pages per day (few usually is < 10).

It helps if you have a back link to kickstart things. Even then if some of your internal pages are not easily reachable from the home page, those page will not get indexed soon enough.

abhi9u··on Twitter's Algorithm: Amplifying Anger, Animosity, and Affective Polarization
A thread by one of the authors of the paper, giving an overview: https://twitter.com/SmithaMilli/status/1663222170560999424
abhi9u··on Ask HN: What are some good resources for learning about low level disk/file IO?
Did you see this discussion? https://news.ycombinator.com/item?id=32965075

As far as books are concerend: - Database Internals

- Designing Data Intensive Applications

- Disk-Based Algorithms for Big Data

- Database Systems by Ullman et al (http://infolab.stanford.edu/~ullman/pub/dscbtoc.txt) Part IV covers implementation details of a database system.

abhi9u··on Deep Learning Systems: Algorithms and Implementation
Wow. How did I miss this gem. It seems the course is already finished so we can't participate in it anymore but we can still watch the lectures.

Thank you for sharing!

abhi9u··on The Art of Code Review
haha, that's a fair point. I did not write the article. I hope the author sees your comment. :)
abhi9u··on All the arguments against EVs are wrong
no, I didn't see this. I am on Google Chrome.
abhi9u··on Saturn's rings are far younger than once thought
Yes, that's true. Saturn's gravity is slowly pulling the particles in the ring. In a few million years, the rings will completely disappear. Source -- StarTalk podcast (https://drpawd.com/search?query=are+saturn%27s+rings+falling...)
abhi9u··on All the arguments against EVs are wrong
Yes, that's a fair point.
abhi9u··on Ask HN: On the Threats of AI
This is what I argue in my post on AI's impact on programming jobs -- my take is that if AI ends up automating low end tasks, then it opens up opportunities for us to spend our time on more creative and challenging problems. The current generation of models know all the data structures and algorithms but do they have the ability to extrapolate their knowledge to come up with novel solutions to unseen problems -- which is what we are good at. Similarly, there are tons of interesting problems which are unsolved in other domains and we could use AI's help to tackle those. Naval Ravikant talks about something similar on Joe Rogan Podcast -- where he says everybody can be rich. His argument was that if every human on the planet could become an engineer, or doctor, or scientist, we would solve all of our basic problems in few years and then humanity would be free to explore creative things. So if AI gets us there, we could be writers, actors, scientists.

https://codeconfessions.substack.com/p/will-ai-replace-progr...

← PreviousPage 2 of 3Next →