HNHacker News
TopNewBestAskShowJobs

KhoomeiK

1,299 karma · joined August 7, 2018

research @ openai

https://rpandey.tech

submissionscomments
KhoomeiK··on Tocharian Languages
I love how historical linguistics, archaeogenetics, and anthropology posts make it to the front page of HN every once in a while. Is this interest all because of Sapiens? Or did Sapiens do well in the ~HN/Silicon Valley community because of some other underlying reason?
KhoomeiK··on The Psychedelic Inspiration for Hypercard (2018)
> "It seemed to me the universe is in a process of coming alive. Consciousness is blossoming and propagating to colonize the universe, and life on Earth is one of many bright spots in the cosmic birth of consciousness."

I had much the similar feeling during my first LSD trip, but haven't met too many others who shared this same idea. Life is the means by which the Universe organizes and becomes aware of itself. It's our Darwninistic telos to get off of this rock and colonize the stars.

KhoomeiK··on AI chatbots are not a replacement for search engines
Recent paper showing how LMs "struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail". In other words, simply scaling LLMs will not result in replacing Google as a search engine. Approaches like theirs ie retrieval augmentation are needed.

https://akariasai.github.io/files/llm_memorization.pdf

This is essentially what I'm building for the content marketing vertical.

KhoomeiK··on Introduction to Homotopy Type Theory
Here's a short paper that I spent several hours grokking as my first intro to Homotopy Type Theory, and which eventually sent me down a journey learning algebra, topology, and logic in order to fully understand HoTT:

https://mathweb.ucsd.edu/~ebelmont/hott.pdf

KhoomeiK··on Ancient grammatical puzzle solved after 2,500 years
Thanks! I'm still working through Rajpopat's thesis, but at a high level, it doesn't particularly "revolutionize" our understanding of the Astadhyayi. It's just further proof of Panini's ingenuity and the consistency & completeness of perhaps the only exhaustive symbolic/computational account of a natural language. Makes me even prouder to consider myself part of this millennia-old linguistic tradition.
KhoomeiK··on PhD student solves 2,500-year-old Sanskrit problem
Shameless plug for my talk on the influence of the classical Indian tradition on Western Linguistics & Natural Language Processing:

https://www.youtube.com/watch?v=NPl17qiaARM

KhoomeiK··on Ancient grammatical puzzle solved after 2,500 years
Shameless plug for my talk on the influence of the classical Indian tradition on Western Linguistics & Natural Language Processing:

https://www.youtube.com/watch?v=NPl17qiaARM

KhoomeiK··on Symbolic Regression is NP-hard
Interesting implications for "neuro-symbolic" computation and the so-called classical-neural synthesis many are looking for in deep learning, and Cognitive Science more broadly.

Does this mean that minds don't ever naturally build symbolic models of the world, or rather just that they try to minimize the computationally intensive (and now NP-hard) task of uncovering a symbolic model? I'd definitely lean toward the latter (ironically due to my intuition).

It also aligns quite well with Kahneman's System 1 - System 2 distinction, where the symbolic System 2 is reserved for situations that most strongly require such computation.

KhoomeiK··on Ask HN: Who wants to be hired? (October 2022)

  Location: Silicon Valley / San Francisco Bay Area (CA), Pittsburgh (PA)
  Remote: Open to it
  Willing to relocate: Open to it
  Technologies: Python, PyTorch, TensorFlow, Keras, HuggingFace, SpaCy, NLTK, NumPy, Pandas, Sci-kit Learn, Flask, JavaScript, Express, React, React Native, PostgreSQL, SQLite
  Résumé/CV: https://rpandey.tech
  Email: '%s.%s@%s.com' % ('rohan', 'pandey', 'gmail')
I'm a research consultant studying at Carnegie Mellon who's won over 10 hackathons with my full-stack experience and published machine learning research papers at top venues like NeurIPS and EMNLP.

My areas of interest include Deep Learning, Natural Language Processing, Neurotechnology, and Genomics, but I also just love doing AI/ML research in a customer-oriented startup context.

Open mainly to consulting positions and potentially new grad roles starting mid-2023.

https://rpandey.tech

KhoomeiK··on What Gödel Discovered (2020)
Not an expert but the proof is in fact entirely constructive and witnesses to the existential statement are constructible.

https://math.stackexchange.com/questions/448527/are-the-g%C3...

KhoomeiK··on Gödel, Escher, Bach: an in-depth explainer
I almost upvoted and then noticed it links to the AI Alignment Forum. Even a broken clock...
KhoomeiK··on John Carmack's new AGI company, Keen Technologies, has raised a $20M round
> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer.

People have thought Deep RL would lead to AGI since practically the beginning of the deep learning revolution, and likely significantly before. It's the most intuitive approach by a longshot (even depicted in movies as agents receiving positive/negative reinforcement from their environment), but that doesn't mean it's the best. RL still faces huge struggles with compute efficiency and it isn't immediately clear that current RL algorithms will neatly scale with data & parameter count.

KhoomeiK··on Tell HN: Turned 44 today and I'm lost
Meaning comes from narrative.

The most powerful narratives are the biggest narratives.

Most big narratives are non-verifiable or non-empirical narratives, e.g. "the creator of the universe died for your sins out of love for you".

Find the biggest narrative that satisfies your epistemological standards and you will also find the most meaning for yourself.

Your narrative need not change how you live your life, it simply provides context—a bigger picture for every action you take.

For me, this is darwinism.

KhoomeiK··on Ask HN: Share your personal site
https://rpandey.tech

Not much of a site to see, but I like to think it gets the point across in a pretty clean 1 page manner. Just a single HTML file.

KhoomeiK··on Ask HN: Who wants to collaborate? (April 2022)
I'm actively exploring potential B2B SaaS applications of my ML/NLP research, and would love to collaborate with someone more on the business side who might have knowledge of problem spaces as well as experience in customer discovery.

Please reach out if interested!

https://rpandey.tech

KhoomeiK··on Ask HN: Who wants to be hired? (April 2022)

  Location: Silicon Valley / San Francisco Bay Area (CA), Pittsburgh (PA)
  Remote: Open to it
  Willing to relocate: Open to it
  Technologies: Python, PyTorch, TensorFlow, Keras, HuggingFace, SpaCy, NLTK, NumPy, Pandas, Sci-kit Learn, Flask, JavaScript, Express, React, React Native, PostgreSQL, SQLite
  Résumé/CV: https://rpandey.tech
  Email: '%s.%s@%s.com' % ('rohan', 'pandey', 'gmail')
I'm a research consultant studying at Carnegie Mellon who's won over 10 hackathons with my full-stack experience and published machine learning research papers at top venues like NeurIPS and EMNLP.

My areas of interest include Deep Learning, Natural Language Processing, Neurotechnology, and Genomics, but I also just love doing AI/ML research in a customer-oriented startup context.

Open mainly to consulting positions.

https://rpandey.tech

KhoomeiK··on The Wisdom in Kung Fu Panda
I'd like to add that Kung Fu Panda actually illustrates a genuine philosophical conflict between Daoism and Confucianism, which played out across early China during the Spring and Autumn period.

Oogway represents Daoism with its search for inner peace and going with the flow/dao. Shifu, on the other hand, represents Confucianism with its concerns for moral duty and worldly social harmony. The internal and external, immaterial and material—each has a role in the construction of a philosophy conducive to success.

"Oogway explains that he can not make the peach tree blossom or bear fruit before it is ready. Shifu retorts that there are things that can be controlled like when the fruit falls or where the seeds are planted. Oogway finalizes by saying that no matter what is done, the tree will still grow to be a peach tree, instead of any other kind of tree and that great things can happen if it is helped and nurtured. Oogway argues the point of Daoism and that nature will take its course. Shifu argues the point of Confucianism and that everything has order." [1]

Also shameless plug for my essay about Culture, Philosophy, and Reinforcement Learning which comments on similar dualities and their relations. [2]

[1] https://medium.com/@gvreligion/quit-dont-quit-noodles-don-t-...

[2] https://medium.com/@rohan.pandey/culture-philosophy-and-rein...

KhoomeiK··on In Praise of Memorization
Memorization also forces you to "reduce dimensionality" in the data—find the underlying patterns of correlation and causality that cause the world to be structured the way it is. You can't feasibly fit every single event in the history of every country in the world into your head, but it becomes a lot more feasible once you extract a narrative that explains the data effectively.

Memorization implicitly requires learning and characterizing them entirely separately as modern education does is pretty faulty.

KhoomeiK··on Brains and algorithms partially converge in natural language processing
For those interested, similar research has been underway since 2019 in the labs of Ev Fedorenko at MIT and Tom Mitchell at CMU.
KhoomeiK··on Ask HN: Who wants to be hired? (February 2022)

  Location: Silicon Valley / San Francisco Bay Area (CA), Pittsburgh (PA)
  Remote: Open to it
  Willing to relocate: Open to it
  Technologies: Python, PyTorch, TensorFlow, Keras, HuggingFace, SpaCy, NLTK, NumPy, Pandas, Sci-kit Learn, Flask, JavaScript, Express, React, React Native, PostgreSQL, SQLite
  Résumé/CV: https://resume.rpandey.tech
  Email: '%s.%s@%s.com' % ('rohan', 'pandey', 'gmail')
I'm an undergrad at Carnegie Mellon who's won over 10 hackathons with my full-stack experience and published machine learning research papers at top venues like NeurIPS and EMNLP.

My areas of interest include Deep Learning, Natural Language Processing, Neurotechnology, and Genomics but I also enjoy building impactful products!

Open to internships, part-time, and some full-time positions.

https://rpandey.tech

KhoomeiK··on Ask HN: Is it worth it to learn C to better understand Python?
Yup yup yup. I've been coming to this realization about myself so much recently too. It's how I taught myself web/app dev with hackathons, facing a huge project head-on and "backtracking" to learn stuff as needed. It's how I taught myself machine learning by jumping into a research project with a prof without ever having written PyTorch. And it's how I'm now teaching myself higher math, having just jumped into the homotopy type theory textbook and an idea I have for application in NLP, and going backwards to learn the required category theory and algebraic topology.

I wonder whether the fundamental structure of curricula could be changed to take advantage of this fact, because I think it's true for a lot of people.

KhoomeiK··on Ant mill
The more one looks at the animal kingdom, the more one wonders where the clear demarcators of individuality are. Emergent behaviors like this suggest to me that maybe the ant colony itself is more of an independent organism than any individual ant.

Isn't this true of humans too when we rely on entire supply chains for our food, water, and electricity?

KhoomeiK··on Ant mill
Can you elaborate on this / provide further reading?
KhoomeiK··on Extremist content on Reddit, Discord worrying trend for young adults in India
Source? The statistics disagree:

https://img.etimg.com/photo/msid-65207066,quality-100/graph....

KhoomeiK··on A connectomic study of a petascale fragment of human cerebral cortex
It's simply a matter of scaling this slice-scan-render technique up to the entirety of a human brain to simulate human thinking, correct? What're the biggest technological hurdles left?
KhoomeiK··on How I took my SaaS from idea to sold in 14 months
$218 MRR = $2616 ARR

With a pretty standard 4x multiple, it's a $10k business. Though I imagine the slower growth etc resulted in an actual valuation ~$8k.

KhoomeiK··on What Does It Mean for AI to Understand?
This is basically my area of research right now, compositional/symbolic representations in neural language models. I'll let you know the answer in a few months :)
KhoomeiK··on What Does It Mean for AI to Understand?
People outside the NLP research community need to understand that a language model does nothing more than calculate probabilties for the next word given some context. The way it learns that probabilistic function can be quite complex, involving billions of parameters, but it's still fundamentally the same. Most of the increase in performance in recent years has come from the ability to train LMs that better generalize this probability function from huge text corpora--functions that better interleave between the datapoints it's trained on.

Humans use language with purpose, to complete tasks and communicate with others, but GPT-3 has no more goals or desires than an n-gram model from the 90s. LMs are essentially a faculty for syntactically well-formed or intuitive/system 1 language generation, but they don't seem to be much more.

KhoomeiK··on The Invention of Chinese
This discussion isn't about the genetic validity of population groupings in East Asia, but rather the construction of Han identity which is a social phenomenon.
KhoomeiK··on The Invention of Chinese
Related [1] [2]: the Han race was really only constructed in the last 100 years, along with the national mythos of several millennia of uninterrupted dynastic civilization.

[1] https://www.hrichina.org/en/content/4573

[2] https://www.youtube.com/watch?v=vhFPZL2L8uM

← PreviousPage 3 of 9Next →