HNHacker News
TopNewBestAskShowJobs

inawarminister

307 karma · joined December 11, 2015

submissionscomments
inawarminister··on Notes on training BERT from scratch on an 8GB consumer GPU
Hopefully tinygrad new drivers will fix this.

Also a lot of people were porting stable diffusion/Llama to AMD I think

inawarminister··on Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
Honesty with a bonus of better performance, as well!

For people in this thread, please search for Llama-descended finetuned models in Huggingface. The newer ones with 65B and 13B parameters are quite good, maybe not exactly substitutions to GPT-3.5-turbo and GPT-4 as of yet but is going there.

I like Manticore-13B myself, it can write Clojure and Lisp codes! Unfortunately it doesn't understand macros and libraries though.

inawarminister··on Venetians are pondering raising their entire city
Jakartan here, will never move even if the whole city get flooded (more than it has been). I'll probably get a houseboat and settle down.

More realistically, Northern Jakarta is sinking, but the Southern part (and the suburbs/exurbs in the surrounding area) are still going to be okay and Jabodetabek will still be home to millions.

inawarminister··on Ask HN: Those with success using GPT-4 for programming – what are you doing?
I've been using it to refactor Clojure codes and port Scheme examples into Clojure.

Maybe because Lisps have very low syntax, I find I can just copy off function/macro definitions as context and GPT3.5 can rewrite the code into something I can use... And by testing in a REPL (in :dev) I can instantly see which code has hallucinations and which work perfectly.

Tbh I find it hallucinates mainly when dealing with mutable states (e.g. atoms) or with pipelines of complex maps transformed by multimethods -

Just using small data bits and normal pure functions make GPT3.5 work perfectly because it doesn't need to take into account code outside the 2048-4096 tokens it's thinking right now?

inawarminister··on Russia’s Nukes Probably Don’t Work – Here’s Why
Destroying Moscow and St. Petersburg might be worth losing Paris, London, Washington, New York, and Los Angeles; but it certainly wouldn't worth losing Jerusalem, Mecca, Rome, and other centers of the worlds history. The others also has a vote, you know.

inawarminister··on Tucker’s Message Fiasco Is the Wakeup Call You Need to Auto-Delete Your Messages
End of history bias is very human.
inawarminister··on The Race to Colonize Mars Perpetuates a Dangerous Religion
There are people that will be willing to lose decades of subjective time getting iced.

Or if we can't do that (which would be weird since many complex mammals can hibernate already) then generation ships are possible. It'll probably be only done by culturally crazy groups, but hey! They're still counted as humans, and their expanding human habitats into new systems is still a win to us all.

inawarminister··on Russia killed its tech industry
Uh, millions of dead Iraqis and Afghans are going to see your wish and say "sure, why not?"
inawarminister··on The Ancient Order of Bali
Well it is The Ancient Order of Water Priests of Bali.
inawarminister··on The Ancient Order of Bali
Well the main problem is that population growth means that the traditional farming productivity - while sustainable for millenia - just isn't enough to feed the whole population of Indonesia and Bali in particular. Bali went from 1.5 million in 1945 (independence) to 2.9 million in 1989. The traditional approach is opening up new land but of course you can see how this is unsustainable.

So unfortunately the switch from biannual harvests to triannual harvests using factory-borne fertilizers and deep water usage was necessary.

(I was taught this in a fertilizer factory visit near Surabaya a few years back, so my information might be highly biased. But demographics data is there and ... It is what it is.)

inawarminister··on How to Make a Hotbed for the Earliest Sowings (2020)
Yeah, many trading cultures never had this injunction: for example Malay sultans rise and fall with the prosperity of their entrepots, exporting spices and rice and importing chinaware and Indian riches.

*

On the other hand... Confucianism puts merchants as one of the lower members of society, because they do not add value as artisans. Edo Japan famously had caste of samurai on the top, farmers on the middle, and merchants on the lowest.

inawarminister··on Thought Examinations, Indoctrination Meetings and Struggle Sessions
If I'm not mistaken, Plato's political cycle is monarchy replaced by oligarchy replaced by plutocracy replaced by democracy, which in turn is replaced with mob rule / anarchy then tyranny and return back to monarchy.
inawarminister··on When do you need chain-of-thought prompting for ChatGPT?
Have you tried Langchain? The Search agent using GPT-turbo is really good in my experience. Now to replicate it using local LLAMA...
inawarminister··on Street fighting engineers vs. martial arts engineers
Well I'm a HEMA swordsman moonlighting as a backend/LLM programmer and in many cases I think while wrestling with my spaghetti code: "How would Meyer deal with it?"

Human cross-domain capabilities are amazing.

inawarminister··on Most technologies aren't races
Yep, on the other hand, the US rival right now while a bit tyrannical, is actually in the similar tier to American academia; maybe a few months or even weeks behind.

And lots of people in the unaligned* world is ready to switch from San Fransisco to Beijing if the latter provides cheap LLM for commercial purposes.

* In the human sense of the world

inawarminister··on An Anxious Asia Arms for a War It Hopes to Prevent
KMT is broadly poised to win multiple cities this year and they're pro One China, though not communism necessarily.

Also Taiwanese military has some pro-One China elements in it last I've heard.

inawarminister··on An Anxious Asia Arms for a War It Hopes to Prevent
US intervening with Imran Khan led to this mess.
inawarminister··on Awesome-totally-open-ChatGPT: A list of open alternatives to ChatGPT
Open-assistant.io is what you want. They've gotten more than 100k RLHF training dataset and is actively tuning up a Chat model based on GPT-NeoX (?) from what I've heard.
inawarminister··on ViperGPT: Visual Inference via Python Execution for Reasoning
Ah, the Solomon solution.
inawarminister··on Load LLaMA Models Instantly
FWIW I was using ChatGPT with few-shot prompting to classify topics, extracting keywords, and sentiment analysis. Perhaps it might be interesting to make a comparison between Alpaca-7B and ChatGPT, hmm.
inawarminister··on American colleges in crisis with enrollment decline largest on record
HN is not a place to fight an ideological war, especially from a losing side.
inawarminister··on Stanford Alpaca web demo suspended “until further notice”
Yes that's because LLaMA is unfiltered, which is very good for general usage.
inawarminister··on GPT4 and the Multi-Modal, Multi-Model, Multi-Everything Future of AGI
Check out KoboldAI community, they're collaboratively finetuning and hosting GPT-J, NeoX, and now LLAMA models for free, as hobbies.
inawarminister··on Responsible AI Challenge
True, it'd be communist politicians here; anybody who praise them get punched by local people, after all.

Political bias is a very weird thing to navigate for IT people. Especially when technical capability, e.g. AI-assisted translation, is above and beyond sensitivity to 'local needs' in each country, or even in each specific regions.

inawarminister··on Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
There's open-assistant.io, which is doing RLHF directly on the open
inawarminister··on FDIC Takes over Silicon Valley Bank
There's a concept in Anthropology/Primitive Economics called "Gift Economy" where participants in a community gifted each other help, goods, food, and rely on being gifted back similar things in higher/lower proportions sometimes in the near future.

If I'm not mistaken, Debt: the First 5000 Years and the Dawn of Everything books examined this.

Your graph is a formalization of such relationship, no?

inawarminister··on AI teaches itself to use an API
fwiw I've been experimenting with 75% Pygmalion 25% ppo_hh mixed GPT-J models and it seems to be 80% as useful as chatGPT... With the exception of coding. Hope we can get there.
inawarminister··on Show HN: Dak – a Lisp-like language that transpiles to JavaScript
Huh it's orthogonal but basically Clojure is an attempt to answer all those four problems, right? Bypassing the which LISP problem by canonical implementation on top of host VMs (of which there are JVM, CLR, JS, Dart and BEAM), bypassing the spec, bypassing the whole object-oriented problem by impure functional approach, and having a BDFL on the top.

Anyway, back to the main point, thank you for the article. I'll try to play around with Dak later today and see if I can use it to do something fun. Play is better to learn something after all.

inawarminister··on OpenAI's Foundry leaked pricing says a lot
It's time to replicate OpenAI services with a real Open Source (or at least source-available, commercial-allowed) licenses.

It's time to make our own Linux. Our own Emacs.

Go to open-assistant.io and other similar initiatives.

Unfortunately GPT-NEOX, LLaMA, and OPT-IML are non-commercial-only. We small scale players should make our own.

Please contact me if anyone is interested in this space.

inawarminister··on France's baby bust
And, well, most of the Coalitions were almost exclusively funded by generous British subsidies. To Austria, Prussia, Sweden, Russia, and a couple dozen of minor powers. Industry made Britain able to afford that.

"In terms of soldiers the French numerical advantage was offset by British subsidies that paid for a large proportion of the Austrian and Russian soldiers, peaking at about 450,000 in 1813."

[0] https://en.m.wikipedia.org/wiki/United_Kingdom_in_the_Napole...

← PreviousPage 2 of 15Next →