HNHacker News
TopNewBestAskShowJobs

pmayrgundter

817 karma · joined December 22, 2018

co-founder bldrs.ai, citizen scientist, traveler

https://twitter.com/PMayrgundter

https://sites.google.com/site/pablomayrgundter

submissionscomments
pmayrgundter··on Celestia: Real-time 3D visualization of space
I have a hobby project to move Celestia to Web and uses its data files

https://celestiary.github.io/

pmayrgundter··on Lego Mechanical Computer
Love this! I once had a go* and failed.. this is really a step forward.

I think there's an important general principle about the role/capabilities of the external driver/operator that's party addressed here in the blog and video. You can see in the video a somewhat complicated manipulation to step the computer through one cycle and the author says:

"Because there is this feedback loop between memory and control logic, you need a fairly careful timing mechanism so that the output of the control holds steady long enough to set the appropriate state of the memory. That said, this control mechanism is basically missing from my computer. Instead, it relies on the operator (me) performing several different motions in the correct order to advance to the next state (as you can see in the video with me flipping several controls)."

This is importance bc it has to do with the thermodynamics of work. The more intelligent the operator, the less work efficient (less intelligent, so to speak) the computer need be. I'm impressed with the level of efficiency here.. his hand motions are rote and not particularly delicate.

The mechanism I had tried was to push a lego "program" (a flat block with protrusions) thru an "interpreter" (a chute with friction gears to drive a system of switches actuated by the program block's protrusions). This meant the driver was just a "push" of the program, which drove the system. Much nicer in theory, but also way harder and so didn't work. I also didn't have an interpreter of any sort. I had a few-bit display driven by the levers, so it was really just an analog machine.

Anyways, if anyone's playing with this, that's a suggestion for one area to focus.. to simplify the role of the operator to the point where there's no intelligence being used.. ideally just linear or rotational force.

* I was once into Quines and thought I was hot stuff after writing a self-printing program that also played the Game of Life[sg]. At work we had a table of legos, and I foolishly bet some colleagues I could build a self-printing program running on a lego machine. But the the computer is no simple feat itself. I'm now freshly inspired. I'd been exposed to the idea of a latch in school, but wasn't clear until now how important it is for state and control of the interpreter. [sg] https://github.com/pablo-mayrgundter/freality/tree/master/fu...

pmayrgundter··on Wikihouse: Open-Source Houses
Their site does have IFC for that project tho and I've loaded it in Bldrs, so seems fine there.

Yeah, IFC is neat. It's actually a dialect of STEP, so able to carry the geometry and tons of BIM metadata. We've estimated the spec is ~7k pages printed. It's like an abstract toolkit for working with building codes

Happy to chat more too. We have our project Discord and DMs on Twitter

pmayrgundter··on Wikihouse: Open-Source Houses
That's really interesting about Github-based versioning. My company (bldrs.ai) makes a webapp that views IFC (soon STEP) and has github integration, so any *.ifc path on GH can be pasted into our search for viz.

I'm not seeing any IFC files on Wikihouse repos, but they look old (https://github.com/wikihouse). Do you know where they're hosting?

pmayrgundter··on Researchers have identified the enzyme that gives urine its yellow hue
Am I getting this right?

1) bilirubin (from red cell breakdown) from bloodstream excreted into the gut

2) gut biota convert it to urobilinogen via bilirubin reductase

3) urobilinogen -> urobilin spontaneous breakdown (still in gut)

4) urobilin reabsorbed into bloodstream, filtered by kidneys, excreted in urine

If so, why not just leave it in gut for excretion with feces?

or perhaps 2.1) urobilinogen reabsorbed into bloodstream first, then 3). Makes a little more sense

pmayrgundter··on Understand how transformers work by demystifying the math behind them
Got me reading the original. It's rad.

Link to PDF and some screens from intro here..

https://twitter.com/PMayrgundter/status/1743096776456867921

pmayrgundter··on Understand how transformers work by demystifying the math behind them
Ah yes, Perceptron. Had a couple typos.. sorry, was on phone.

The bio-inspiration was via Frank Rosenblatt, who is referred to in that article tho yeah, the history is over in his article:

https://en.wikipedia.org/wiki/Frank_Rosenblatt#Perceptron

"Rosenblatt was best known for the Perceptron, an electronic device which was constructed in accordance with biological principles and showed an ability to learn.

He developed and extended this approach in numerous papers and a book called Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, published by Spartan Books in 1962.[6] He received international recognition for the Perceptron.

The Mark I Perceptron, which is generally recognized as a forerunner to artificial intelligence, currently resides in the Smithsonian Institution in Washington D.C."

Your Juergen page is interesting, tho no direct comment on Rosenblatt there. He does cite the work on this page:

https://people.idsia.ch/~juergen/deep-learning-overview.html (refs R58, R61)

My reading is that a long-known idea, about multi-variate regression, was reinterpreted by Rosenblatt by 1958 via the bio-inspired Perceptron, and then that was criticized by Minksy and others and viable methods were achieved by 1965. When I was taught NNs by Mitchell at CMU in the 1990s (lectures similar to his book Machine Learning), this was the same basic story. Also reminds me of a moment in class one day when a Stats Prof who was surveying the course broke out with "but wait, isn't this all just multivariate regression??" :) Mitchell agreed to the functional similarity, but I think that helps highlight how the biomimicry was crucial to developing the idea. it had laid hidden in plain sight for a century.

Agreed, and I was aware, there has since been criticism of the biological plausibility of backprop.

Your further links with refs to backprop in transformers are interesting; I hadn't seen these. It's clear the term is being used like you say, tho I still see ambiguity of it utility here. Autodifferentiation, gradient descent, multi-variate regerssion etc. are ofc in common use and scanning these papers it's not clear to me the terms aren't simply to a point of conflation. What had stood unique for me with backprop was a coherent whole-network regression. This to me looks like a piecewise approach.

But anyways, I see your point. Thanks!

pmayrgundter··on Understand how transformers work by demystifying the math behind them
So like in my sister reply, I don't see the Backprop, but maybe I'm missing it. This article does use the word, but in a generic way

"For example, when doing the backpropagation (the technique through which the models learn), the gradients can become too large"

But I think this is more of a borrowing and it's not used again in description and may just be a misconception. There's no use of the Backprop term in the original paper nor any stage of learning where output errors are run thru the whole network in a deep regression.

What I do see in Transformers is localized uses of gradient descent, and Backprop in NNs also uses GD...but that seems the extent of it.

Is there a deep regression? Maybe I'm missing it

pmayrgundter··on Understand how transformers work by demystifying the math behind them
Agreed Hebbian learning isn't used.. just meant it as an example of what would signal a NN.

For Backprop, I'm basing this off the development of the Perception. Wiki supports this and its bio-inslired origin[1].

As for its use in Transformers, if you mean simple regressing of errors or use of gradient descent, I'd agree, but that's not usually called Backprop and the term isn't used in the original paper. The term typically means back propagating the errors thru the entire network at a certain stage of learning, and that's not present in Transformers that I can tell.

Happy to see any support for your claims tho.

https://en.m.wikipedia.org/wiki/Backpropagation

pmayrgundter··on Understand how transformers work by demystifying the math behind them
This is all true in a neutral net, but Transformers aren't Neural Nets in the traditional sense. I was under that impression originally, but there's not a back propagation or Hebbian learning here, which were the key bits of biomimicry that earned classic NNs their name.

Transformers do have coefficients that are fit, but that's more broad.. could be used for any sort of regression or optimization, and not necessarily indicative of biological analogs.

So I think the terms "learned model" of "weights" are malapropisms for Transformers, carried over from deep nets because of structural similarities, like many layers, and the development workflow.

The functional units in Transformer's layers have lost their orginal biological inspiration and functional analog. The core function in Transformers is more like autoencoding/decoding (concepts from info theory) and model/grammar-free translation, with a unique attention based optimization. Transformers were developed for translation. The magic is smth like "attending" to important parts of the translation inputs&outputs as tokens are generated, maybe as a kind of deviation on pure autoencoding, due to the bias from the .. learned model :) See I can't even escape it.

Attention as a powerful systemic optimization is the actual closer bit of neuro/bio-insporation here.. but more from Cog Psych than micro/neuro anatomy.

Btw, not only is attention a key insight for Transformers, but it's an interesting biographical note that the lead inventor of it, Jakob Uzkereit, went on to work on a bio-AI startup after Google.

pmayrgundter··on Possible Meissner effect near room temperature: copper-substituted lead apatite
Thank you. I was never able to find this again. Really enjoyed it!
pmayrgundter··on Can a pill prevent deaths from venomous snakebites?
Went digging a bit. It's a "small molecule" (C21-H20-N2-O5) and commercialy available at $10/mg[1]. Presumably this would come down quite a bit if it were widely stocked.

The studies say they are looking at doses of 10mg/kg[2]. Similar dosing to a pain pill.

On a first read, looks like a partial solution as it only delays onset of venom pathology for a day, tho not sure I'm interpreting that correctly.

Otoh, it looks like the pills can continue to be taken for many days.. so perhaps there's promise for the venom to be slowly metabolized somehow that isn't inhibited. Might also bring down the cost of antivenom if it's now feasible to funnel patients into central treatment over greater distances, so stocks could be better used.

[1]https://pubchem.ncbi.nlm.nih.gov/compound/155815#section=Che...

[2]https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9695340/

https://clinicaltrials.gov/study/NCT04996264

pmayrgundter··on First new U.S. nuclear reactor since 2016 is now in operation
I wouldn't disagree with the high mortality from pollution toxins. I've never looked closely, but also seems common sense.

As for how much greening benefits offset costs, I was mildly frustrated by the downvoting :) and went researching on the economic effects in particular. The only study I can find on CO₂ fertilization is recent and stunning:

"we apply the CO₂ fertilization effect... backwards to 1940, and, assuming no other limiting factors, find that CO₂ was the dominant driver of [American agriculture] yield growth"

https://www.nber.org/system/files/working_papers/w29320/w293...

As they note, the extrapolation is from a study period of 2014-2020, but it's expected they have underestimated the prior role of CO₂ fertilization, as CO₂ response becomes saturated as levels rise and that's not represented in their model.

This is 2021 research funded by DOE & USDA, run by NBER, with PIs from Harvard and Columbia who are well cited within their field of environmental economics. Given the pedigree and that conclusion, this should be front-page news in science mags at least. I couldn't find any reporting on it except for Epoch Times (https://www.theepochtimes.com/science/nasa-satellite-data-su...), which gets an eye-roll from me, but guess they got this one :)

I read the full thing. It's great. Crosses high-resolution satellite CO₂ anomaly observations (NASA OCO-2) with crop yield records, wind modeling, sensitivity testing on non-ag land and the result is broadly repeated: +1ppm CO₂ -> +~0.5-1% yield. CO₂ levels went from ~310-410ppm in that period.

If true, it seems at odds with the normal account that gave nitrogen fertilization the lead role for increasing crop yields this past century and lifting most of the world's population out of regular food insecurity, and the growth of our population into billions. Seems more likely both were necessary.

It also makes sense, as C3 plants (e.g. soy, wheat and most plants besides hot/dry grasses) respond positively to CO₂ partial-pressures up to 800-1000ppm. C4 low-CO₂ adapted corn also fertilizes, just to a lesser extent. The authors also note the negative effects of heat stress, but this wouldn't have been as significant in the extrapolation period.

From Epoch Times: “The paper’s first author, Taylor, said: 'We reiterate that climate change will have a large negative impact on agriculture in aggregate, especially in places exposed to extreme heat. And higher CO2 may even lower food nutrition. But the countervailing fertilization effect should also be taken into account.'”

pmayrgundter··on First new U.S. nuclear reactor since 2016 is now in operation
The actuarial account for fossil fuels is important, but I've come across this set of summary items many times and it ignores the positive effects of CO2, which have been substantial. This is a great benefit for many ecosystems and also improves crop yields, to your monetary cost point.

The overall phenomenon is well documented, so I'll give just one highly credible account: "The greening [over the last 35 years] represents an increase in leaves on plants and trees equivalent in area to two times the continental United States… Results showed that carbon dioxide fertilization explains 70 percent of the greening effect" - NASA Apr 2016, https://climate.nasa.gov/news/2436/co2-is-making-earth-green...

I'm not wanting to rehash the climate debate, nor am I implying how much it offsets negatives and would grant all of your points otherwise. I do think it's wise to do whatever we can to move towards cleaner carbon burning to remove toxic and particulate emissions, and limiting CO2 emissions as well, but also within fair economic tradeoffs. I also support developing nuclear, fission and fusion.

pmayrgundter··on Veritasium: The SAT Question Everyone Got Wrong [video]
None of the explanations gave me an intuition for it except the circle rolling down a straight line with the same length as its circumference.

A roll down the line will rotate the circle once. Coming back the same. But rolling around one corner will add half a circumference, and another half for rolling around the other. So you get 2 * 0.5 extra circumferences, and so + 1 C. Somehow that helps with the other polynomials for me too. Super cool.

pmayrgundter··on The largest number representable in 64 bits
anyone really interested, i'm just rereading an earlyish (2010) paper by Sergeyev.. it's amazing

https://www.theinfinitycomputer.com/wp-content/uploads/2020/...

pmayrgundter··on The largest number representable in 64 bits
came here and searched for "busy". good job :)

To the sibling comment about arbitrariness, we could use a hybrid where we trade off some bits from IEEE FP to introduce far reaches and also some precision there.. so like, keep 32 bits or 54 bits for IEEE compatibility, then switch to "extended" ranges for e.g. BB numbers, higher alephs, etc..

There was this one system for calculation with infinities that avoided the Hilbert Hotel problem.. can't find it but was called smth like Infinioid or some other play on the name. Would be neat to bolt those on too :)

Edit: "grossone" is the calculus for infinities.. love this work! https://www.theinfinitycomputer.com/

pmayrgundter··on The Weight of New York City: Subsidence from Anthropogenic Sources
As a longtime NYC resident, the conventional wisdom is that much of the landfill for the buildings of the city went to expand shoreline of Lower Manhattan.

This was well reported for major digs like WTC/WFC, which created Northwest Battery Park City.

It's also seen in local landmarks like the Ear Inn on West Spring (one of the oldest bars in the city) which has an old painting inside showing the shoreline being not far outside the building, whereas it is currently a full block or two

For more: https://en.wikipedia.org/wiki/Lower_Manhattan_expansion

pmayrgundter··on Rumours of SR-71 Blackbird Successor from Lockheed's Skunkworks
The US has been bombing Syria for the past decade

https://en.m.wikipedia.org/wiki/Operation_Inherent_Resolve

pmayrgundter··on When do "brains beat brawn" in chess? An experiment
w00t, much better. thanks!

Good names in there :)

pmayrgundter··on When do "brains beat brawn" in chess? An experiment
I thought it does need to be trained (via self-play) to recognize patterns bc the game space is too large to exhaustively visit

Looks like it used to be brute force, as you say, but has since incorporated an NN after ita loss to AlphaZero

https://www.chess.com/terms/stockfish-chess-engine

pmayrgundter··on When do "brains beat brawn" in chess? An experiment
Nice piece
pmayrgundter··on Quadcopters can now visually track targets more effectively
what could possibly go right
pmayrgundter··on First word discovered in unopened Herculaneum scroll by CS student
A note on the type and number of the texts, from the site:

"Early attempts to open the scrolls unfortunately destroy many of them. A few are painstakingly unrolled by an Italian monk over several decades, and they are found to contain philosophical texts written in Greek. More than six hundred remain unopened and unreadable.

What's more, excavations were never completed, and many historians believe that thousands more scrolls remain underground.

Imagine the secrets of Roman and Greek philosophy, science, literature, mathematics, poetry, and politics, which are locked away in these lumps of ash, waiting to be read!"

pmayrgundter··on NASA finds water and organics in Bennu asteroid sample
Good q. No, looks like Stardust (comet) and Hayabusa (asteroid)

https://en.wikipedia.org/wiki/Sample-return_mission#2000s

pmayrgundter··on Brightline Orlando's first train: 125 mph from MCO to MIA [video]
thx all, yeah, suspected as much. I guess it's also their maiden trip, so maybe they'll tune up the route in the future. I think when Acela started in Northeast it took awhile before it was regularly up to speed
pmayrgundter··on Brightline Orlando's first train: 125 mph from MCO to MIA [video]
How is this 125 mph when it's same travel time as by car?

YT vid says Journey Time: 3 hr 33 min

Google maps says this 3h30m by car MCO to MIA.

Security overhead on the train looks medium, with TSA-style scanners but otherwise normal walk-up access when train arrives.

1h flight (tho add 1-2h for airport overhead)

Price: $79 (Smart), $149 (Premium)

pmayrgundter··on Bun v1.0.0
Thanks sir! Been looking forward to this and spinning it up right now :)
pmayrgundter··on Astronomers discover new quiescent galaxy
Thanks for the reference
pmayrgundter··on Astronomers discover new quiescent galaxy
Ah thanks! They mention Alma in the interview.

Maybe the specific claim was that wavelength at a resolution good enough to resolve individual stars, or a star per pixel often enough

← PreviousPage 4 of 14Next →