I share your awe. I feel like a kid in a candy store with all these incredible AI breakthroughs coming out these days! There's a place for cynicism and pessimism, but the kid in me who loves technology for its own sake is just absolutely on cloud 9.
193 karma · joined December 28, 2011
Sliding window chunking, RAG, etc. seem more sophisticated than the other document LLM tools, so I would love to try this out if you ever add the ability to run LLMs locally!
I assume MoE is still prohibitively expensive to train, which is why we're not seeing massive MoE models?
Big, if true. Their own testing puts present GPT4 at 82%, but 73.2% still seems very competitive.
I completely agree about the trend of closed-off spaces being terrible. I'm heartened by the number of open-source and open platforms that have picked up the mantle.
Lack of content has been my biggest issue with lemmy by far. There's just not enough content right now (once the many topics I have blocked because of zero interest are removed) to sustain my mindless scrolling habit. I frequently find myself at the bottom of the barrel.
I keep hoping that this results in me just wasting less time on the phone/computer, but my addiction is overpowering.
I realize right after saying that, my slight critique will be a bit ironic :D
I really like the variety of the comments, but some of them lean a bit towards being milquetoast and lack the edge that HN posters often seem to have. While there's often helpful posters who are clearly experts and offer encouragement, most HN posts will also include sharp, unnecessarily pointed critiques. There's also those comments which seem well-informed at first glance (and again, are usually negative), but which an expert would see through the jargon to see are empty of substance.
Adding a bit more of that will make the responses seem even more convincing. Again, great work!
That's sad to hear. Linux has always the rock solid beacon of the open-source world. I hope the spirit of OSS which produces alternatives to established incumbents in record time whenever the need arises keeps choices alive for users.
No. You can have the private key in an offline hardware wallet like Ledger, with the lightning app on your machine waiting for the device to sign it.
> And the hotwallet and lightning channel can only transmit, across all channels, as much as is in this hot-wallet, and each channel has a potentially higher and rising transaction cost to open as lightning network will generate more transactions to the blockchain, not less. So you can keep your cake safe, or eat it, but not both.
Let's break this down:
>lightning channel can only transmit, across all channels, as much as is in this hot-wallet
No. Each channel has its own limits, and there's no limits to the number of channels you can open.
> each channel has a potentially higher and rising transaction cost to open as lightning network will generate more transactions to the blockchain, not less If you want to open a lot of channels, yes, each one will incur a blockchain transaction. Keep in mind though that you don't need a channel per payment/per payment provider. Payments are _routed_ to your recipient via the network, in up to 20 hops, so you don't need to have a channel open to your recipient directly.
Also, in theory this should lead to _lower_ transaction fees on the network, not higher. Lightning transactions need to be segwit by design, so there's a 75% discount to each transaction. Since each channel can have an unlimited number of transactions on it, this should lead to a massive number of transactions moving out of the main blockchain into LN channels. Additionally, the channels can be opened whenever you want, so you don't have to wait till you're paying for your coffee to open the channel. If you don't mind waiting for a few blocks, you can open the channel for cents.
We don't know how this plays out in practice, of course, so we'll have to wait and see.
Edit: formatting
By design, the Lightning hot wallet can only hold small amounts of BTC (0.042 BTC currently), and yes, this would be essentially a hot wallet. The funds are held in a trustless, multi-sig, timelocked payment channel between you and your channel peer (which does not have to be the intended recipient). A channel can stay open indefinitely, and it will be possible to replenish it as needed. The idea is to have it function like a checking account. I fail to see how this is any different from a regular BTC wallet on your phone, or even your bank/investment app (except safer).If you're really paranoid about the funds in your hot wallet, you're free to create an m-of-n multi-sig wallet with somebody else you'd trust, which renders that attack vector useless.
Also, your LN wallet needs to remain online _only_ when it needs to transact. It can go offline the rest of the time with no problem. When it needs to make a LN transaction, it'll need to remain online for the duration of the transaction (a few seconds), or else the fund recovery mechanisms will kick into place.
AFAIK, this is not correct. Many of the Go playing algorithms before AlphaGo used MCTS or some variant. The true breakthrough of AlphaGo was deep reinforcement learning.
> AlphaGo's performance without search The AlphaGo team then tested the performance of the policy networks. At each move, they chose the actions that were predicted by the policy networks to give the highest likelihood of a win. Using this strategy, each move took only 3 ms to compute. They tested their best-performing policy network against Pachi, the strongest open-source Go program, and which relies on 100,000 simulations of MCTS at each turn. AlphaGo's policy network won 85% of the games against Pachi! I find this result truly remarkable. A fast feed-forward architecture (a convolutional network) was able to outperform a system that relies extensively on search. https://www.tastehit.com/blog/google-deepmind-alphago-how-it...
I don't know whether AlphaGo Master (the next version of AlphaGo that was trained purely with self-played games and has not been beaten in 60+ games) even uses MTCS.
Yeah, I suspect you're right. Eliezer was alluding to this with the AlphaGo victory as well:
> ... Human neural intelligence is not that complicated and current algorithms are touching on keystone, foundational aspects of it. https://www.facebook.com/yudkowsky/posts/10153914357214228?p...
I can't decide if I would be bummed or excited if that turns out to be the case. On the one hand, we'd be that much closer to AGI. On the other, we'd be continuing down the path of brute-forcing intelligence, rather than depending on those elegant, serendipitous breakthroughs that much of human progress has been built on.
With SC2, no AI even comes close to beating even a silver level player, so even a 5 year timeline seems really soon. Let's see if DeepMind can beat it!
What's your totally unscientific guess, Gwern?
This is the second time today I'm reading about "solving X with TensorFlow", which I find very interesting. It's not, "solving X with machine intelligence/AI/an application I wrote". I don't really have a point, just thought it was interesting!