'If it switched to superposition, we should see overlapping mixtures on top of each other, ... But that’s not what we see. The neurons stay interpretable and generalized, but coarser'
236 karma · joined May 8, 2024
'If it switched to superposition, we should see overlapping mixtures on top of each other, ... But that’s not what we see. The neurons stay interpretable and generalized, but coarser'
So many possibilities.
https://techcrunch.com/2026/05/20/anthropic-will-pay-xai-1-2...
Taking a good model like GLM5.2 and just fine tuning it on coding can decrease real world performance due to mechanics like catastrophic forgetting. There is also other interesting behaviors were training on a broad training set can improve coding performance because there is positive transfer.
There is 100% an effort to make solid coding focused models, but it is very hard to do that without including capabilities across a broad set of adjacent tasks.
I've been in NLP since the LSTM days and it's hard for me to look at LLMs and not just think they are incredible. It's truly a different level of expressiveness. So much of capabilities research is pointing to LLMs effectively learning a world model.
RLVR is also proving really effective. It is hard for me to imagine a world in the future where LLMs aren't at human level performance across a wide variety of tasks.
I fully acknowledge that current LLM labs have a financial interest in people believing AGI is very near, but from what I'm reading in the literature and seeing myself experimenting with the SOTA models it doesn't seem totally unreasonable.
What evidence are you seeing that makes you confident that AGI in the soon-ish future is a complete myth?
Section 1 doesn't say anything
Section 2 seems to boil down to: "improve cyber security and maybe use AI if we can find funding for it"
Section 3 proposes building a benchmark for evaluating cyber security performance of models that developers can choose to benchmark against. This seems like a good idea, I know Jack Clark has been a huge advocate for government's getting in with benchmarking.
Section 4 says to prioritize prosecuting cyber crimes. Not sure why they wouldn't already be prosecuted.
Section 5 doesn't say anything
Your comment makes it sound like they are miles apart, which the benchmark doesn't seem to support.
Edit: I looked at the data more and the two models are only basically equal when looking at the mean of all the tests. Gpt 5.5 significantly outperforms opus 4.7 in coding, while opus 4.7 significantly outperforms in "decision making." I'm not seeing details on what decision making explicitly means.
Alaskan winters are hard regardless of how many friends you have.
I wonder if for those of us that haven't used Facebook in years the recommendation algorithm is essentially default. Which much like the default youtube algorithm, is completely garbage. But if we did use it (which I have no intention of doing), it would start being more reasonable.
The problem I could never solve was the speed, and from reading the paper it doesn't seem like they managed to solve that either.
In the end, for my work, and I expect for this work, it is only usable for pre generated terrains and in that case you are up against very mature ecosystems with a lot of tooling to manipulate and control terrain generation.
It'll be interesting to see of the authors follow up this paper with research into even stronger ability to condition and control terrain outputs.
Google photos isn't perfect either but I never saw these kind of issues when I was still using it.
Face recognition in general just isn't as good as Google Photos.
It's still an amazing piece of software and I'd never go back, but it isn't perfect yet.
Medicaid
FY 2023 Budget: $900.3b ($620b federal, $280b state) [1]
FY 2023 Budget not spent on benefits (admin overhead): 5% ($45b) [2]
SNAP
FY 2024 Budget: $100.3b [3]
FY 2024 Budget not spent on benefits (admin overhead): $6.5b [3]
TANF
FY2024 Budget: $31.5b ($16.5b federal, $15b state) [4]
FY2023 Budget spent on program overhead: %10.1 ($3.2b) [5]
Total Admin Spending $54.7b -> $169 per person in the US
So not totally negligible but also not exactly a basic income
[1] (https://www.macpac.gov/topic/spending)
[2] (https://www.congress.gov/crs-product/R42640) See figure 4
[3] (https://usafacts.org/answers/how-much-does-the-federal-gover...)
[4] (https://www.gao.gov/assets/880/872093.pdf)
[5] (https://acf.gov/sites/default/files/documents/ofa/fy2023_tan...)
I think this article from Beijing University does a great job of highlighting some of the issues.
Web page:
https://fzzfyjy.cupl.edu.cn/info/1035/11343.htm
Pdf: https://www.law.pku.edu.cn/docs/20210927092933450384.pdf
Furthermore, it clearly is a system that is important to the CCP and has an effect.
From the Baidu page:
建立社会信用体系是保持国民经济持续、稳定增长的需要
Establishing a social credit system is necessary to maintain sustained and stable growth of the national economy
Certainly Mainland China does need a credit system, and undoubtedly the Social Credit System will and has helped in that regard, but it does have legitimate flaws with regards to privacy.
Its goals extend beyond ensuring creditworthiness to
社会信用体系具有揭示功能,能够扬善惩恶,提高经济效率;
The social credit system has a revealing function, can promote good and punish evil, and improve economic efficiency;
And its integration with the National Healthcare Security Administration, and other government and private entities extend its reach far beyond what the Western credit systems do.
gov.cn page on social credit system plan https://www.ndrc.gov.cn/xxgk/zcfb/tz/202406/P020240604321155...
gov.cn page on social credit system suggested changes/opinions https://www.gov.cn/zhengce/202503/content_7016535.htm
baidu page on social credit system https://baike.baidu.com/item/%E7%A4%BE%E4%BC%9A%E4%BF%A1%E7%...
Are you making the distinction that its name is actually the social credit system and not social score? Or that the system isn't fully in place yet.
Or perhaps are you suggesting that gov.cn and baidu are part of the deep state's propaganda plan against China.
1. Instagram, Facebook, Youtube, Twitter (basically all major American social media platforms) are not accessible without a VPN. Some major VPN providers have also been banned as well. The counter argument is that many Huawei products are also banned. But I actively use Harmony OS on my Huawei smart watch. I can view Bilibili content, XiaoHongShu, QQ, and other platforms on any device without issue.
2. State media propaganda. My first time seeing state movies was shocking how in your face the propaganda was. It is completely blatant. From talking with locals about it, they recognize it as propaganda (although the Chinese word for propaganda doesn't have the same negative connotation as in English). I haven't watched state sponsored news to get a feel for their bias, but from the amount I have seen they are certainly selectively with their content. Everything focuses and the achievements Xi JinPing has recently achieved. The idea of a media outlet reporting on something silly the president has done is absurd.
3. The surveillance infrastructure in China surpasses even the UK. The number of cameras almost looks like a gag. Once again, the nature of how opaque the government is means it is difficult to say with any degree of certainty what they use the data for, but they certainly collect a lot.
You can change it from Google account > Data & Privacy > History Settings > youtube History
If you have youtube premium + a general purpose ad blocker + disable watch history its really hard to tell if you are being tracked.
If you do decide to disable watch history, be prepared for just how terrible the median youtube interest is. All recommendations become beyond worthless.
This truly is shocking. If you are reviewing every single line of every package you intend to use how do you ever write any code?