linguae

This reminds me of how I learned about Zipf's law. 19 years ago, I spent the summer after my sophomore year in a research lab at UC Santa Cruz as part of an NSF-funded summer research program for undergrads. My project that summer was to investigate index design for the Ceph distributed file system in order to enable faster file search. This was a hot topic at the time, especially with Spotlight being introduced to Mac OS X Tiger, and we wanted to see how to enable similar functionality for a distributed file system. I learned about Zipf's law, which states that there is a power law distribution of word frequencies in many natural languages. I remember downloading the Project Gutenberg corpus and writing a Python script where I stemmed the words and computed the frequencies of the stemmed words. The resulting distribution confirmed Zipf's law.

I would love a tablet running macOS, which I prefer over Windows and desktop Linux. I have a Microsoft Surface Pro 7. It performed fine under Windows 10, though performance suffered when upgrading to Windows 11. The Surface Pro is a nice piece of hardware, but I wish it had macOS instead of Windows. The iPad Pro definitely has enough processing power to run macOS, but it runs iPadOS, which is too restrictive for me as a developer.

If Apple went through the effort making macOS touchscreen-ready for this new MacBook Pro, then perhaps we’ll see a tablet with macOS one day. I’d buy an iPad Pro with macOS in a heartbeat.

On the Windows side, Windows 2000 and XP/Vista/7’s Classic mode, though Aero was quite nice. I’m not a fan of the interfaces from 8 onward, and I didn’t like Luna in XP.

On the Mac side, it’s a toss up between the platinum interface of Mac OS 8/9 and Mac OS X Tiger. With that said, I have a work-issued MacBook Pro running Golden Gate as my daily driver, and Liquid Glass is alright. It’s not what I’d prefer if I had a choice, but it’s not a dramatic departure from previous versions of macOS. A Cheetah or Puma Mac OS X user should be able to navigate Golden Gate with ease, mostly.

I love this quote! When I taught programming language paradigms at San Jose State as an adjunct, I introduced this quote as a motivating factor for learning different ways of programming, particularly introducing functional programming, logic programming, Smalltalk, and the Common Lisp Object System to senior-year students who coded mostly in Java, Python, and C (though this was during the COVID pandemic, right before the advent of vibe coding).

Alan Kay (of Smalltalk fame) has a similar, related explanation: computing is a pop culture with little regard for the past (https://link.springer.com/content/pdf/bbm:978-3-319-90008-7/...). There are thousands, maybe millions, of developers who have little knowledge of the history of our field. I’ve found this true even of graduates from highly-ranked computer science bachelor’s programs. Think of all of the current-day LLM researchers and engineers who have never heard of Minsky, McCarthy, Lisp machines, and other aspects of the history of AI before we learned that attention was all we needed.

It wouldn’t surprise me if Apple’s UI/UX teams are staffed with people who have never heard of Don Norman, Bruce Tognazzini, Bill Atkinson, Andy Hertzfeld, Susan Kare, Larry Tesler, and other pioneering figures who contributed to Apple’s legendary 1980s and 1990s software.

It’s a lack of onboarding and knowledge transfer, combined with an Eternal September of thousands new CS grads being constantly produced annually, who are sadly not being taught enough history of computing at their universities.

My dream as a CS professor is to help rectify this, at least amongst my students.

Yup, just like during the bad old days of Internet Explorer's dominance. I remember encountering websites that only worked on the Windows version of Internet Explorer as late as 2006. We had a nice time in the late 2000s and early 2010s when web developers coded against standards instead of against the market-leading web browser. There was competition between Firefox, Opera, Safari, and a revitalized Internet Explorer. I don't remember IE 7 and later being that bad, though by then I had switched to Firefox on Linux/Windows and Safari on Macs.

Then came an upstart web browser from Google named Chrome....and now we're back to the bad old days of web developers coding to the dominant web browser again.

In many ways NeXTSTEP can be thought of as a “pragmatic Smalltalk environment” that traded some of Smalltalk’s dynamism for improved compatibility with the Unix ecosystem. Recall that NeXT’s original target market was higher education, which even in the 1980s started relying on Unix. This was the era of the Unix workstation, and NeXT can be considered a competitor to Sun, SGI, HP, DEC, and other Unix workstation vendors. NeXT’s differentiator was its Smalltalk-inspired object-oriented environment, built on Objective-C.

I wonder, though, what a pure Smalltalk workstation would’ve looked like, and what hardware would’ve been necessary to make it fast. After all, the late 80s were the era of Lisp machines, though Lisp machines were much more expensive than NeXT cubes ever were. If I recall correctly, Smalltalk on the Xerox Alto was slow, but David Patterson (of RISC fame) had a PhD student who worked on a performant version of the Smalltalk VM, and some of that work went to a spiritual successor named Self, which heavily influenced the design of the Java VM. I remember Java feeling slow in the mid-1990s, but that might be due to bad memories of Java applets rather than a fundamental problem with Java VMs.

This is sad news. I was born in 1989 and thus I don’t remember Cringely’s articles, but I remember watching Triumph of the Nerds in the mid-2000s. I love that documentary; it’s a nice capture of the early days of personal computing and of Silicon Valley culture of the era. I wish there were a follow-up that covered the 2000s and beyond, particularly the rise of social media, the gig economy (Uber, Lyft, AirBNB), cloud computing, big data, cryptocurrency, and generative AI.

Amazingly, this doesn’t look like slop to me, unlike the slop articles and images I’m flooded with on the Web these days. The world of Idiocracy actually had some taste, unlike the timeline we’re on today.

I had the same experience; grad school is the best preparation for grad school. I had wonderful experiences participating in NSF Research Experiences for Undergraduate projects during two of my summers as an undergrad, and I also had a wonderful eight-month internship in a research lab at a major Japanese IT company, but grad school was a different beast, even though I was a grad student at the place I did my NSF REUs. The internships felt like the “demo version” of the real thing.

Come to think of it, I had similar whiplash in industry of full-time jobs being far more stressful than my internships, which I now realize were more-or-less “demo versions.”

To make a long story short, after grad school and working as a researcher in industry for nearly a decade before becoming a professor at a community college (where the emphasis is teaching), I learned that I love doing research as a hobby but not as a career, where there are many pressures.

I call it the “it’s not delivery—it’s DiGiorno” construct. I generally tune out when I see this construct, which is so prevalent on LinkedIn these days, which is inundated with clichéd writing.

Unfortunately the price hikes are affecting the entire industry, and it's not just high-end rigs for gaming and local LLMs. Game consoles and even the Raspberry Pi have gone up in price. In fact, Apple was one of the last holdouts; it didn't start raising prices until June.

I paid $96 for 32 GB of DDR5 RAM for my Framework 13 laptop back in December 2023. When I checked a few months ago, the same RAM cost around $400; I don't know what prices are like now.

I agree. I would switch to an open source spiritual successor to even Tiger in a heartbeat if it were available.

The difficult part is replicating the ecosystem of applications. From a technological standpoint I would prefer a modern Linux distribution with KDE or Cinnamon over using 2005-era Mac OS X Tiger, though I’d gladly use a version of Tiger that was retrofitted with a modern Darwin core, modern libraries for security purposes, and an up-to-date web browser. However, Linux does not have Microsoft Office, the Adobe Creative Suite, Preview.app, Keynote, Dictionary.app, and other pleasant Mac apps. The FOSS alternatives are not as polished. Even venerable Office 2008 for Mac feels nicer than LibreOffice, and it’s compatible with the XML-based Office formats.

I guess for now I could run Tiger in an emulator, though admittedly current operating systems serve my needs, even if I prefer the polish of Tiger and Snow Leopard. I get really good performance running PowerPC Tiger inside QEMU on my base M4 Mac Mini and on my Ryzen 9 3900 desktop, definitely not as snappy as running Tiger on actual contemporary PowerPC hardware, but fast enough to actually be productive.

This summer I’ve been investigating agentic coding with local LLMs, and while I’m far from an expert, one thought that has been on my mind is leveraging techniques from “old-school” AI such as heuristic search to guide agents when it comes to planning. The use of Datalog in this article resonates with me, since logic programming was a major part of old-fashioned symbolic AI. I’m very curious about this combination of “old-school” AI and LLMs.

I'd love for Apple to bring back the Mac Pro, but Apple hasn't had an affordable Mac Pro tower since the 2012 model. Apple got rid of the tower in 2013 in favor of the "trash can" model. When the trash can languished for years without updates due to limitations with its design, Apple brought back the tower Mac Pro in 2019, but at a dramatically higher entry MSRP, which priced out many users of tower and trash can Mac Pros (I have a "trash can" and was very disappointed by the 2019 Mac Pro's entry MSRP). The introduction of the Mac Studio revealed a large price gap between similarly-specified Mac Studio and Mac Pro models. Given the lack of expandable RAM in Apple Silicon Mac Pro models, and also given the lack of support for NVIDIA GPUs in macOS, there weren't a lot of customers for Apple Silicon Mac Pros when the Mac Studio was available at a lower price, and thus the Mac Pro got discontinued.

In many ways, the Mac Studio is the spiritual successor to the trash can Mac Pro. Apple Silicon's architecture solved the thermal problems that limited the trash can Mac Pro.

Last week I purchased a MacBook Pro M1 Max from www.macsales.com, which is owned by Other World Computer, a maker of Mac peripherals. My computer comes with a 1-year warranty. It was my first time purchasing from them.

Yup. Yesterday it was $799 for the base Mac Mini M4, a $200 increase from the original MSRP before June’s price hikes.

I paid $499 for my M4 back in April with the academic discount since I predicted the next generation base model would cost more. Sadly my prediction was correct: $599 to $899 is steep. I understand why; the prices of DIMMs, GPUs, and SSDs have also skyrocketed. But $899 for a base Mac Mini is painful, even if the alternatives on the current market are also painfully expensive.

Russ Cox is one of the persons behind Xv6 (https://pdos.csail.mit.edu/6.828/2011/xv6.html), an x86 port of Version 6 Unix, which is the version of Unix described in the famous Lion’s book (https://en.wikipedia.org/wiki/A_Commentary_on_the_UNIX_Opera...). Xv6 was prepared for MIT’s 6.828, an undergraduate operating systems course. In this syllabus from 2018 (https://pdos.csail.mit.edu/6.828/2018/overview.html), students are expected to write significant subsystems in Xv6.

I think many good computer science programs offer a similar course where students make modifications to an open source Unix-like OS like Xv6, Minix, Linux, or one of the BSDs. I took such a course at Cal Poly San Luis Obispo in 2008.

However, I don’t know of any undergraduate programs where students are required to code an entire OS from the boot loader all the way up to basic drivers. Building an OS is part of the final chapter of From NAND to Tetris, so perhaps there might be some universities out there that do require their students to write an OS from scratch as part of the larger curriculum.

I’m not a political conservative and I personally wouldn’t move to Idaho, but Idaho has a lot of appeal. There are plenty of conservatives who used to live in California who moved to Idaho in the past decade due to a combination of California’s high housing prices and wanting to move to a more politically compatible state.

Additionally, Idaho has a lot of beautiful natural scenery.

I’m quite optimistic about the long-term future of local LLMs for privacy and cost control reasons. An LLM running on my own hardware, even if it’s not a laptop but a home server, is one where I don’t need to worry about token limits, token fees, privacy, and “rug-pulling” from the vendor.

In the short term, the big challenge is being able to afford hardware that can run a ~30B model. Last month I got to experiment with LLMs on a NVIDIA RTX 6000 Ada Generation as a visiting researcher during my summer break. I see the power of local LLMs for agentic coding; they’re no Claude, but they are quite useful. I wish I had gotten into local LLMs before hardware has gotten prohibitively expensive and in some cases unavailable; Apple discontinued certain Mac Minis and Mac Studios with high amounts of RAM due to the RAM shortage.

Hopefully high RAM prices don’t become a new normal, though the next year or two doesn’t look good.

I also enjoy my Blackwings, though my normal pencil of choice is the Mitsubishi Hi-Uni HB, which is cheaper than the Blackwing but is still an awesome pencil. I think my supply of pencils will last at least a decade. I take a lot of handwritten notes when doing research, and I prefer pencil over pen when working in computer science and math.

In some areas like the San Francisco Bay Area, schools would have to pay astronomically more in order for teachers to compete against software engineers. I’m a community college professor in the Bay Area making a low six-figure salary, but even a townhouse in the neighborhood my college is in is easily $1 million for a two-bedroom, and rent for a two-bedroom apartment is easily $3500/mo. A modest salary boost can help me rent a two-bedroom apartment, but if I hope to buy closer to work, my salary would need to double, and there’s still the challenge of saving for the down payment, since there’s no such thing as RSUs in a not-for-profit institution.

At least teacher housing provides a way for teachers to obtain secure housing closer to work without having to directly compete against highly-paid corporate workers with RSUs.

I want 2006-era Facebook. No algorithmic feed. No viral posts. No AI slop. Just updates from my friends and acquaintances.

There were exceptions, and I use “were” and not “are” because I don’t know any modern Silicon Valley companies that value loyalty, though I know some people at Apple and Google with long tenures.

Hewlett-Packard under its founders believed in loyalty, and even had a philosophy dubbed The HP Way (https://en.wikipedia.org/wiki/The_HP_Way) that seem like an interesting cross between some of the positive aspects of working for a Japanese company (loyalty, sense of care by the company) and the positives of working in Silicon Valley (flexible work hours, profit sharing), plus some other ideas (Friday afternoons devoted to personal projects).

This philosophy was not commonplace even in Silicon Valley at the time; the founders lamented “hire and fire” companies in Silicon Valley. Eventually HP abandoned the HP Way once HP started hiring outsiders as CEOs instead of hiring from within, beginning with Carly Fiorina.

Maybe I’m looking through rose-colored glasses, but while people need money, there was a time where there was more idealism from the founders and the employees. Steve Jobs could’ve quit business in 1985 after his failed coup; he was rich enough to never have to work again. Yet he started NeXT, and he was able to get people to join his company.

I feel we’re in a timeline in Silicon Valley where there’s little idealism. Employees seek money, and who can blame them when a two-bedroom apartment can easily rent for $4000/mo or more, and when a house is easily $2 million? Leaders seek not only money, but power, and their desires for power have been more blatant in the past year.

Where are the Steve Wozniak and Gary Kildall figures of Silicon Valley these days?

I agree regarding the West Coast in general. Seattle and Los Angeles are also similarly expensive, and even Sacramento (my hometown) isn’t much of an improvement. A few months ago I was floored to see a $13 burrito at a place that was just $8 pre-pandemic. Sacramento is not inundated with rich people like the Bay Area, so what gives? The average Sacramento resident can’t afford those prices.

I went to St. Louis, Missouri on a business trip two years ago and was pleasantly shocked by the food prices there. I felt like I entered a time machine to 2016 California.

Regarding sushi in Japan, prices run the gamut from cheap conveyor-belt sushi chains (like Hama Sushi and Sushiro) and grocery/convenience store sushi (which is surprisingly good here) to ultra-expensive, reservation-only restaurants with sushi served by world-renowned chefs, and everything in between. I’m not an expert on sushi, but I find that $5 grocery store sushi in Japan often beats what costs $20 back in the Bay Area. To be fair, though, the Bay Area has many excellent sushi restaurants, and while it’s more expensive in the Bay Area than Japan for the same quality, the counterargument is a meal at a Bay Area restaurant is much cheaper than a flight to Japan. I just happen to be fortunate enough to go to Japan twice a year, so I just wait until I’m in Japan to have sushi.

I don’t know where you live, but in the Bay Area prices have exploded since COVID. I remember a decade ago when it was still possible to get a super burrito for under $10; in fact, the price was $7.75 for a carne asada super burrito at El Farolito in San Francisco. A burger was under $10, a deli sandwich was about $8-9, and tacos were around $1.50 at Taco Bell and no more than $3 at a proper taqueria. The prices weren’t cheap, but they never felt bank-breaking. Unfortunately prices have skyrocketed since COVID. Unfortunately my income hasn’t matched; I made the individually fulfilling but poor financial decision of pursuing a research career and not working for OpenAI/Anthropic/FAANG/NVIDIA. Thus, two years ago I quit eating out in the Bay Area, except when socializing. It’s sad making six figures and feeling priced out of dining out, just like I’m priced out of housing and RAM since I can’t compete with Big Tech. I now cook, which I guess is healthier but is also time-consuming.

Japan, on the other hand, is heaven for those with US dollars. A bowl of ramen is around ¥1,100 here (I’m here on a research visit), and they generally taste amazing! Even a Big Mac meal from McDonald’s is far cheaper than in the Bay Area.

Additionally, as a former industry researcher myself who now teaches at a community college, there is less research freedom in industry, especially with very free places like 1970-era Bell Labs and Xerox PARC half a century in the rear view mirror. Focused industry research is definitely valuable, but there seems to be fewer places in modern society that allow a researcher to be driven primarily by curiosity.

I’m an noob to Lean myself, but my best explanation is that asserts are checks that happen at runtime. Imagine compile-time asserts, where the compiler is able to determine the validity of assertions without needing to run the program. Formal verification techniques make this possible. In Lean, it is possible to write a proof that, if true, guarantees that the code works.

I’m out of the loop these days in systems research since I largely focus on programming languages and AI these days (though I still love systems) and I treat research more as a side hobby rather than a full-fledged career. It’s disappointing to hear about toxic systems labs. There’s also the “funding-or-perish” and “publish-or-perish” pressures of academia. This is one of the reasons why I teach at a community college, where 8 months of the year I focus on teaching, leaving me 4 months of break per year where I could do research without having to worry about my tenure chances or about funding, though it would be nice to be able to pay some students to help with research projects, and it would also be nice to have the funds to buy expensive equipment such as GPUs with large amounts of RAM.