JZL003

Hah I was thinking the same thing

I've been using nix for my personal server and it's a lot of fun.

And I don't mind reading 'code as docs' in general but it's pretty much required here. Mostly the option names are pretty understandable, and it's helpful to see in the code exactly what it does, not just a human version.

But truly invaluable is sourcegraph (I guess github search is better but I still don't like it) so you can search through other peoples config https://sourcegraph.com/search?q=context:global+file:.*nix+B...

On the other hand, I wanted to have emacs compiled with sqlite (needed for magit) and it took me hours to get working, then had to keep modifying to pin the version so it didn't recompile constantly

I know it's probably in rough shape but I'd like to use this myself, even if it takes a lot of code to modify. I guess I'd need an android app too tho

Copy on write is really nice, especially when I often face a very very large read-only matrix (200+GB) and want to do some embarrassingly parallel processes on subsets of it. I haven't found a language which makes it as easy, not python (although not unexpected), not Julia even

This reminds me of a thing cory doctorow talks about how tech companies control the narrative to focus on fun sexy problems while they have fundamental problems which expose the lie.

For example uber/self driving cars always talking about the trolley problem, as if the current (or near future) problem is that self-driving cars are so good they have to choose which one. Not the current very difficult problem of getting confused by traffic cones.

I know these problems are more fun to talk about and also could be a problem at some point, but we have some current problems about training models separate from what happens if they become smarter than humans

Sort of related but I've really enjoyed setting up a urlwatch - https://urlwatch.readthedocs.io/en/latest/ - setup. Especially once you get past the boilerplate of pupetteer and can boot up a chrome instance to scrape websites with javascript. I start to feel like I'm taking control of the web in a push not pull way

There's an amazing power to just monitor websites with no sweat and skim in the morning

- new job openings for companies you like

- new job openings/closings from your current company

- products you're waiting to go on sale/back in stock/available refurbished (got 70% some nice headphones)

- covid sewage stats, if you want to know about spikes

- apartment listings

- github releases you care a lot about (<3 yabai)

- legal-ese for critical websites

Personally I rent a little digital ocean droplet for $5 since I also self host a RSS reader, personal telegram bot, etc (and it's very useful to set up little http site for experimentation) but could do it on your laptop since it doesn't have to run every day at the same time

Yeah I'm too far commmitted. I hope they'll let us buy the number off them

But my theory is that too many google execs use it so it'll hurt them too if it gets canned. They did update the app semi recently

This is not the same, and might not get house-mate approval, but I like writing little notes in-situ where I need them. For laundry, little checklists of things I forget to wash (bathroom rugs smh), how long each cycle takes. Houseplants can keep when they were last watered nearby. Only works for high-use areas but still useful

I agree it's so good. Either the audiobook or run through text to speech and it's still ongoing

I know it depends on the analysis, but I often am doing somewhat embarassingly parallel things. So just knowing GNU parallel for mid-scale things (and R/python basically parallelism, although shared memory is a bear), and how to temporarily scale across the cloud to like 500 core, is huge.

It's not that i think anything is wrong, but isn't it strange there's no jump I see in WFH comments during the pandemic when everyone was doing it

Yeah me too, I've scraped libby pretty easily but didn't know there was an API.

A niche thing which can affect stuff is some libraries only show their full catalog if you're logged in (and potentially which partner library you're a part of, if the library offers multiple). I don't remember if I just read this in their documentation or when scraping, found at least a few difference so logged in first

Also 1M one time does go very fast compared to GCP's 1M per month

If you did give a pay as you go, price per character I'd be very interested. I know small-medium scale personal use isn't necessarily your target but I turn a lot of audiobooks into audio. Even on big months it's only 1-1.5 million characters

Or even if I could buy 1 month and then use those credits over multiple successive months I would really start considering it. It's nicer on my ears than even GCP's new neural voices, and I've listened to over 1k hours over the past year or two

https://iiif.io/get-started/talks/ Cool, although each museum hosting individually and encoraging direct linking means people won't have backups so if one of these small museums goes down, all the links go down. It'd be cool if they had a copy, or stored globally with all the high powered funding

(Even if it's stored in a backup place, the direct links mean if the DNS/server lapses, it all goes down?)

No! I use it constantly because in normal gmail I can only have like 5 versions open ( I have a lot of tabs) before my mac pro explodes. But for HTML gmail I can leave emails open for reference without exploding.

I use it so often, it loads so fast for multiple email accounts, instant refresh, it was so good

I use more scientific computing and I've found it amazingly, surprisingly good. Academic programming has some quirks where they might use a single character variable name, passed along to multiple functions. So if I'm 3 functions deep, I have a hard time tracking it - it gets it very often.

Even some versions of "I want to do this, how would I modify the code to make it happen" works pretty good

It's worth looking at julia's JuMP, I find it very ergonomic, fast, and easy to try differnet LP solvers

Also https://neos-server.org/neos/solvers/index.html is an amazing resource I don't see used often. These solvers are very expensive and it's not obvious which solver will solve a given problem fastest (if it supports the problem's type) because it depends on which heuristics they use. So you can submit example (medium-sized) problems here, try out a bunch, and get your company/buy the one which you know works best

Me too! It's fun, when an impactful event happens in the story and it's fixed

A serial fiction book I read has only text but I still want an audio book. It takes a little bit to get used to but with text to speech, your mind figures it out and starts to put emotion in the monotone voice. It's not as good as a real narrator but Google's text to speech is maybe 60-70% as doable

Yes! Me too, there's a whole world out there and it's codified in bullet-proof programs like CPLEX or gurobi. Sadly they are expensive and it's not obvious if a given problem's size and complexity will tailor itself to some of their heuristics.

Have you been doing it by hand or using a big program?

You can have nice foreign function interface between R->Julia and Julia ->R. If you're already happy pulling out slow functions into RCpp, then maybe there's no speed benefit. But there are some very nice, very fast libraries in Julia, where if you have a tight inner loop, it could be worth looking into

It reads and writes a lot like python (but nicer IMO), I don't think the learning curve is immense to try it for small optimizations. And it's also not unreadable so other people can verify your code

You can use the email alerts from banks/credit cards to automatically fill out purchase information across all credit cards. Google pub sub let's you get notified when it happens within milliseconds and then you just have to parse the emails

It's the best way I've found to get a cross company 'api' for purchases

Also chat bots like telegram will work well this

I don't even mind (although it bothers me that python doesn't have pipes ala R, or any sort of nice lambda syntax) but I wonder how much work has been put to take python-like dialect and hammer it over and over into being fast

NumPy giving a whole other set of verbs you have to memorize to just wrap around C, then JAX+etc which take that subset and make it faster, then the pypy+pyston to rewrite big parts, with numba to do individual functions if you really want for loops. I know it's an over simplification and it's all pretty amazing work, but it's exhausting to me how many companies and people have put work into taking a fast language and trying to make subsets fast

yeah I've wondered too. Although I think for me I just feel better when it's colder

I did debate writing how 'headache'y I felt periodically with timestamps (e.g. make a telegram bot -> google sheet) and then add in some weather data and see if it was actually correlated if I plotted them

Also meteograms are a nice consise way of getting a sense of the weather

https://www.meteoblue.com/en/weather/forecast/meteogramweb/t...

When I look at science data I don't look at lists of numbers, I don't know why that's always how weather is presented visually

For any other person who wants more advanced weather, I like meteo-blue's multimodel graphs https://www.meteoblue.com/en/weather/forecast/multimodel/tex... Much more information dense, if less pretty but it's helpful to see a bunch of models to get a sense of the uncertainty. It's what meteorologists I know do often, and you can quickly learn which models are short vs long-term and uncheck them

I do wish I could make my own graphs but parsing all the different weather models is a pain, maybe there's some API I don't know about or an open source consistent parser

did you find a weather API you found to be particularly accurate?

Also like O(N^2) is surprisingly doable even for very big N. Maybe I shouldn't have been surprised, but another problem was N^2 for a pretty big N (100k) with a dynamic programming. I was worried it would never finish

But making a 100k by 100k matrix in julia really isn't so bad (and can use memoization), just rent a big machine on GCP for an hour or two, and it chewed through it fast

I totally agree with this, both the part about having to be clever with the model implementation and how fast and usable it is. For my own research, it ended up that it was a type of set cover problem. To me it felt large, ended up with almost a million non-zero values to optimize (and had to do 10k of them), but the state of the art is so fast. CPLEX optimizes them in like 3 seconds and can tell me that it's provably optimal. Even open source solvers (HiGHS) solves in like 10 seconds

It is true that when I phrased the problem wrong (with multiplication not addition), it was taking forever, and I had to mess around a long time to figure it out. ChatGPT actually sorta helped a little

But it does feel like magic and not used enough. I think Julia JuMP goes a long way towards just expressing the problem easily. It might take more work/tricks to get it to work (big-M tricks are crazy to me) but at least as a first pass if it just works is worth a try, I think

And to be fair, I really wanted it to be provably optimal for the fun of it. But when I set even a 10 minute timelimit, it got a really good solution across the board much faster than I could have found with manual programming