abetlen

I would add the following two numbers if you're generating realtime text or speech for human consumption:

- Human Reading Speed (English): ~250 words per minute

- Human Speaking Speed (English): ~150 words per minute

Should be treated like the Doherty Threshold [1] for generative content.

[1] https://lawsofux.com/doherty-threshold/

Also worth checking out https://github.com/saharNooby/rwkv.cpp which is based on Georgi's library and offers support for the RWKV family of models which are Apache-2.0 licensed.

You can see for yourself (assuming you have the model weights) https://github.com/abetlen/llama-cpp-python

I get around ~140 ms per token running a 13B parameter model on a thinkpad laptop with a 14 core Intel i7-9750 processor. Because it's CPU inference the initial prompt processing takes longer than on GPU so total latency is still higher than I'd like. I'm working on some caching solutions that should make this bareable for things like chat.

This is not true, GPT-3 can perform chain-of-thought reasoning through in-context learning either by one/few-shot examples or zero-shot by starting a prompt with "let's think step by step" (less reliable).

GPT-3.5 (what's being used here) is a little better at zero-shot in-context learning as it's been intstruction fine-tuned so it's only given the general format in the context.

I think you're focusing on a few narrow examples where LLMs are underperforming and generalising about the technology as a whole. This ignores the fact that Microsoft already has a succesful LLM-based product in the market with Github Copilot. It's a real tool (not a party-trick technology) that people actually pay for and use every day.

Search is one application, and it might be crap right now, but for Microsoft it only needs to provide incremental value, for Google it's life or death. Microsoft is still better positioned in both the enterprise (Azure, Office365, Teams) and developer (Github, VSCode) markets.

My go-to heuristic is three point estimation, basically a weighted average of the best, worst, and average case [0].

(Best + Worst + 4 * Average) / 6

One nice property is that it imposes a distribution that adjusts for longer tailed risks.

https://en.wikipedia.org/wiki/Three-point_estimation

If you run a Kubernetes cluster for self-hosting software or development I highly recommend setting up a Tailscale subnet router [1]. This will allow you to access any IP (pods or services) in your cluster from any of your Tailscale-connected computers. You can even configure Tailscale DNS to point to the DNS server in your cluster to connect using the service names directly ie. http://my-service.namespace.svc.cluster.local

[1] https://tailscale.com/kb/1185/kubernetes/#subnet-router

pikchr is awesome. A project I did recently was a WASM-compiled pikchr library to generate diagrams directly in the browser [1]. Here's a very early demo of a live editor you can play around with [2].

Not fully-featured yet but what I'd like to eventually do is set it up in a similar way to the mermaidjs editor [3]. They encode the entire diagram in the url. That makes it really easy to link to from markdown documents and has the nice benefit that the diagram is immutable for a given url so you don't need a backend to store anything.

[1]: https://www.npmjs.com/package/pikchr-js

[2]: https://pikchr-editor.insert-mode.dev/

[3]: https://mermaid-js.github.io/mermaid-live-editor

Cool trick, thanks for sharing. I don't get why there isn't a suitable snscanf function that takes the buffer length as an argument and returns the number of bytes parsed?

If you're getting into this stuff a great resource I found is the ZipCPU Tutorial [0] by Dan Gisselquist. The tutorial covers both Verilog design and Formal Verification methods. It uses open source tools like Verilator and SymbiYosys so getting started is pretty easy.

[0]: https://zipcpu.com/tutorial/

> I'm pretty sure you could also maintain a temp table and use some kind of "insert...where...returning" construct to squeeze that into a recursive query.

I'm not sure if this is possible in SQLite, as far as I know the WITH clause is limited to SELECT statements.

> Are they actually efficient in sqlite even for trees?

Recursive common table expressions work by adding returned rows to a queue and then performing the recursive select statement independently on each row in the queue until it's empty.

You can use WITH RECURSIVE to traverse a tree by adding the root node to the queue and recursively visiting adjacent rows until the queue is empty. This works correctly and quickly because trees have only a single path between nodes. If you try the same query on a DAG though it will return every path to a given node, you then have to perform a GROUP BY to find the shortest path outside of the recursive query. In the worst case, if you have a graph with many paths between nodes, this method is exponentially slower than a standard BFS.

The issue I found with WITH RECURSIVE queries is that they're incredibly inefficient for anything but trees. I've looked around and there doesn't seem to be any way to store a global list of visited nodes. This means that when performing a traversal of the graph the recursive query will follow all paths between two nodes.

I've actually been working on an extension to perform breadth first search queries in SQLite on general graphs [0]. The extension is actually based off of the transitive closure extension. You can use it on any existing SQLite database as long as you can wrangle your edges into a single table (real or virtual) and the node ids are integers (I'm planning on removing this constraint in the future).

[0]: https://github.com/abetlen/sqlite3-bfsvtab-ext

My brand new laptop was stolen while I was in school and I had to finish the year with a used 10-inch netbook I borrowed from a friend. Because of the tiny display and underpowered processor I had to get creative, I installed Arch plus i3 and it worked like a charm. In fact, I noticed my productivity actually went up. Eventually I got a new laptop, I switched off Arch to Ubuntu but I've been using i3 ever since.

If you're an i3 user here are a couple cool things to check out:

- scratchpad [1]: Little known i3 feature. Let's you put individual windows in a scratch workspace and then pull them up quickly over your regular workspaces. Really useful docs and things like that, I keep my time tracker in a scratch workspace.

- rofi [2]: This is a dmenu replacement with a nicer interface that lets you combine lists of commands in a single search window. I have it configured so I can search through my open windows by title or run any program from the same interface.

[1]: https://i3wm.org/docs/userguide.html#_scratchpad

[2]: https://github.com/davatorium/rofi

I may be misreading that, but isn't the table saying that 82% of residents with dementia require assitance in their daily living activities?

From having spent time volunteering at a nursing home, that 87% number is not far off from my experience. I'd add that they tended to put volunteers with people who had less severe forms of cognitive impairment.

That being said, you would occasionally meet people who seemed mentally impervious to aging, I had the pleasure to speak with an 87 year old computer programmer who was still actively using all kinds of technology. He talked to me about punch card programming accounting systems on IBM 1401's and dealing with technology skeptical secretaries who would double check every computer calculation by hand.

I'm not sure if you're from outside North America, but I don't think nursing homes here are likely to have LAN parties unless we make some big medical breathroughs in the next few decades.

According to this survey of Canadian nursing homes, around 87% of residents suffer from some kind of cognitive impairment such as dementia, alzheimer's, or stroke related trauma [1].

Even assuming that you still have any friends around by the time your in a nursing home, the odds that neither of you suffer from cognitive impairment is about 1.7% (someone let me know if that math is wrong).

[1]: https://www.cihi.ca/en/dementia-in-canada/dementia-care-acro...

They really were. Here's a version of that video that includes Lorinda Cherry writing a talking calculator by piping together 3 unix commands.

https://youtu.be/XvDZLjaCJuw?t=828

You're right. There is also Chilean town / research station that requires all residents to have their appendix removed [1].

[1]: https://en.wikipedia.org/wiki/Villa_Las_Estrellas

Fun fact, all doctors who plan to winter at an Antarctic station must have their appendix removed beforehand [1]. The reason is that they're usually the only doctor available.

[1]: https://www.antarctica.gov.au/about-antarctica/people-in-ant...

While you can use recursive common table expressions to perform graph queries, certain queries like breadth-first traversal are not very efficient for anything but trees (not even DAGs). See my post on the sqlite forum from the other day for more details

https://sqlite.org/forum/forumpost/cd5ff0a1d4?t=h

OTP = Open Telecom Platform [1], though I think the full name is avoided nowadays.

On an unrelated note, I've always been fascinated with the Erlang VM. The idea that you could hot-load modules and have multiple versions of the same module running at once seems really useful. I wonder why other runtimes haven't adopted these features?

[1] https://en.wikipedia.org/wiki/Open_Telecom_Platform

This is true, but won't help very much either. According to Google Support [1], they don't decide what is or is not fair use, leaving it up to the courts. This means the default position is that nothing is fair use unless the content creator is willing to go through the expensive process of proving it.

Funny enough, they cite a case where a content creator (h3h3 productions) was able to prove fair use in court as an example of the system working. If you look into that case though you'll see that the claimant was only an individual musician and not a big studio, and even then the legal fees were hundreds of thousands of dollars.

[1] https://support.google.com/youtube/thread/1281991?hl=en

When I was a teenager, my mom (a psychiatrist) gave me her copy of Frankl's treatise on Logotherapy titlled The Will to Meaning. That book was so over my head, but something that's always stuck with from it is: life is unconditionally meaningful, and we can discover that meaning through creative actions and experiences, as well as how we come to accept unnavoidable sufferring.

I agree that asking for reviews was a lot less common at the time (2012), but he actually says "asking for reviews ... wasn't something that came to the front of mind ... it felt dirty at the time". To me that reads like he was aware it was something apps were doing but he wasn't comfortable with it. So when the CEO of the company asked him "why aren't we higher rated in the app store?" he probably wished the answer was "they're cheating" instead of "because I feel it's unethical to ask for reviews".

I think it's telling that his first hypothesis for why Barnes & Noble was getting more positive reviews was that they were paying for them. Honestly, my first thought was "maybe they're just asking more customers". I wonder if he thought that the competitor was doing something shady because he'd been put on the spot and so his first response was instinctively the most defensive one.

The video tells the story of No Man's Sky, the buildup of unmeetable expectations prior to it's release, the online anger towards the game developers after the game's dissapointing release, and the hard work of those developers to not only fix the game post-release but also extend it.

Weird right? I learned Pascal in highschool in Ontario around 2010 as well. I remember being really frustrated at the time because we weren't learning some modern language like Java or Python. Instead we were stuck with Turbo Pascal, writing in an IDE so old it only supported 8 character long filenames. Thank god we had an amazing teacher which ended up making it one of the best CS classes I've ever taken.

Agreed, Adam mentioned many times in his screencasts that his intention for Tailwind UI was to create a source of income to keep working on these projects. I'm happy he went this route rather than a Patreon or starting to charge for content like the screencasts.

Any resources you can share that worked better for you?

Interesting, that's probably further than I was even considering.

Part of me is thinking "how upset would I really be if I went to pay for something and got an 'under construction' message at the last minute", probably not that upset. However if I thought I had paid for something and later found out the order was cancelled it would be a different story.