HNHacker News
TopNewBestAskShowJobs

segfall

396 karma · joined July 30, 2021

submissionscomments
segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
AWS Billing reported that I was charged $28.43 over the course of 7 hours, for a total transfer of 306.916 GB. I believe this capture the ~25k page views the site received before I switched over to another static file host.

Lesson learned for me. Thanks to everyone in this thread for saving my wallet.

segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
Thank you. Unfortunately, you'd have to write that stylesheet from scratch. For this demo, I just wrote them in the Svelte components.
segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
sql.js is in memory, but the database file is only partially loaded from the static file host.
segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
Great catch, simonw (and everyone else in this thread).

I've reviewed some other static hosts and settled on wasabi for now. When my S3 bill rolls in, I'll reply to this comment with the damage. A scary number will help others avoid my oversight.

segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
For added fun, you can try switching between different databases:

http://static.wiki/en/United_States

http://static.wiki/zh/美国

http://static.wiki/fr/United_States

http://static.wiki/enwikiquote/United_States

http://static.wiki/simple/United_States

segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
Thank you, Phiresky. My little side project only exists because of your work.
segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
Yep, you're right. The wikitext -> markdown conversion strips out some links, noticeably on the disambiguation pages, for an unclear reason.
segfall··on Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file
OP here. Happy to take questions, with the caveat that my work here is mostly glue.

Some background: https://github.com/segfall/static-wiki

The datasets I generated: https://www.kaggle.com/segfall/markdownlike-wikipedia-dumps-...

The original author of this approach: https://news.ycombinator.com/item?id=28013514