zulko

I made a map of Wikipedia pages and events once [1] and a key decision was: which pages to show first, and which to show only when the user zooms in?

I went with "let's show pages with the most words first". If people spent more time writing a page, it must be a great topic, right? Unfortunately, this also surfaces all the wars, crimes, accidents and controversies that generated the most coverage. These are the most active pages on Wikipedia.

[1] https://landnotes.org/?location=u0q9wgry-5&paneTab=about

Possibly not what you're asking for, but I wrote a generally-accessible intro to why it can be tricky to assemble many DNA fragments with "Golden Gate Assembly", a mainstream method which relies on short sequence overhangs. The Sidewinder method discussed in this thread aims to solve that "short overhang" problem.

https://zulko.github.io/bricks_and_scissors/posts/overhangs/

It does, why?

Nice, how does your timeline work under the hood? Does it read from wikipedia? What could be interesting in your project is to be able to compare timelines. See for instance this website specifically for comparing composer works (with timelines pre-extracted from wikipedia):

https://zulko.github.io/composer-timelines/?selectedComposer...

Yeah it would be nice if Wikipedia would host it, but it would probably require some more serious ground work so the project fits in the wikipedia ecosystem. Could be a pipeline Wikipedia -> Wikidata -> Atlas.

There are many projects that could be done with with wikipedia and LLMs, for instance "equalizing" all languages by translating all pages into all other languages where they are missing. Or, more surgically, finding which facts are reported in some languages of a page but not others, and adding these facts to all languages.

For now, it seems that wikipedia doesn't want to use generative AI to produce wikipedia pages, and that's understandable, but there may be a point where model quality will be too good to ignore.

Thank you for reporting this!

This is also very true of the events reported in Wikipedia, see this animated timeline of (a hopefully representative set of) historical events reported in Wikipedia. Is really is "Europe meets the world":

https://www.reddit.com/r/MapPorn/comments/1l3xl8x/events_fro...

I agree with others in this thread that this more probably "information-biased" than "eurocentric" on the part of the Atlas creator. Pretty sure they wish non-european history was easier to find and aggregate as it would make the project much more compelling (I certainly had this problem with https://landnotes.org/).

I am hoping LLMs will do a lot of good at bridging gaps and surfacing world historical information that didn't make it yet to centralized projects like Wikipedia.

Total plug but this year I scraped 400,000 wikipedia pages with Gemini to create landnotes.org, an atlas where you can ask "what happened in Japan in 1923":

https://landnotes.org/?location=xnd284b0-6&date=1923&strictD...

https://github.com/Zulko/landnotes

My plan has been to overlay historical map borders on top of it, like the Geacron one from this post, but they all seem to be protected by copyright - and understandably so, given the amount of work involved.

Very nice but the fonts render as white on white background on my phone which makes it difficult to read. Screenshot: https://ibb.co/29CqpTx

Fascinating, I knew about the "Wikipedia degrees of separation" and whe wikigame (https://www.thewikigame.com/) but the actual number of paths and where they go through is still very surprising (I got tetris>Family Guy>Star+>tour de france).

If anyone is looking to start similar projects, I open-sourced a library to convert the wikipedia dump into a simpler format, along with a bunch of parsers: https://github.com/Zulko/wiki_dump_extractor . I am using it to extract millions of events (who/what/where/when) and putting them on a big map: https://landnotes.org/?location=u07ffpb1-6&date=1548&strictD...

You should try the words from previous days (clicking on the date below the title). So far it’s been pretty random how many tries someone will need to find a word, the same person who needed 2 tries one day might need 10 tries another time. Just like wordle, you might get lucky or unlucky on your first guesses.

In terms of the music being played, piano rolls can be different from "normal piano music" because it's not played live by a real human, so it can have complex parts with full chords, additional voices, all with perfect rhythm and no wrong notes. This can be very compelling when well executed on the right songs (and it can also sound "mechanical" on others).

There isn't a huge market for piano roll recordings, and these recordings are rare. It's a niche topic that can attract

- Older people who have known the time piano rolls (say, until the 1950s)

- People nostagic of old times in general (in particular the 1910s-1940s), the age of early jazz with stride piano and early Broadway.

- Music scholars, because some of these rolls are of historical/musical importance, in particular those "recorded" by George Gershwin or Fats Waller and other big names. A lot of material exists only as piano rolls.

For the example of the Gershwin CD I posted above, it was produced by musicologist Artis Wodehouse [1] in parnership with the yamaha disklavier pianos iirc [2], so my guess is this was a passion project above all, with a bit of Yamaha marketing.

[1] https://www.artiswodehouse.com/biography/ [2] https://usa.yamaha.com/products/musical_instruments/pianos/d...

When I was about 10 I picked my first ever CD at a music shop, and it was a recording of the Gershwin piano rolls, because the cover photo caught my eye [1]. I didn't really understand what I was listening to, I assumed "piano roll" was a musical genre, like "rock'n'roll", until years later when my English became good enough to read the CD's booklet.

It was also a time when all these midi files started being available, like the 6000 rolls from Terry Smythe [2], and I figured out transcribing these could be a good way to learn old-school Jazz, which is otherwise difficult to find as sheet music.

[1] https://www.youtube.com/watch?v=BX9MCyO6smk

[2] https://archive.org/details/terrysmythe.ca-archive/mp3s/Ampi...

This used to be one of my main hobbies, I listened to thousands of these and I am super grateful to the people scanning and hosting these collections.

Some software I wrote for piano roll analysis and transcription:

- Unroll: https://zulko.github.io/unroll-online/ - upload a piano roll midi file and have it quantized and converted to lilypond sheet music. More about the process in this blog: https://zulko.github.io/blog/2014/02/12/transcribing-piano-r...

- Pianola: https://zulko.github.io/pianola/ - upload a piano roll midi file, and it plays with the piano roll and keyboard animation (you can zoom on some parts, slow down etc).

Some transcriptions made with these tools:

- Hindustan: https://github.com/Zulko/sheet-music--hindustan

- Gershwin - Sweet and Lowdown: https://github.com/Zulko/sheet-music--Gershwin-sweet-and-low...

- Gershwin - Limehouse Nights: https://github.com/Zulko/-sheet-music--Gerhswin-Limehouse-Ni...

Insulin today is produced cheaply in genetically modified microbes, this is the technology that Chargaff is alluding to, which was first succesful 2 years after his letter, in 1978.

In his 1976 essay on (or against) genetic engineering [1] Erwin Chargaff wrote "But screams and empty promises fill the air: Don't you want cheap insulin? (...) And how about a green man synthesizing his nourishment: 10 minutes in the sun for breakfast, 30 minutes for lunch, and 1 hour for dinner?" Nice to see that scientists are actually trying.

[1] https://www.science.org/doi/10.1126/science.11643312

Oooh I had missed that thanks!

> reduce it do only sections that are likely to be relevant (eg. "Life and career")

True but I also managed to do this from HTML. I tried getting pages wikitext through the API but couldn't find how to.

Just querying the HTML page was less friction and fast enough that I didn't need a dump (although when AI becomes cheap enough, there is probably a lot of things to do from a wikipedia dump!).

One advantage of using online wikipedia instead of a dump is that I have a pipeline on Github Actions where I just enter a composer name and it automagically scrapes the web and adds the composer to the database (takes exactly one minute from the click of the button!).

Same experience here. Been building a classical music database [1] where historical and composer life events are scraped off wikipedia by asking ChatGPT to extract lists of `[{event, year, location}, ...]` from biographies.

- Using chatgpt-mini was the only cheap option, worked well (although I have a feeling it's dumbing down these days) and made it virtually free.

- Just extracting the webpage text from HTML, with `BeautifulSoup(html).text` slashes the number of tokens (but can be risky when dealing with complex tables)

- At some point I needed to scrape ~10,000 pages that have the same format and it was much more efficient speed-wise and price-wise to provide ChatGPT with the HTML once and say "write some python code that extracts data", then apply that code to the 10,000 pages. I'm thinking a very smart GPT-based web parser could do that, with dynamically generated scraping methods.

- Finally because this article mentions tables, Pandas has a very nice feature `from_html("http:/the-website.com")` that will detect and parse all tables on a page. But the article does a good job pointing at websites where the method would fail because the tables don't use `<table/>`

[1] https://github.com/Zulko/composer-timelines

If you like this type of maze, I used to make similar ones based on traffic-light systems.

http://zulko.github.io/blog/2014/04/27/viennese-mazes-what-t...

There is a 2022 french movie with the same premise (an elderly woman on her way to a hospice asks the taxi driver to drive her through Paris):

https://www.imdb.com/title/tt14586118/

A few years ago I started an alternative to PrinceXML called ReLaXed.js [1], it's always been sufficient for my reports but it may lack some pagination/layout features that Paged.js may have as they seem to have given this much more thoughts (still wrapping my head around whether paged.js could be "plugged into" Relaxed).

[1] https://github.com/RelaxedJS/ReLaXed

Edinburgh Genome Foundry | Senior Software Engineer - Synthetic Biology and Automation | Edinburgh, UK | ONSITE | Full-time

We are looking for a computational software engineer to join one of the largest automated platforms for synthetic biology globally.

The Foundry is a unique facility at the University of Edinburgh specialized in large-scale genetic engineering projects for academic and industrial research. We are automating the assembly of genetic constructs up to chromosome size, with an ambition to deliver custom DNA seven days a week with minimal human intervention from order to shipping.

You will use your computational and organizational skills to help us develop the services that orchestrate our platform, making it smarter, faster and more reliable. This is a great opportunity to contribute to exciting research projects (e.g. in gene and cell therapy, metabolic engineering) while developing expertise in Laboratory Information Management Systems, software architecture for automation, computer-aided DNA design and manufacturing, and web development.

Your work will be mainly open-source and have visibility in the Synthetic Biology and Laboratory Automation communities, enabling biological projects at an unprecedented scale!

Duration: fixed term until November 2020 in the first instance. Full job description and application (closing May 7th 2019): https://www.vacancies.ed.ac.uk/pls/corehrrecruit/erq_jobspec...

EGF website: https://genomefoundry.org

EGF Software Projects: https://edinburgh-genome-foundry.github.io

I believe this could work but the website lacks samples (is it better than wikipedia, youtube videos and other tutorials ?) and a proof of authority (what makes you guys experts in the field). For 500$ you need to have a better idea of what you are buying.

Edinburgh Genome Foundry | Senior Software Engineer - Synthetic Biology and Automation | Edinburgh, UK | ONSITE | Full-time

We are looking for a computational software engineer or computational biologist to join one of the largest automated platforms for synthetic biology globally.

The Foundry is a unique facility at the University of Edinburgh specialized in large-scale genetic engineering projects for academic and industrial research. We are automating the assembly of genetic constructs up to chromosome size, with an ambition to deliver custom DNA seven days a week with minimal human intervention from order to shipping.

You will help us develop the services that orchestrate our platform, making it smarter, faster and more reliable. This is a great opportunity to contribute to exciting research projects (e.g. in gene and cell therapy, metabolic engineering) while developing expertise in Laboratory Information Management Systems, software architecture for automation, computer-aided DNA design and manufacturing, and web development.

Your work will be mainly open-source and have visibility in the Synthetic Biology and Laboratory Automation communities, enabling biological projects at an unprecedented scale!

Duration: Until Nov. 2020 in the first instance.

Full job description and application: https://www.vacancies.ed.ac.uk/pls/corehrrecruit/erq_jobspec...

EGF website: https://genomefoundry.org

EGF Software Projects: https://edinburgh-genome-foundry.github.io

Edinburgh Genome Foundry | Senior Software Engineer - Synthetic Biology and Automation | Edinburgh, UK | ONSITE | Full-time

We are looking for a computational software engineer to join one of the largest automated platforms for synthetic biology globally.

The Foundry is a unique facility at the University of Edinburgh specialized in large-scale genetic engineering projects for academic and industrial research. We are automating the assembly of genetic constructs up to chromosome size, with an ambition to deliver custom DNA seven days a week with minimal human intervention from order to shipping.

You will use your computational and organizational skills to help us develop the services that orchestrate our platform, making it smarter, faster and more reliable. This is a great opportunity to contribute to exciting research projects (e.g. in gene and cell therapy, metabolic engineering) while developing expertise in Laboratory Information Management Systems, software architecture for automation, computer-aided DNA design and manufacturing, and web development.

Your work will be mainly open-source and have visibility in the Synthetic Biology and Laboratory Automation communities, enabling biological projects at an unprecedented scale!

Duration: fixed term for 2 years in the first instance. Full job description and application: https://www.vacancies.ed.ac.uk/pls/corehrrecruit/erq_jobspec...

EGF website: https://genomefoundry.org

EGF Software Projects: https://edinburgh-genome-foundry.github.io

Author here (very happy to see this on HN). I have done exactly that (automated "next slide" and audio playing at specific slides) but haven't contributed back to the repo yet. It took just a few lines of code in my slideshow, without requiring to modify the library (one example of why I claim that it is "hackable").

I defined a Vue component "next-step(after=5)" which triggers a next slide after 5 seconds, and an "audio(source, timeSegment)" component which just plays an audio. Then your slideshow looks like that:

  slide(steps=2)
    next-step(:after="2" v-if="step === 2")
    audio("hello.mp3")
    
    h1 Slide Title

Maybe not as good as a built-in solution, but good enough and very easy to tweak for everyone's needs.

Your PDF will automatically reference the h1, h2, etc, as sections and subsections in your PDF (i.e. you should see them in the sidebar's document tree in PDF viewers that have such a feature). You can also certainly generate a TOC using some javascript framework. For a quick google search I found this:

https://css-tricks.com/automatic-table-of-contents/

You can include Javascript in the pages, but it won't be animated in the end.

Maybe you would be interested by this project to make slideshows with Pug/SCSS/Vue.JS. There you can make plenty of animations:

https://zulko.github.io/eaglejs-demo/#/

Hyphenation can be tuned via CSS but I have never been happy with it:

https://www.w3schools.com/cssref/css3_pr_word-break.asp

From what I remember LaTeX has better algorithms, both in how to distribute words between lines, and in knowing where in a word it is ok to cut.