lorey

<div class="hidden">Actual name: Batman</div>

Most explicit CSS rules allow you to spot this, implicit rules won't and possibly can't.

Personally, this feels like the direction scraping should move into. From defining how to extract, to defining what to extract. But we're nowhere near that (yet).

A few other thoughts from someone who did his best to implement something similar:

1) I'm afraid this is not even close to cost-effective yet. One CSS rule vs. a whole LLM. A first step could be moving the LLM to the client side, reducing costs and latency.

2) As with every other LLM-based approach so far, this will just hallucinate results if it's not able to scrape the desired information.

3) I feel that providing the model with a few examples could be highly beneficial, e.g. /person1.html -> name: Peter, /person2.html -> name: Janet. When doing this, I tried my best at defining meaningful interfaces.

4) Scraping has more edge-cases than one can imagine. One example being nested lists or dicts or mixes thereof. See the test cases in my repo. This is where many libraries/services already fail.

If anyone wants to check out my (statistical) attempt to automatically build a scraper by defining just the desired results: https://github.com/lorey/mlscraper

Isn't this what contracts were created for?

Roses are red, their logo is blue. They scrape your data, but sue if you do.

Hey Jörg, long time follower. Really like how the landing page turned out! Congratulations. For anyone to lazy: https://www.paydevs.com/

I feel the landing page needs a minimum example of how to get value, not only how to install it. Think stripe, that to this day shows how to integrate it with a few lines of code. When looking at the tutorial, I also feel a lot of steps necessary to check it out. Why not provide basic functionality out of the box or a quick start to git clone?

On another note, having built startup search myself [1], I think the demo does not do the technology justice [2]. People will most likely judge the product based on it, but the offered startup search does not work as expected, e.g. when trying "food delivery". Feel free to get in touch, will happily provide some data :)

[1] https://startupradar.co

[2] https://demo.qdrant.tech/

related: "choose boring technology" https://news.ycombinator.com/item?id=20323246

Which is in turn a threat to the open web in general. Could not agree more.

Really interesting tool. I've always found the task of finding the right CSS selectors tiresome, so I created tool which automatically extracts the right selectors for you (and builds a scraper as well). Open Source on GitHub: https://github.com/lorey/mlscraper

I've been using a markdown-based calendar in Obsidian together with a regular Google Calendar, and so far, it's been working great for me. For markdown, I simply create interconnected notes for years, months, weeks, and days and note down goals, tasks, birthdays, and whatever happened that day (or needs to happen). For scheduling, I use regular Google Calendar events as you want to keep the benefits of invites and all.

A preview can be seen here https://github.com/lorey/obsi/tree/master/example/output/cal....

I've also created a script to automatically generate all the files for me: https://github.com/lorey/obsi

I can't provide the UI, but here's my tool to generate a calendar for Obsidian: https://github.com/lorey/obsi/

Preview here: https://github.com/lorey/obsi/tree/master/example/output/cal...

Am I the only one that thinks this blog post is not too different? A blog post about a youtube video, no additional value.

I've only looked into importing cards automatically briefly. My feeling was that keeping progress might be an issue if cards change (and thus get re-created). Do you deal with that?

any source on that?

As an example for the non-Germans here, here's N26: https://www.northdata.com/N26+Bank+GmbH,+Berlin/Amtsgericht+...

note: it shows management not ownership

There's also Kitekraft from YC:

- https://medium.com/kitekraft/kitekraft-takes-off-with-y-comb...

- https://www.kitekraft.de/

I think this is how philosophy has been for the last 2000 years...

I thought I had heard that term before. Found the following article in my bookmarks describing how to make your phone more boring and less usable on purpose. Not the same topic, still highly recommended. https://nomasters.io/posts/dumber-phone/

Also, I really like to use this minimalist launcher from time to time: https://f-droid.org/en/packages/com.jkuester.unlauncher/

These are my bookmarks for pitchdeck collections:

https://www.pitchdeckhunt.com/

https://opendeck.app/

https://angelmatch.io/pitch_decks

https://starthouse.xyz/ (down)

Thank you so much, happy to hear! There's a more versatile version coming soon :)

Will do, just haven't decided yet :)

I would love to see this as a flake8 extension. An own tool with own formatting, workflow, etc.? I'm not sure...

In case anyone wants to detect the selectors automatically, here's a small python library I wrote that does it for you: https://github.com/lorey/mlscraper

If you're interested in the mathematical background, I can highly recommend: https://medium.com/@kentbeck_7670/decisions-decisions-or-why...

The number of highly controversial opinions on this, to me, only shows that it makes sense to try to map a potential path. While the criticism in the comments here may be valid, it is mostly unconstructive and without any proper explanation or reasoning. It may not be perfect, but it's a great overview of what to look into.

Damn, I really like the design/layout. Anyone know where to find it by any chance?

Read this by chance because I usually come to HN for tech articles, but I feel exactly the same and have been recommending this ever since. One of the most thrilling and exciting stories I've read so far.

Background information: https://en.wikipedia.org/wiki/Richard_von_Weizs%C3%A4cker#Sp...

Speech in German: https://www.youtube.com/watch?v=C3ZAzpk4IbE

If you're wondering what this is:

> We, representatives of the German Federal Government, business and science communities, are striving to set up a high-performance, competitive, secure and trustworthy data infrastructure for Europe. To this end, we have drawn up the foundations for a federated, open data infrastructure based on European values, giving it the provisional pro-ject name ‘GAIA-X’.

Here's the executive summary: https://www.bmwi.de/Redaktion/EN/Publikationen/Digitale-Welt...

Author here. For many crawling/scraping projects I wanted to extract websites' linked social media profiles, e.g. facebook, twitter, or github profiles, etc. The easiest way for me was to use regular expressions to parse the urls and extract contained information. This is a collection of these regular expressions [0]. You check the urls against these regular expressions and get the platform, username, and other information from the url back. There's also a Python library [1] and an API [2].

[0] https://github.com/lorey/social-media-profiles-regexs

[1] https://github.com/lorey/socials

[2] https://github.com/lorey/socials-api