gaojiuli

the default timeout is 30s, please refresh it and try again. I will quickly iterate this system to make it stable and reliable.

this tool is based on ChatGPT

Thank you! Do you have any suggestions on this project?

This is weird. I'm using certbot for the SSL.

You can also try to use http rather than https.

You can try to add any "list page" to have a try.

The purpose of this project is making all web sites support RSS. The Feed43 needs users knows how to configure the rule, and the rule needs to be updates when the origin web site changed.

That's why we created this tool

A library letting any web site provide APIs.

I wonder about that too.

Could you add supporting for json, xml, excel, etc.

What's the advantages of Sukhoi compared to the following.

Scrapy: https://github.com/scrapy/scrapy PySpider: https://github.com/binux/pyspider Gain: https://github.com/gaojiuli/gain

When crawling online articles such as news, blogs, etc. I want to save them in markdown files but not databases. For instance, I can download my wordpress blogs as markdown files with requests and tomd in a single python script without configering another language environment.

Tomd is a python tool. Pandoc is a haskell tool.

Tomd has the ability of converting the HTML that converted from markdown. If a HTML can't be described by markdown, tomd can't convert it.

Maybe the number of lines of code is less. And the logic of tomd is easier to understand.

When crawling online articles such as news, blogs, etc. I want to save them in markdown files but not databases.

Thanks.