Hacker News Reader

deathemperoron I built an online PDF management platform using open-source software

just curious: what do you use to convert web pages to pdf?

from my perspective of someone from Southeast Asia, Son Goku is definitely Sun Wukong the Monkey King from Journey To The West

https://en.wikipedia.org/wiki/Monkey_King

deathemperoron You have a right to know why a health insurer denied your claim

I'm a software engineer working in the insurance industry in Vietnam. The company provides claims service (TPA) for insurers. I find the conversation super interesting as our healthcare and insurance system share the same problems with the US (everywhere else too I think). I'm super excited that my work piece by piece improve the industry one way or another.

deathemperoron Ask HN: What would be your stack if you are building an MVP today?

Nextjs deployed on Vercel Hasura with Postgres Apollo server deployed on Lambda using serverless framework.

deathemperoron SvelteKit 1.0

Same here. I use Apollo Client. React. Hasura

deathemperoron The Business Impact of Code Quality

I’ve just tested it out with our code base and it’s pretty impressive providing very useful intelligence of my team. Codescene provides metrics like he wrote in the post and many more.

deathemperoron YaLM-100B: Pretrained language model with 100B parameters

My mind is blown. Thanks for sharing. Especially with the movie analogy. I’m a very movie person and I imitate my personality traits a lot based on characters on movies…

deathemperoron Show HN: I make $3K/mo from a browser extension

congratz on the success of new launch, Trung.

deathemperoron A digital museum of video game levels

The Zanarkand brings back so much memories I've spent hundreds of hours with

deathemperoron Automated Hydroponic System Build (2020)

thanks for the sources. I'll be digging deeper.

deathemperoron Automated Hydroponic System Build (2020)

interesting. could you point to more details? I'd like to know more since I'm a fish breeder and a lot of fishes need a controlled environment for breeding.

deathemperoron Ask HN: What are best tools for web scraping?

I signed up for proxycrawl, used the javascript api to access a SPA website written in React and it just show a blank page. https://api.proxycrawl.com/?token=aDcC1lB-NZ5_r4vMSN-L3A&url... (I don't mind my token is exposed)

deathemperoron Ask HN: What are best tools for web scraping?

I've just finished my research on web scraping for my company (took me about 7 days). I started with import.io and scrapinghub.com for point and click scraping to see if I could do it without writing codes. Ultimately, UI point and click scraping is for none-technical. There are many data you would find it hard to scrape. For example, lazada.com.my stores the product's SKU inside an attribute that looks like <div data-sku-simple="SKU11111"></div> which I couldn't get. import.io's pricing is also something. I need to pay $999 a month for accessing API data is just too high.

So I decided to use scrapy, the core of scrapinghub.com.

I haven't written much python before but scrapy was very easy to learn. I wrote 2 spiders and run on scrapinghub (their serverless cloud). Scrapinghub support jobs scheduling and many other things at a cost. I prefer scrapinghub because in my team we don't have DevOps. It also supports Crawlera to prevent IP banning, Portia for point and click (still in beta, it was still hard to use), and Splash for SPA websites but it's buggy and the github repo is not under active maintenance.

For DOM query I use BeautifulSoup4. I love it. It's jQuery for python.

For SPA websites I wrote a scrapy middleware which uses puppeteer. The puppeteer is deployed on Amazon Lambda (1m free request first 365 days, more than enough for scraping) using this https://github.com/sambaiz/puppeteer-lambda-starter-kit

I am planning to use Amazon RDS to store scraped data.