Hacker News Reader

Comment by weishuhn | Hacker News Reader

Creator here! Wasn't expecting this to happen :) The site is definitely still in Beta so I appreciate any and all feedback. I just launched it a few days ago. It's been my COVID project and I finally got to the point where I felt comfortable having others use it.

The biggest hurdle was the speed of the graph creation. Basically taking a 250,000,000 paper/2,500,000 citaiton db and creating graphs that could be up to 200k papers and 3-4mm citations. For that I ended up learning/using Rust (which was a great experience).

The plan is to keep it totally free and hopefully get some institutional support once I get a better handle on demand and costs.

Ask me anything!

EDIT: As you are going through the site, be sure to use the purple "+" buttons to create your own graphs centered on the topic of your choice. That combined with the in-graph keyword filters are probably the most powerful ways to quickly zero in on the most relevant literature.

View on HN

Very nice work. I especially liked the ability to build up a collection of papers, that the response time was good, and that the SQL could be edited directly.

Do you have any plans to add a graphical visualization of top/central papers?

That is the most requested feature and something I'm working on. It's a fun (and hard) design/data problem. Which of the 5k-150k papers do you show in the graph? And then how do you render them in a way that is both visually appealing but also conveys the most import information?

Displaying graph data in a user-friendly way is indeed a challenge. The nicest user interface I've seen is on metacademy "explore" views:

https://metacademy.org/graphs/concepts/bayesian_logistic_reg...

and src code for the graph view is here: https://github.com/metacademy/metacademy-application/blob/ma...

they do some clever hiding of edges so graph is not overwhelming, but still only O(100) nodes.... for O(100k) nodes you'll need to do some selection for sure ;)

Just curious, what source are you using for the citation graph? I seem to remember looking for an API to something like ACM digital library at one point and not really finding what I wanted, but maybe I just didn’t know how to look.

I love this idea btw, I’m going to use it to find some holiday reading!

Which crate are you using for graph manipulation?

I go through a lot of the details in my post on the Rust subreddit: https://www.reddit.com/r/rust/comments/kfiaqn/just_wanted_to...

But long story short, I end up doing most of the graph analysis by passing in the citations, using PyO3, to graph-tool in python then returning the data I need about each paper. I am planning on moving that over to Rust. But not being an academic I wanted to get feedback on the quality of the results before making it difficult to quickly test different types of algorithms.

Are you planning on open sourcing parts of it?

Eventually I'd like to move the site to open source, but right now the repo isn't in a place where I can do that. As for specific parts, it's pretty purpose built and this is my first Rust project and so I'm not sure which parts would be helpful to the community. And I doubt they would meet the communities standards just yet :)

It’s better to open source sooner rather than later even if it’s not in a place you’d want it to be. Like some of the work you have to do might be done by the community.

Reply on news.ycombinator.com