Hacker News Reader

Comment by thomasfromcdnjs | Hacker News Reader

Hey Nico, this looks awesome.

I built out pretty much the exact same thing for JSON Resume (far uglier) but uses vector embeddings to rank relevant jobs -> https://registry.jsonresume.org/thomasdavis/jobs

My prompt when running it through GPT-3.5 sucks, doesn't always get location and salary. GPT-4 is way better but $$$$.

I've seen about 3-4 projects doing this with HN WhoIsHiring projects now, I wonder if we should all just share an API (of GPT parsed HN job posts) and pool some $$$. (In my version, I also created a JSON Schema that represents a Job Description)

scripts: https://github.com/jsonresume/jsonresume.org/tree/master/app... api code: https://github.com/jsonresume/jsonresume.org/blob/master/app...

View on HN

Amazing, thank you! Very cool

Did you do any sort of benchmark between embeddings and direct inference?

How did you tune the distance for relevance, how did you decide on chunking and how many results would you include?

I want to try filtering using the distance of embeddings RAG style, but I wonder about how that changes the results

The current method the app uses feels pretty good, it does get you the best matches, and sometimes it also includes a few “extra” suggestions that typical filters would have left out, but that end up being potentially interesting matches

I'm not exactly sure what you mean by direct inference?

Do you mean that you pass the resume and job posting to the initial GPT prompt? and rely on the returned "fit_for_resume" property? (I'm curious if you tried using OpenAI function outputs too)

If so I didn't try that. And thinking about it, I like your style. (There's no reason why you couldn't do both anyway)

Another question; Did you try asking for a "fit_rating"?

Here is the prompt/call I used for GPT'ing the HN posts -> https://github.com/jsonresume/jsonresume.org/blob/master/app...

As for the distance of the vector similarity search. I am just using Supabase's pg_vector plugin support, with 3704 dimensions using the default search (cosine).

Sorry, yes, that’s exactly what I meant by “direct inference”; putting the resume and all necessary in the prompt and “trust” the fit evaluation done by the LLM

Thank you so much for the insightful comments and pointers

If you were expanding the functionality/features of LLM-assisted job searching, what do you think are the most useful challenges to tackle?

I have some wild ideas about distributed job searching systems running on people’s command lines hahaha

I think there is lots of crazy ideas that need to be built, the next job I find I will probably just rely on tools like these for the fun of it.

I've got a few that I will chime in later with. A simple one for now, is put a couple buttons next to the job listings in your UI.

Generate cover letter - Make a cover letter based off your resume and the job description

Generate tailored resume - Create a permutation of your resume to promote things relevant to that job description

Yes, those would be amazing

Haven’t been able to get it down though. I’ve tried a few prompts in those lines, but I never really like the outputs

The iteration is still very helpful in fleshing out a good personalized resume/cover letter, but I feel the app needs something a bit more automated/straightforward for the users

Any ideas on how to structure good prompts or a workflow to make it work well in terms of user experience?

Reply on news.ycombinator.com