HNHacker News
TopNewBestAskShowJobs

Ephil012

381 karma · joined January 30, 2017

etp256 at nyu dot edu
submissionscomments
Ephil012··on Slack AI Training with Customer Data
To add on to this: I think it should be mentioned that Slack says they'll prevent data leakage across workspaces in their model, but don't explain how they do this. They don't seem to go into any detail about their data safeguards and how they're excluding sensitive info from training. Textual is good for this purpose since it redacts PII thus preventing it from being leaked by the trained model.

Disclaimer: I work at Tonic

Ephil012··on Jones Act Contributions to Offshore Wind Difficulties
I recently attended a talk by someone at Balsa Research last week about the Jones Act. Balsa Research is trying to get it repealed. Highly recommend checking them out.

https://www.balsaresearch.com/

Ephil012··on 23andMe updates their TOS to force binding arbitration
Your best bet is probably to go through a doctor and get testing from a medical genome sequencing service that is covered under HIPAA. I am not 100% sure if this is bulletproof, but it is probably better than going through a DTC company. Plus, most DTC companies like 23 and me use imprecise genome sequencing and not full genome sequencing like many medical providers do.
Ephil012··on [dead]
Dbrand Sues Casetify
Ephil012··on 'Adopt an axolotl' campaign launches in Mexico to save iconic species [,,,]
AP didn’t provide a link to the official campaign page I think. Here’s the link https://www.restauracionecologica.org/adopciones
Ephil012··on Ousted OpenAI CEO Makes Plans for New Artificial Intelligence Company
Out of curiosity, why do you say the bit about not using CloudFlare's DNS? Is using it incompatible with archive.is?
Ephil012··on Ask HN: Avoiding irrelevant or undesirable model context in RAG
At my company, we developed an open source library to measure if the context the model received is accurate or not. While not exactly the same as what you're asking, you could in theory use it to measure when an LLM deviates from the context to tweak the LLM to not always use the provided context.

Shameless plug for the library: https://github.com/TonicAI/tvalmetrics

Ephil012··on Exploring GPTs: ChatGPT in a trench coat?
I tried out the Assistants API and noticed that similarly bad performance, but with a catch. Apparently if you combine all the files into one single text file, then the performance is amazing. But if it's spread across multiple files the performance is pretty bad.

Analysis here if anyone is curious https://news.ycombinator.com/item?id=38280718

Ephil012··on Exploring GPTs: ChatGPT in a trench coat?
Here's the catch. I did an analysis earlier myself of the assistants API and discovered this good performance is ONLY for if you combine into a single text file. If you try multiple files it fails.

Here's my post with the analysis. https://news.ycombinator.com/item?id=38280718

Ephil012··on Show HN: Demystifying Advanced RAG Pipelines
Unfortunately, right now the LLM cost is just a fundamental issue. I think it is hard to get around because comparing answer quality usually involves understanding the question and answer itself which is a task that's really well suited to LLMs.

One thing we have considered is some forms of evaluation could be replaced simply with using the embeddings of the question, context, and answer instead of using the LLM model for analysis. The idea is you could compare all the embeddings to get a rough idea of the performance based on similarity. That should in theory reduce costs. The only other alternative is just to use less advanced models which are cheaper.

Ephil012··on Show HN: Demystifying Advanced RAG Pipelines
Pretty cool tutorial. As a side note, it is pretty hard to evaluate these pipelines for quality once you build them since there's not many standard practices yet given how new this all is. If it's helpful to anyone else, we built a free open source tool within my company that is basically a collection of premade metrics for determining the quality of these pipelines. https://github.com/TonicAI/tvalmetrics
Ephil012··on Show HN: Tonic Validate Metrics – an open-source RAG evaluation metrics package
Right now, we are planning on adding more metrics in the future. We also have considered adding the ability of using vector embeddings instead of LLMs for analyzing the responses (which should bring some efficiency gains). Plus, we might add the ability to analyze the distribution of vectors in your vector db to compare it to the distribution of vectors for user queries.

We also have a UI for visualizing all of these metrics that is free at the moment with no paid tier at all. We are planning to add more features into that UI for better visualizations along with improving the ability to keep track of different versions of your LLM system for comparison.

Ephil012··on Show HN: Tonic Validate Metrics – an open-source RAG evaluation metrics package
No problem! If you have any questions in the future, feel free to open a issue on Github. Also, we got a free UI for visualizing the metric logs (no paid tier or anything like that at the moment) that we'd love people to try out. https://validate.tonic.ai
Ephil012··on Show HN: Tonic Validate Metrics – an open-source RAG evaluation metrics package
Hi all, if anyone has any questions about the open source library, Joe and I will be around today to answer them.
Ephil012··on Kagi finally let me lay Google Search to rest
With all these comments about how Kagi is so much better than Google, I feel like I am living in a alternate world. My experience has been the exact opposite. I've been having to use !g so often on Kagi I turned it off as my default search engine. The thing is I really want to love Kagi, but can't. It just seems inferior in terms of parsing my search queries and returning relevant results.

Let me give an example. On Kagi, I searched up "Overcoming addiction to Reddit". On Kagi, most of the top results are actually just Reddit posts about drug addiction / addiction in general and not Reddit addiction [1]. If you scroll down, the is a result about internet addiction, but that isn't what I asked for either. Only once you get to the very bottom do you actually get links about Reddit addiction itself. Even then, there aren't that many.

If you do the same query on Google, I get much more results that are actually about Reddit addiction [2]. On top of that, the relevant results are way closer to the top than on Kagi. It even includes high quality links like a Hacker News post on Reddit addiction. I believe Kagi included the same link, but it was hidden all the way on the third page as opposed to being somewhat near the top on Google. Don't get me wrong, Google's results aren't perfect and it has a lot of irrelevant stuff show up in my query about Reddit addiction. But it's still better than Kagi.

My theory is that Kagi just saw the words "overcoming addiction" and "Reddit" and assumed I wanted to look for links about addiction in general on Reddit. Basically it's almost like it did "overcoming addiction site:reddit.com" for the top results. Albeit, after the top results it does show links from other sources. Meanwhile, it seems Google did a better job at parsing the query and realizing I was looking up a specific type of addiction, a Reddit addiction. Kagi did not do this.

My next problem with Kagi is just the lack of results in general. It seems Kagi likes to return a limited number of results per search. It seems it usually gives 20-30 results and sometimes gives more (~70 results) if you give it a very generic query. The problem with this is it makes Kagi terrible for research. In daily life, I never even read past the first few pages on Google. But when researching a niche topic for a paper, you sometimes need to go way past the first page to find what you're looking for. This might be because when doing research on something vague or niche it's hard to get accurate matches on the first page of search. As a result, having a large number of search result pages in general is useful for when this happens. But Kagi simply doesn't do this. It's limit on how many results it returns just kills the ability for you to actually go more in-depth without issuing a modified search query. I seriously doubt Kagi's index is literally so small it can only return around 70 something results at max, so I just have to think this is some self-imposed limitation which ultimately harms the search quality.

Another problem I have with Kagi are lenses. Lenses get hyped up by them, but in practice I found them utterly terrible. I searched up Lo Mein without lenses on and my first results were some pretty high quality recipes. I then turned on the recipe lens and the quality noticeably decreased. Instead of returning Lo Mein recipes from actual recipe sites, it instead returns a Reddit post that links to a gif of someone making Lo Mein. Issue is that gif is actually on gfycat which is shut down. So the top result is useless. The rest of the results aren't even Lo Mein recipes, but instead chicken recipes for instance. So in the process of trying to use lenses to increase my search quality, it actually made it worse.

Also, a tiny nit is that Kagi isn't as good at getting quick answers. If I look up a question on Google, it will usually display some quick summary at the top with the answer to the question. Kagi has quick answer too, but you have to manually click on it to get Kagi to generate the quick answer. I get they're a startup trying to save money, but this is just another reason for me to use Google over Kagi.

As a whole, I haven't found Kagi to be useful at all like many commenters claim it is. I really wanted Kagi to work out as again I love the idea of Kagi. But ultimately, I can't justify using it over Google because the quality has just been so much worse in my experience. Ultimately, I am paying money for Kagi, yet I am getting a lower quality product than Google which is free. At that point, it doesn't make sense for me to continue with Kagi.

1. https://kagi.com/search?q=overcoming+addiction+to+reddit

2. https://www.google.com/search?q=overcoming+addiction+to+redd... (Google may show different results to you because Google might be customizing results based on past searches)

Ephil012··on Ask HN: Why does Apple want me to expose my iPhone password to repair my screen?
FYI, if you do this option they’ll likely still ask you to disable find my iirc
Ephil012··on Ask HN: Why does Apple want me to expose my iPhone password to repair my screen?
While I agree it’s not best practice to enter your password on a random device, turning off find my is pretty standard for Apple repairs. I think one of the reasons why is that Find My enables activation lock which will prevent them from fully restoring the iPhone if they need to for any reason. Also, if they decide to replace it with a new unit then you need activation lock disabled so the old one can still be usable. With it still enabled, the old iPhone would essentially be bricked for life otherwise since it’s linked to your account.

As far as the Apple support app, I think that Apple does have a legitimate app under that name. However, usually find my is disabled under the find my app, not the Apple support app (unless Apple stores use a custom version with it in there or it’s hidden in the regular app where I can’t find it). It’s also possible the support app just redirected you to the find my app. Either way, if you don’t trust the app, you could just go to a web browser at home and disable it via the web app for find my so they can proceed with the repair. I don’t know if third party repair shops will also make you disable find my, but I know Apple usually asks you to disable find my first

Ephil012··on iOS 17 is available today
You can try going to Settings > General > Reset then tap "Reset Keyboard Dictionary". I think this wipes everything iOS has stored about your typing habits so it should hopefully fix the issue
Ephil012··on Procreate Dreams
Similar to the other commenter’s suggestion, you might want to do a local backup of the iPad that’s encrypted (iirc encrypted backups include more data). Hopefully the procreate artwork will be saved in that backup. However, you need to unlock the iPad first to do that I think. Even though the screen is busted, you can turn on voiceover using Siri and unlock the iPad using that potentially. Although, this requires that the screen is still able to detect touch input despite being broken / unable to display anything. Even if you don’t think it can detect touch input it’s still probably worth a shot just in case it can
Ephil012··on My Books Were Used to Train AI
https://archive.ph/cvkPt
Ephil012··on Show HN: A website chatbot that also uses APIs
Out of curiosity, is this product page built with tailwind ui or something similar? I noticed a lot of product pages have a similar look (as another commenter mentioned) and I was curious what people use for these pages since it seems like they have the same ui elements like they came from some ui framework
Ephil012··on Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
Most of that is in their equity comp which is quite weird in how it works. So those numbers are highly inflated. The equity is valuable only if you sell it or if OpenAI makes a profit. Selling it might be harder given they're not a public company. On top of that, the profit is capped so there is a limit to how much money can be made from it. So while it's 900k on paper, in reality, it might not be as good as that. https://www.levels.fyi/blog/openai-compensation.html
Ephil012··on Show HN: Openlayer – Test, fix, and improve your ML models
FYI you have a typo on your homepage. On one of the charts under “Solve efficiently” you misspelled salary as “salarry”.

As a side note: your product is pretty cool :)

Ephil012··on WeVPN Is Shutting Down
Just wanted to say I have been using your product for years and it’s been pretty great. I picked it up years ago for some lifetime vpn promotion on stack social that y’all ran. Been using it ever since and it’s been amazing.
Ephil012··on Show HN: ThoughtCoach: Helping to improve mental health with AI
How do you prevent cases where the AI might say something harmful (e.g. suggesting the user kill themselves)?
Ephil012··on Starlink V2 Satellites in Trouble
I think there’s a general trend online where people post tweets on places like Reddit where others might assume the tweet is factual. As a standard disclaimer, just because someone claimed on Twitter that SpaceX messed up the reaction wheels does not mean it’s true. Unless there’s more concrete proof, we should not automatically assume it’s true.
Ephil012··on Amazon officially becomes a health care provider after purchase of One Medical
CVS bought Aetna. Aetna doesn’t own CVS

Source: https://www.cvshealth.com/news/company-news/cvs-health-compl...

Ephil012··on Nutlope/roomGPT: open-source clone of Interior.AI
Sorry for the stupid question, but what do you mean by this? Adding a LoRA?
Ephil012··on Ask HN: Who is hiring? (March 2023)
You posted in the wrong thread. This is for companies hiring. Here is the thread for people looking for work https://news.ycombinator.com/item?id=34983765
Ephil012··on Daily.place
Cool website! One minor nit: coffee shop is misspelled on the website. It instead says "coffe shop"
Page 1 of 3Next →