HNHacker News
TopNewBestAskShowJobs

thewataccount

1,110 karma · joined September 1, 2022

submissionscomments
thewataccount··on StableCode
I can't seem to find a demo, if anyone has a chance to test it, how does it compare to replit and wizardcoder?
thewataccount··on StableCode
> Stability AI, Apple, Meta, etc are clearly at the finish line

I'm very optimistic and expect them to catch up. I've used the open models a lot, to be clear they are starting to compare to GPT3.5Turbo right now, they can't compete with GPT4 at all. GPT4 is almost a year old from when it finished training I think?

I expect open source models to stay ~1.5 years behind. That said they will eventually be "good enough".

Keep in mind too though that using and scaling GPUs is not free. You have to run the models somewhere. Most businesses will still prefer a simple api to call instead of managing the infrastructure. On top of this many business (medium and smaller) will likely find models like GPT4 to be sufficient for their workload, and will appreciate the built in "rails" for their specific usecases.

tl;dr - open models don't even compare to GPT4 yet (I use them all daily), they aren't free to run, and a API option is still preferably to a massive if not most companies.

thewataccount··on Downfall Attacks
Does anyone know what type of workloads this effects the performance of the most? Is this specialty-type of workloads or are general webserver/database/coding/compiling/gaming/desktop usages effected?
thewataccount··on StableCode
I was impressed enough by replit's 2.7B model that I'm convinced it's doable. I have a 4090 and consider that the "max expected card for a consumer to own".

Also exllama doesn't support non-llama models and the creator doesn't seem interested in adding support for wizardcoder/etc. Because of this, using the alternatives are prohibitively slow to use a quantized 16B model on a 4090 (if the exllama author reads this _please_ add support for other model types!).

3B models with refact are pretty snappy with Refact, about as fast as github copilot. The other benefit is more context space, which will be a limiting factor for 16B models.

tl;dr - I think we need ~3B models if we want any chance of consumer hardware to reasonably run coding models akin to github copilot with decent context length. And I think it's doable.

thewataccount··on Downfall Attacks
That doesn't make any sense. You want to prep a fix for distribution ASAP, and release it so people can be protected from it as soon as possible.

Also many attacks are not really feasible to detect if it's happened - that's like trying to know if the mailman read your postcard.

You can develop executable scanning techniques, but people are a lot more concerned about preventing it in the first place than catching it in the act. Why would you leave people vulnerable so they can watch data get stolen - instead of just fixing the issue?

thewataccount··on Cloud Provider Gets $2.3B Loan Using Nvidia's H100 as Collateral
The funny part to me is they'll likely use that money to buy more H100's.

> Last month, Inflection AI built a supercomputer worth hundreds of millions of dollars powered by 22,000 NVIDIA H100 compute GPUs.

What? InflectionAI has relatively little brand recognition, do I not know something about them, are they just the first in line, or is nvidia just selling this many cards in general? I can only imagine all the typical ones like lambda, AWS, openai, apple, <insert large tech company here> are buying even more?

EDIT: Wait if the collateral is for a 2.3B USD loan - how many cards do they have?!

thewataccount··on If you have first-hand knowledge of the FDA’s torpor, get in touch
> Because the old product is no longer on the market, the FDA will not approve a generic. Even though the exact same product /was/ on the market.

This isn't the case, it doesn't need to still be on the market.

However if the approved product was produced with chemicals that are now banned (or the approved drug itself was banned) - then you _can't_ make a generic because you would have to use a banned chemical to do it.

---

Similar shenanigans to prevent generics include slightly modifying existing generics like ketamine for example, before getting it approved for depression treatment. You cannot patent ketamine for depression, so you can slightly modify it and patent that instead.

thewataccount··on Cognitive AI for ADHD
I have many files and many pages of an actual notebook. For the most part dates are near useless because I rarely refer back to todo-lists. I've tried todo lists but they just end up being a pile of todo items I never look at, get around to, remember what they're for, or don't have time/priority to do it.

Obsidian helps a lot for actual notes. That's the only thing keeping me remotely organized.

> At some point it will become a hardened habit and honestly much better than some AI crutch. There are better uses for AI in my opinion..

That's very true, what I would love to see is have it intelligently pull todo items from my obsidian notes. I'm personally not a fan of using AI for anything if it's not local though, especially with access to all of my notes.

thewataccount··on Cognitive AI for ADHD
From my experience with ADHD - calenders are excruciatingly hard to use consistently. Either it "takes too much time" or I completely forget I was trying to use a calender.

Having a tool that can automatically add items from infodumps/todolists would be useful.

I don't udnerstand the chrome extension part though.

thewataccount··on On-disk HNSW index for Postgres with pg_embedding
Forgive me I'm not super familiar with the vector indexes outside of the basic tsvector for text search.

What's the difference between pg_embedding, pg_vector, and tsvector? Are they compariable/interchangable? And how do you know which one to pick?

My understanding is pg_vector has poorer performance compared to some dedicated vector databases, does pg_embedding perform better?

Sorry if these are silly questions.

thewataccount··on Run Llama 2 uncensored locally
You're not wrong that you almost certainly will want to finetune it to your use case.

I'm looking at it from the perspective of the "tinkering developer" who just wants to see if they can use it somewhere and show it off to their boss as a proof-of-concept. Or even deploy it in a limited fashion. We have ~6 developers where I work and while I could likely get approval for finetuning, I would have to first show it's useful first.

On top of this, I think that for many use cases the given censored version is "good enough" - assigning IT tickets, summarizing messages, assisting search results, etc.

Given the level of "nobody knows where to use it yet" across industries - it's best that there's already a "on the rails" model to play with so you can figure out if your usecase makes sense/get approval before going all-in on finetuning, etc.

There's a lot of companies who aren't "tech companies" and don't have many teams of developers like retail, wholesalers, etc who won't get an immediate go-ahead to really invest the time in fine-tuning first.

thewataccount··on Run Llama 2 uncensored locally
I like the combination of releasing the raw uncensored + censored variants.

I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.

thewataccount··on “Web Environment Integrity”: Locking Down the Web
I'm not sure "I'm not a monopoly, look my competitor owns 2% of the market" is an effective defense anyway?

I think only Safari really counts as a "competitor" at 20% market share and doesn't share the chromium base (compared to edge, etc).

thewataccount··on “Web Environment Integrity”: Locking Down the Web
> Still excuse after excuse after excuse to just not use Firefox.

What happens when Firefox refuses to implement WEI - then <insert large social media companies> start to require it?

If WEI becomes a common requirement then Firefox will effectively be forced to implement it or it won't be usable as a web browser at that point for the average consumer

thewataccount··on “Web Environment Integrity”: Locking Down the Web
FWIW brave genuinely has multiple privacy patches that are useful and can't be done properly with extensions in chrome.

Several of these either can't be done via a js extension to chrome, or can be detected/bypassed. Brave does them in-engine which is the better way to do it.

https://github.com/brave/brave-browser/wiki/Fingerprinting-P...

thewataccount··on The matrix calculus you need for deep learning (2018)
I understand you don't have a free trial, is there any chance you have a demo somewhere of what it actually looks like though? Like a tiny sample lesson or something along those lines? It looks interesting but I'm just uncertain as to what it actually "feels" like in practice vs lets say Brilliant, etc.

I only see pictures, I'm curious the extent of the interaction in the linear algebra/matrix calc specifically

thewataccount··on Microsoft's AI shopping announcement contains hallucinations in the demo
Well there isn't a human that never "hallucinates" in meaning we use for LLMs aka gives "incorrect answers" confidently.

Human's brains use lots of heuristics - we don't "think step by step" through everything - instead we rapidly construct an answer for almost everything.

What we say is "hallucinations" for AI in humans is "misspeaking, misremembering anything, off by 1 math/counting, missidentifying someone, using the wrong variable/method when programming, etc."

thewataccount··on Microsoft's AI shopping announcement contains hallucinations in the demo
This is something I'm not sure people understand.

LLM's only make a "best guess" for each next token. That's it. When it's wrong we call it a "hallucination" but really the entire thing was a "hallucination" to begin with.

This is also analogous to humans - who also "hallucinate" incorrect answers, usually "hallucinate" incorrect answers less when they "Think through this step by step before giving your answer", etc.

thewataccount··on Techdirt has been deleted from Bing and DuckDuckGo [fixed]
> but not for all and even when we do it sometimes looks different in various ways

Personally I'd like to know if that means "we have a manual list we re-added". I think what myself and probably many people are most interested in is "do you rely on bing for < 95% of word match searches (no knowledge graph)"

> when we do it sometimes looks different in various ways

IMO this is also vague - I assume this means they're knowledge graph augmented? I really want to know if they have their own indexer (vs a manual list + caching bing results) for text matching.

thewataccount··on Techdirt has been deleted from Bing and DuckDuckGo [fixed]
The real question is what extent do they index sites themselves?

If they're manually adding a entry for "insert site here", maybe some "caching" of bing's results for the biggest most common searches then IMO that's still 99.999999% ~= 100%

thewataccount··on Techdirt has been deleted from Bing and DuckDuckGo [fixed]
I understand that you need these to be a full "search engine".

What we really want to know is if for links alone - what extent do you index? Let's ignore the knowledge graph, let's say we just want a simple word match.

I'm surprised you didn't have techdirt's homepage indexed at all. "techdirt" did not return "techdirt.com". To me this means you aren't indexing the home pages for popular websites at all - let alone their content - and rely effectively exclusively on bing with maybe some "caching".

Talking about links alone - exact word matches with the domain name (or page content other than your knowledge graph), what do you index?

What specifically do you do other than adding sites manually to a list and I assume "caching bings results"? To me this would still be 99.999999% ~= 100% reliant on bing for search results.

This isn't meant to be an attack in anyway, I'm just looking to clarify what's going on.

thewataccount··on Techdirt has been deleted from Bing and DuckDuckGo [fixed]
I feel like this makes it clear the level of reliance DuckDuckGo has on Bing. I know it's been a bit ambigious the full extent to which they do, and they claim to have their own indexer, but there's no way this is a coincidence. Surely if you have your own indexer you would have seen techdirt before?

At this moment "techdirt" only returns the wikipedia article, twitter, then mostly unrelated mentions of it. Surely DDG would have seen at least their homepage before?

> Most of our search result pages feature one or more Instant Answers. To deliver Instant Answers on specific topics, DuckDuckGo leverages many sources, including specialized sources like Sportradar and crowd-sourced sites like Wikipedia. We also maintain our own crawler (DuckDuckBot) and many indexes to support our results. Of course, we have more traditional links and images in our search results too, which we largely source from Bing. Our focus is synthesizing all these sources to create a superior search experience.

https://duckduckgo.com/duckduckgo-help-pages/results/sources...

thewataccount··on LLaMA2 Chat 70B outperformed ChatGPT
The fact Guanaco 33B is at 65% with Vicuna 13B at 70.43% immediately makes these results non-sensical from my own experience with them. Heck from my experience Guanaco 33B is better then Vicuna 33B!

Not to mention GPT4 at 95% and ChatGPT at 89% - I use chatgpt(3.5turbo)/gpt4 daily for work, and I rarely ever bother with 3.5turbo because of how unreliable it's answers are compared to gpt4.

So whatever this is effectively measuring is useless for comparing these models, especially across work types.

thewataccount··on Alpine Linux does not make the news
I'm confused as to why package signing isn't standard for all package managers.

Didn't Pypi just remove signing too?

thewataccount··on Alpine Linux does not make the news
Most of the big packages have alpine variants yeah
thewataccount··on OpenAI shuts down its AI Classifier due to poor accuracy
The paper I linked in the parent's comment as the "Simple proof of concept" on page 2, and like you said outlines it's limitations as both negative to performance and also easily detectable and determinable.

Their improved method instead only replaces tokens when there's many good choices available, and skips replacing tokens when there are few good choices. "The quick brown fox jumps over the lazy dog" - "The quick brown" is not replaceable because it would severely harm the quality.

Essentially it's only replacing tokens where it won't harm the performance.

It's worth noting that any watermarking will likely harm the quality to some degree - but it can be minimized to the point of being viable.

thewataccount··on OpenAI shuts down its AI Classifier due to poor accuracy
https://arxiv.org/pdf/2301.10226.pdf

Here's a decent paper on it.

It covers private watermarking (you can't detect it exists without a key), resistance to modifications, etc. Essentially you wouldn't know it was there and you can't make simple modifications to fool it.

OpenAI could already be doing this, and they could be watermarking with your account ID if they wanted to.

The current best countermeasure is likely paraphrasing attacks https://arxiv.org/pdf/2303.11156.pdf

thewataccount··on Transparent field level encryption for Django with pgcrypto PostgreSQL extension
These are django fields, so you would only use them for your "sensitive columns" like payment information, emails, other rarely used stuff.

I checked the readme and I might have missed it but this doesn't seem to be suggesting you replace every column with these, these are just helpers to make encrypting specific columns easier.

Because it's rarely used columns, or columns you'd need to wait for an external api anyway (email, sms, payment, etc) the performance impact should be minimal. You wouldn't need these fields to be indexed.

The attack surfaces this addresses is the compromise of postgres or it's host, or miss handled backups. Preferably you'd be using this on top of full-disk encryption.

EDIT: The use of PGP is weird to me though, why not AES?

thewataccount··on Transparent field level encryption for Django with pgcrypto PostgreSQL extension
At least in my experience you would only encrypt the fields that are sensitive.

So of course anything related to payments, possibly emails/ips, that type of stuff.

In a lot of services you don't usually need to refer to these fields except on the settings pages and possibly a checkout page, things like that. So the Username, preferences, and content of your site will likely be unencrypted.

Because you can avoid using those types of fields on most pageloads/api calls it should have minimal impact, especially since those places often include using an external api you have to wait for anyway (email, checkout, etc).

thewataccount··on ChatGPT for Android – Pre-orders
Are we suddenly against native mobile apps altogether?

I get the cynicism with apps like threads, being mobile-app only.

But OpenAI's business model is selling API access to a LLM - chatGPT is just a basic consumer front end. They've given no indication that they would want to deprecate the very actively developed web front end - and I see no motivation for them to do that.

Unlike social media platforms - OpenAI doesn't need to control the consumer UI - they make money either way. Alternative UI's and adblockers aren't a threat to them. They also don't have "engagement stats" to worry about.

I'm just saying we shouldn't get pitch forks out just yet, "native app" is often a selling point.

← PreviousPage 3 of 12Next →