1,110 karma · joined September 1, 2022
I'm very optimistic and expect them to catch up. I've used the open models a lot, to be clear they are starting to compare to GPT3.5Turbo right now, they can't compete with GPT4 at all. GPT4 is almost a year old from when it finished training I think?
I expect open source models to stay ~1.5 years behind. That said they will eventually be "good enough".
Keep in mind too though that using and scaling GPUs is not free. You have to run the models somewhere. Most businesses will still prefer a simple api to call instead of managing the infrastructure. On top of this many business (medium and smaller) will likely find models like GPT4 to be sufficient for their workload, and will appreciate the built in "rails" for their specific usecases.
tl;dr - open models don't even compare to GPT4 yet (I use them all daily), they aren't free to run, and a API option is still preferably to a massive if not most companies.
Also exllama doesn't support non-llama models and the creator doesn't seem interested in adding support for wizardcoder/etc. Because of this, using the alternatives are prohibitively slow to use a quantized 16B model on a 4090 (if the exllama author reads this _please_ add support for other model types!).
3B models with refact are pretty snappy with Refact, about as fast as github copilot. The other benefit is more context space, which will be a limiting factor for 16B models.
tl;dr - I think we need ~3B models if we want any chance of consumer hardware to reasonably run coding models akin to github copilot with decent context length. And I think it's doable.
Also many attacks are not really feasible to detect if it's happened - that's like trying to know if the mailman read your postcard.
You can develop executable scanning techniques, but people are a lot more concerned about preventing it in the first place than catching it in the act. Why would you leave people vulnerable so they can watch data get stolen - instead of just fixing the issue?
> Last month, Inflection AI built a supercomputer worth hundreds of millions of dollars powered by 22,000 NVIDIA H100 compute GPUs.
What? InflectionAI has relatively little brand recognition, do I not know something about them, are they just the first in line, or is nvidia just selling this many cards in general? I can only imagine all the typical ones like lambda, AWS, openai, apple, <insert large tech company here> are buying even more?
EDIT: Wait if the collateral is for a 2.3B USD loan - how many cards do they have?!
This isn't the case, it doesn't need to still be on the market.
However if the approved product was produced with chemicals that are now banned (or the approved drug itself was banned) - then you _can't_ make a generic because you would have to use a banned chemical to do it.
---
Similar shenanigans to prevent generics include slightly modifying existing generics like ketamine for example, before getting it approved for depression treatment. You cannot patent ketamine for depression, so you can slightly modify it and patent that instead.
Obsidian helps a lot for actual notes. That's the only thing keeping me remotely organized.
> At some point it will become a hardened habit and honestly much better than some AI crutch. There are better uses for AI in my opinion..
That's very true, what I would love to see is have it intelligently pull todo items from my obsidian notes. I'm personally not a fan of using AI for anything if it's not local though, especially with access to all of my notes.
Having a tool that can automatically add items from infodumps/todolists would be useful.
I don't udnerstand the chrome extension part though.
What's the difference between pg_embedding, pg_vector, and tsvector? Are they compariable/interchangable? And how do you know which one to pick?
My understanding is pg_vector has poorer performance compared to some dedicated vector databases, does pg_embedding perform better?
Sorry if these are silly questions.
I'm looking at it from the perspective of the "tinkering developer" who just wants to see if they can use it somewhere and show it off to their boss as a proof-of-concept. Or even deploy it in a limited fashion. We have ~6 developers where I work and while I could likely get approval for finetuning, I would have to first show it's useful first.
On top of this, I think that for many use cases the given censored version is "good enough" - assigning IT tickets, summarizing messages, assisting search results, etc.
Given the level of "nobody knows where to use it yet" across industries - it's best that there's already a "on the rails" model to play with so you can figure out if your usecase makes sense/get approval before going all-in on finetuning, etc.
There's a lot of companies who aren't "tech companies" and don't have many teams of developers like retail, wholesalers, etc who won't get an immediate go-ahead to really invest the time in fine-tuning first.
I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.
I think only Safari really counts as a "competitor" at 20% market share and doesn't share the chromium base (compared to edge, etc).
What happens when Firefox refuses to implement WEI - then <insert large social media companies> start to require it?
If WEI becomes a common requirement then Firefox will effectively be forced to implement it or it won't be usable as a web browser at that point for the average consumer
Several of these either can't be done via a js extension to chrome, or can be detected/bypassed. Brave does them in-engine which is the better way to do it.
https://github.com/brave/brave-browser/wiki/Fingerprinting-P...
I only see pictures, I'm curious the extent of the interaction in the linear algebra/matrix calc specifically
Human's brains use lots of heuristics - we don't "think step by step" through everything - instead we rapidly construct an answer for almost everything.
What we say is "hallucinations" for AI in humans is "misspeaking, misremembering anything, off by 1 math/counting, missidentifying someone, using the wrong variable/method when programming, etc."
LLM's only make a "best guess" for each next token. That's it. When it's wrong we call it a "hallucination" but really the entire thing was a "hallucination" to begin with.
This is also analogous to humans - who also "hallucinate" incorrect answers, usually "hallucinate" incorrect answers less when they "Think through this step by step before giving your answer", etc.
Personally I'd like to know if that means "we have a manual list we re-added". I think what myself and probably many people are most interested in is "do you rely on bing for < 95% of word match searches (no knowledge graph)"
> when we do it sometimes looks different in various ways
IMO this is also vague - I assume this means they're knowledge graph augmented? I really want to know if they have their own indexer (vs a manual list + caching bing results) for text matching.
If they're manually adding a entry for "insert site here", maybe some "caching" of bing's results for the biggest most common searches then IMO that's still 99.999999% ~= 100%
What we really want to know is if for links alone - what extent do you index? Let's ignore the knowledge graph, let's say we just want a simple word match.
I'm surprised you didn't have techdirt's homepage indexed at all. "techdirt" did not return "techdirt.com". To me this means you aren't indexing the home pages for popular websites at all - let alone their content - and rely effectively exclusively on bing with maybe some "caching".
Talking about links alone - exact word matches with the domain name (or page content other than your knowledge graph), what do you index?
What specifically do you do other than adding sites manually to a list and I assume "caching bings results"? To me this would still be 99.999999% ~= 100% reliant on bing for search results.
This isn't meant to be an attack in anyway, I'm just looking to clarify what's going on.
At this moment "techdirt" only returns the wikipedia article, twitter, then mostly unrelated mentions of it. Surely DDG would have seen at least their homepage before?
> Most of our search result pages feature one or more Instant Answers. To deliver Instant Answers on specific topics, DuckDuckGo leverages many sources, including specialized sources like Sportradar and crowd-sourced sites like Wikipedia. We also maintain our own crawler (DuckDuckBot) and many indexes to support our results. Of course, we have more traditional links and images in our search results too, which we largely source from Bing. Our focus is synthesizing all these sources to create a superior search experience.
https://duckduckgo.com/duckduckgo-help-pages/results/sources...
Not to mention GPT4 at 95% and ChatGPT at 89% - I use chatgpt(3.5turbo)/gpt4 daily for work, and I rarely ever bother with 3.5turbo because of how unreliable it's answers are compared to gpt4.
So whatever this is effectively measuring is useless for comparing these models, especially across work types.
Didn't Pypi just remove signing too?
Their improved method instead only replaces tokens when there's many good choices available, and skips replacing tokens when there are few good choices. "The quick brown fox jumps over the lazy dog" - "The quick brown" is not replaceable because it would severely harm the quality.
Essentially it's only replacing tokens where it won't harm the performance.
It's worth noting that any watermarking will likely harm the quality to some degree - but it can be minimized to the point of being viable.
Here's a decent paper on it.
It covers private watermarking (you can't detect it exists without a key), resistance to modifications, etc. Essentially you wouldn't know it was there and you can't make simple modifications to fool it.
OpenAI could already be doing this, and they could be watermarking with your account ID if they wanted to.
The current best countermeasure is likely paraphrasing attacks https://arxiv.org/pdf/2303.11156.pdf
I checked the readme and I might have missed it but this doesn't seem to be suggesting you replace every column with these, these are just helpers to make encrypting specific columns easier.
Because it's rarely used columns, or columns you'd need to wait for an external api anyway (email, sms, payment, etc) the performance impact should be minimal. You wouldn't need these fields to be indexed.
The attack surfaces this addresses is the compromise of postgres or it's host, or miss handled backups. Preferably you'd be using this on top of full-disk encryption.
EDIT: The use of PGP is weird to me though, why not AES?
So of course anything related to payments, possibly emails/ips, that type of stuff.
In a lot of services you don't usually need to refer to these fields except on the settings pages and possibly a checkout page, things like that. So the Username, preferences, and content of your site will likely be unencrypted.
Because you can avoid using those types of fields on most pageloads/api calls it should have minimal impact, especially since those places often include using an external api you have to wait for anyway (email, checkout, etc).
I get the cynicism with apps like threads, being mobile-app only.
But OpenAI's business model is selling API access to a LLM - chatGPT is just a basic consumer front end. They've given no indication that they would want to deprecate the very actively developed web front end - and I see no motivation for them to do that.
Unlike social media platforms - OpenAI doesn't need to control the consumer UI - they make money either way. Alternative UI's and adblockers aren't a threat to them. They also don't have "engagement stats" to worry about.
I'm just saying we shouldn't get pitch forks out just yet, "native app" is often a selling point.