authorfly

Just for the right order of history here - this prediction disparity is false - GPT-3 came out in 2020 and a decent slew of researchers and startups had access in 2021. Though unreliable, slow and costly, davinci could mimic chats back then (It was a main feature sample), but OpenAI banned chat applications until IT released ChatGPT in Nov 2022.

For a whole year prior to ChatGPT, "davinci-instruct-002" was at the level of ChatGPT for most purposes, but applications using OpenAIs models had to be approved, and chat apps were disallowed (as were any completions > 300 tokens).

The competitors of that time (GPT-J, GPT-20b-neo) and early BigScience stuff was behind OpenAI, especially on instruction training. So we didn't see applications until OpenAI changed it's application approval process in November 2022. A good reference for where things were is "Machine Learning Street Talks" 4 hour 2020 GPT-3 video - although it expressed doubt about GPT-3, you can see there through many of the examples that the tech was close to ChatGPT relative to the previous iteration (GPT-2, T5, BERT etc).

However, for experts, it was obvious from about Spring 2021 when GPT-3 started taking off in the research zeitgeist with actual research usage, with loads of tweets and recognition of what it meant for the field (both in terms of impact to grant applications, ongoing projects and the future of language models being decoder only for the reasonable short term).

The real gap in prediction disparity was that most experts, possibly due to a bias towards their funding areas researching BERT etc, totally ignored GPT-2 and assumed encoder architectures or other fields (RNNs etc) were still better paths. Established Researchers* would have predicted GPT-3 was 10 years away in 2018 or 2019 which is funny in hindsight. However, even in 2013 (when I was not a researcher but a student), people in computer vision felt that arbitrary tasks/arbitrary recognition was nigh impossible unless using coding schemes etc (which limited to certain image types anyway)

* And I say this because I was researching in 2019, but I was more optimistic after seeing T5 and GPT-2. The field did not ignore CLIP, but there wasn't much obvious apparent research to be done with CLIP initially. Computer vision was all about semantic segmentation and recognition of disease etc in the 10s.

Does anyone else get the sense that the definition has been bastardized by the conflation of the two concurrent previous uses of "agent"?

i.e. in AI, biology and informatics, "Agent" typically meant something: That had a form / self / embodiment. That could sense the environment and react to those perceptions. That possibly could learn, adapt, or change to various degrees of complexity, which would entail optionally being an "intelligent system".

Meanwhile in common parlance, Agent meant: Someone who acts or behaves on behalf of another adaptively to accomplish something with some degree of freedom.

And this might explain why so people say agent/agentic necessarily refers to "tool use" or "being able to overcome problems on the happy path" or "something capable of performing actions on an infinite loop while reacting" (the latter two in my opinion, conflates the meaning of "Intelligent system" or "Intelligent behavior"). Meanwhile, biologists might still reply to a single cell seemingly inert, or a group of bacteria in a colony, as an Agent (a more behaviouralist/chemical "look-deep-down" perspective)

I think a lot of disappointment is that biologists/OG AI enthusiasts are looking for something truly adaptive, sensing, able to behave, "live" indefinitely, have acquire or set goals, and which might be able to if intelligent, work with other agents to accomplish things (e.g. a "society"). Meanwhile, people who just want an "AI HR Agent" just want something that can communicate, interview, discern good applicants, and book the interviews plus provide summary notes. These two things are very different. But both, could use tools etc (the key difference from ChatGPT which is enabling this new concept to be more useful than ChatGPT, alongside various forms of short term memory rather than "fresh-every-time-conversations).

Probably a multilingual version will be needed, like with BERT and RoBERTa. I should hasten to add for multi language tasks(beyond detection), either simpler methods for tasks like multiple language classification/prediction(e.g. word frequency, BERTopic like approaches or SVMs) or LLMs are generally a better candidate.

There are a couple of reasons.. 1) That size (even for the large) is too much for multiple languages with good BLEU scores. 2) Encoder and decoder models don't tend to get trained for translation as much as e.g. GPT models with large translation texts in their datasets across multiple languages (with exceptions such as T5 translation task).

Agentic AI comes out of historical AI, systems computing and further back biological/philosophical discussion. It's not about tool use although ironically, animal tool use is a fascinating subject not yet corrupted by the hype around intelligence.

I implore you to look into that to see how some people relate it to autonomy or AGI or ASI(wrongly, imo - I think shoehorning OOP and UML diagrams plus limited database like memory/context is not a path to AGI. Clever use of final layers, embeddings and how you store/weight them (and even more interesting combinations) may yield interesting results because we can (buzzword warning) transcend written decoding paradigms - the Human brain clearly does not rely on language).

However what gets marketed today is, as you say, not capable of any real agent autonomy like in academia - they are just self-recursive ChatGPT prompts with additional constraining limits. One day it might be more, but libraries now are all doing that from my eye. And recursion has pros but emphasizes the unreliability con/negative of LLMs.

How did you find this kind of team? I know you note B2B/unicorns can have these, but I have never found them myself. I find it difficult: Reach out to the CPO/CEO with something innovative or genuine passion for the industry area, and they either get you in excited and others in the C/tech suite are against you / already busy with projects, OR they see you as a threat and just talk to learn.

I am also wary because I have seen such teams before (e.g. a HTML5 team, yes silly as it sounds, web sockets and SVG felt like they opened entire new ways of making games), and a data science team using BERT etc (outdated totally by ChatGPT and made useless). Likewise I lead a team to adapt React (before it was predominant) which helped, and I was liked on the team by most for helping them code with less bugs (some of the old guard did not like using React). However, once that happened, it was done. My managers were changed, for someone who loved making lots of plans. I then got over-managed to the point I changed my career as it was obvious this manager had no interest in any future risks in the next 3-5 years of their time in that seat.

I just don't see stories where these teams somehow make a 2nd product for the company which actually successfully scales. They usually get eaten by being overtaken technically, or eaten by the main product. So what do you do after the "eaten" point? Does the company keep your around?

There are a few "The Psychology of Money" books/talks including by Charlie Munger. Which one are you referring to may I ask?

I suspect he is arguing the line which many "Global crash incoming" channels propose that:

A) Low birth rate leads to a Population/Population disposable income total(not per capita) will decrease, and thus gross capitalization decreases reducing stock value. Decreasing the benefit of investing in index funds potentially catastrophically. B) Inflation will be/remain high(or above earning increases) for core expenses (housing, food, water, gas, transport)

Sometimes with the argument that B happening will cause A to happen (e.g. more expensive = less attractive for young immigrants, less attractive to have kids)

I agree with some of it (for example disposable income inequality and core inflation issues are indeed causing a cost of living crisis which is discouraging growth), but I advise NOT to go down the rabbit hole of these channels, as they can give you a lot of anxiety.

Just be aware that Google runs a lot of other integral services that hoover up data anyway and are practically unavoidable for modern life. For example, I blocked all domains including google and I was then unable to use local rent listing websites, unable to comment on forums, and unable to browse university research publication lists, because they were captcha blocked. In fact one of the only upsides of Apps I see is that they tend to avoid captchas.

They fail in a nasty way too - nobody expects you to block them - so you'll simply see blank pages, etc. Likewise for Firebase which it seems 40% of startups use.

I agree with you as a technical guy. It's not the worst. But as to why is happens:

If your business-focused founder or Marketing guy is going to the conference, it'll be Youtube, even though Vimeo is accessible. Unless you go make the account, share it to them, and have a SOP, why would they look at Vimeo?

The problem is when you get any sort of size the non-tech people choose Google as a default and then it (with weak force) slowly black holes up what they do. Just like people chose Microsoft Office (and still do for mature corps) for all kinds of admin, not necessarily technical people using Google (although I would say >80% of the tech graduate emails I am given for contact past graduation are gmail which is a sign of the times!)

Not trying to be "that douchey old times" guy but back in 2005 you could access many APIs for things like weather, temperature, ongoing projects, lists of conference locations, lists of RSS subscriptions, insect population predictions, all sorts.

Today they still exist, but with the exception of the most basic and dated corporate free APIs (e.g. reddit) they are gated by API keys and often access limits.

It was truly rare to see access limits in 2005; APIs were slow and unreliable but not limited.

If anyone documented the loss of APIs (particularly in 2008 IIRC?) I would to read more about it myself too. I think Tom Scott did a video on this, but I can't find it.

What's been added since then?

I remember Blender back then. The days of watching squares slowly render over 2-7 minutes each for some cool project I made with lighting.

Has much new been added to editing/functionality or just speed optimisations?

I agree it is still reasonable.

But some examples of things that are profitable under the old regime, but maybe not at 20x the limit:

API services that consume media, whether to do some business logic specific thing or something specific, e.g. Transcriptions for videos, file conversion API, OCR API provider, etc.

If you try a video app without using WebRTC and peer traffic your bandwidth can blow up too. Even if you use something like push notifications with base64. There are lots of traps that use Bandwidth.

You should not accept this contract.

You are being asked to accept liability for people without an incentive to deliver good work of whose reputation you don't know.

The more unspecified the liability and acceptance/delivery or services description is, the more likely you are to end up with an endlessly "unhappy" client who won't make more payments but won't say it's all finished.

I suggest you run. Never take liability without responsibility, especially not without any large upside possibility. In the absence of clarity, clients will try and argue that you own the liability because you directed other people for the final product. You want to specify details and work against them with a healthy 20% buffer of extra quality/work to make sure the client can never argue you didn't fufil the work. This is the opposite of that. Don't do it.

Problem is eventually you need it for Google Ads, or YouTube really for many things. And none of them are "vital" if they go wrong. They aren't core income pipeline parts. And then you get sucked in.

B2B and go to a conference? Where are you putting your video demo reels/talks? YouTube

Consumer and testing marketing messages for something non-consumer? Google Ads.

Your B2B client wants you to provide your SaaS via a private endpoint on GCP? What else are you going to do than use GCP?

Then once you are on it on company computers, it's a short walk to Google Docs and then Google Workspace, etc.

Yep. I have responded in another post with some stats/surveys. Exactly, the aggregation of people hides the trends, and the fact that a 35 year old is not competing with other 35 year olds, but also with other 21 year olds working at cafes, contributes to this.

A few things here and a few stats:

- Firstly, I admit I can't find where I heard the age 35 specific claim to reference it. I have listed two sources below and found an even stronger claim by Thomas Sowell (that most people at one year of their life in America earn top 20% income), but it wasn't that statement I was looking for, so I will keep try to find it. But income decile movement surveys in the UK supports this, as do dynamic income surveys conducted in the US, so I'll share those to you:

- Income is highly mobile. When tracking what income groups have, within groups there is great variation. Half the people with top 20% of income drop out of the top 20% over 5 year periods, with a significant proportion of those dropping out (30%) dropping to below average income. In other words 15% of the people earning the top quintile of income will earn below average income in 5 years time.

... Meanwhile, showing upwards mobility, 20% of the people earning the 20% lowest in the UK moved to the top 40% of income by just 5 years later in this study (Figure 19 / corresponding table) - https://www.gov.uk/government/statistics/income-dynamics-201...

- You seem to assume income goes up over time, however in western countries, income tends to peak at 30-50, hence 35 in the stat. It peaks at age 37 and stays flatish until it declines from age 44 in the UK, for example: https://www.statista.com/statistics/824464/mean-disposable-i...

- Historically, older people earned more in the 40s and 50s, but due to COVID or the simple growth in proportion of people economically inactive or in part time roles at those ages, you can see how in the UK in the last 20 years that trend disappeared: https://www.ons.gov.uk/peoplepopulationandcommunity/personal...

- Children and pensioners essentially earn nothing for the purposes of this stat, but as they still spend (or indirectly through parents consume) money, they're in this total. Pensioners are reasonably part of this stat because they are spending in the economy, but are seldom members of the "working population"

The main resource I would suggest on income mobility is "Chasing the American Dream: Understanding What Shapes Our Fortunes" which is based on the Panel Study of Income Dynamics (PSID) [https://onlinelibrary.wiley.com/doi/abs/10.1111/jacc.12716] - it does date to the 70s/80s, but it focuses on: can you achieve top income, can you have income security. Their major finding is that A) Most people through randomness and effort end up earning above-average incomes during the middle stage of their life, and that B) Wealth, but not income, is cumulative.

This is also historically constant, for example Thomas Sowell wrote about how most Americans at the time would be in the top 20% of income for at least one year of their life in "Economic Facts and Fallacies" and "Basic Economics". (obviously the UK quintile study notes decreased lower income exit mobility which is bad for poorer people and means dynamic income mobility is reducing).

What's interesting is that the people who are able to predict and come up with the idea (e.g. a researcher using AI in 2021) are often not the best ones to execute on it (typically, lack of experience or capacity to handle the pain of growth while marketing).

What's most interesting is that most people aren't just "one type". In your life you go through multiple roles. Just like how most people, regardless of their income at age 20 will be earning top 35% income by age 35, people move up in their roles (and struggle at a young age to understand that this will happen to them). It's all about timing and age.

I believe some investors go for multiple teams purely because the ratio of those types is different - like a different risk profile. They invest in one with 80% AI boffins, and one with 80% business folks, and one will likely win in the circumstance of the moment (in the ChatGPT era - mostly the business folks. In the data driven AI era - mostly the AI boffins)

How are you using the GPU on the AMD 13? Is it GPT4All with Vulkan or something?

I have struggled to get the GPU detected/used.

Oh right, I just wondered if it was a loan word from German. I am hearing it more and more in English.

In regard to your Kingdom question, no.

- Humans are always peer relative, no exception.

If everyone around you is affected by the same thing, it doesn't affect you as much. It doesn't affect your chances to marry, have children or gain status, which tend to drive a lot of people (look at mimetic competition). When interest rates go up, it exposes the 10-15% of the population who were heavily debted, but the majority do fine, moan together, and move on, saving more cautiously.

So it does make a difference. Online Kingdoms are transient and flipping a switch; information advantages are more abound. That is not so true offline - the people you meet in person are roughly speaking, going through the same challenges on a macro economic scale as you.

People here rightly have an issue with the fact that technology is allowing consolidation and reducing the number of entrepreneurship options which have no fundamental reliance on a platform entity. Even though the rise of online sellers (Etsy, Ebay etc) has made the number of self-employed people or company directors rise on paper, it had not kept up with increases in population, and there are far less self-employed people earning a wage above the median salary wage than there were in say the year 2000. If you look at Microconf stats you will see that today the majority of entrepreneurs are earning similar amounts to employees; once you adjust for skill difference (entrepreneurs tend to have higher market value than the average employee), you can see something is going a little awry in the economy.

I agree with your point you'd waste too much time building from scratch. However, generally the bar for businesses and software has risen; I think this is because whereas in 2000 you might spend 50% of your time working with polished products from big companies, today you spend 80-90% of your time in Google Docs/Instagram. A single bug or slow load in that remaining 10% makes you quickly search for an alternate solution. But the fact building on existing platforms can abate the problem of delivering high quality value to the customer does not remove the risk it introduces to rely on a specific platform with arbitrary or non-existent customer support.

Back in the 2000s for example, people actually believed that technology would enable more global competition and more distributed business. So it's curious that it has turned on it's head.

Furthermore there is reason to be personally optimistic: over the course of their lifetimes, most people reach at least the top 30th income percentile for 10 years or more. In other words, with more experience and skill, most people earn enough to save money, and this applies across almost all demographics. Income correlates far more with age than basically anything else. Which is great because if you feel rubbish now in your 20s, your upwards potential is massive. It doesn't matter if in 20 years the dollar has collapsed, your income will likely be in the top 30-40%, and that will give you a significant edge in the economy, whatever it is relatively like for your peers.

Prison.

And dependency chaining. But yes, lots of BM25.

The 2000s and even 2010s was a wonderful and fairly theoretical time for linguistics and NLP. A time when NLP seemed to harbor real anonymized general information to make the right decisions with, without impinging on privacy.

Oh to go back.

Out of interest how come you use the word "mode" here?

Just watch out because socketio must be version matched for client-server or you will get the most annoying errors and state inconsistencies in the world. That's a scary production update let me tell you. Version 2.3.0 still scares me to this day after trying to upgrade that without production downtime.

I sympathize with you, I had one too. Luckily it was small.

It seems luck of the draw. My old React projects (old as in 2018) still work great with class components. I guess the Vue guy did say he would be more revolutionary, when he launched it.

How did you get his real email?

Not true at any company or university lab I have worked at recently. Particularly research labs and professors have switched to just accepting academic emails recently which I somewhat oppose personally.

One small tip I didn't appreciate for far too long with "mac-peer" laptops that have fans is to use them unplugged. My laptop almost never spins up fans when it is unplugged, and I can get a good batch of work done with the battery levels of today and fast charge over lunch. It's still not quite as ideal nor as long a battery life as Mac. But it feels so beautiful.

For the first time on Linux I feel better, like I am not just making sacrafices for values but like the actual whole all-around experience is better in most ways compared to my work Mac (M2 Pro so fans abound and not as aesthetically pleasing as the Airs IMO). It's instantly snappy, I have a nice large SSD, I've already swapped out RAM, no issues with key software, I have a theme with a desktop experience I prefer over the Mac one, and I can go to a presentation and type without fans stressing me out. As someone in AI for a while, personally, I don't value GPUs or NPUs, but that would be a difference. That's really leaps and bounds over Linux from 2016 or 2010 on Laptops.

You are right, but it's a little misleading (as it sounds like is the usefulness of your work nowadays) - Comparisons on language modelling prowess of BERT/T5 being compared to the default, non-instruct GPT-3 or OPT-3 isn't really that useful if done by size, because in practice we don't use 1.3B generative models, and more importantly, because focusing on default decoding generation without an instruct/PPO step is not how these models are used practically. The instruct models blow this performance out of the water, but instruct plus better-performance-at-size for GPT models completely shows the dominance of decoder-only architectures in my opinion for now.

I think you have to consider that in 2020/2021 many PhDs and Professors attempted to shift grant funded research with BERT and T5 to explore how they could compete with GPT-3 or to express other properties of it that supposedley outdid GPT-3. Very few (besides sentence transformers) succeeded. It's not like this is an unexplored niche. A lot of people in denial were trying to keep on with BERT research for a while despite the fact their work was essentially made obsolete by GPT-3.

(and notably Table 1 and Figure 4 are cherrypicking the smallest size with the largest gaps in task difference, and a size we know decoding is not performative at - 1.3B param mark - the characteristics and conclusions the authors come to (wow, BERT is trained on less data but does better!) obviously can't be made at larger sizes because the actual GPT models become much larger)

Yeah so - I would hire you if it was for a relevant role.

But for the sake of another perspective, here's how your description reads:

"I’ve made 4 personal projects that currently have over 1000+ daily active users (numbers are ~1500, ~6000, ~14,000, and ~430,000) all created within the span of the last year. I’ve also sold software." --> "I have made personal projects - in fact so many I'm always at it, with a bit too many to make you comfortable. These are so successful and so quickly created that I can clearly do these once I'm with you - and I have the skills to sell software, so I could monetize them. If and when I want to. This says nothing about my ability to follow requirements or work with others"

(my advice: focus on the most successful project, say it's non-profit, and try saying "listened to customer feedback" instead)

"On top of all that, I have an Engineering degree, 4 years experience, no breaks in employment, study on my afternoons and weekends (C, Go, C#) and take extra university courses, and have some other high level achievements/recognition."

That all sounds fine.

"You should be able to coast a bit in life." --> Honestly if you lead to employers with saying that you've sold software personally and have 4 side-projects, you haven't been searching for a job long enough to realize that's unwise in this market - you've potentially been coasting on your job search. Have you been studying C/Go/C# instead of applying to jobs because you prefer studying?