HNHacker News
TopNewBestAskShowJobs

james-revisoai

153 karma · joined April 16, 2020

submissionscomments
james-revisoai··on Tell HN: Duolingo's “resurrection nudge” goes beyond dark patterns
Duolingo overcame historic monetisation proportion challenges in Q4 2022, but almost immediately after that safe harbour, AI came along.

Now Duolingo are close to OpenAI and have collaborated with them, but with the likes of YC funding AI language learning competitors, I am not surprised they have deployed this and other experiments.

That said, Duolingo have a tendency to run A/B tests and regional AB tests. What country are you in OP? This may sound strange but they may have done this test in a country where it is definitely legal in order to measure what in theory would happen doing this organically as a feature in other areas etc.

james-revisoai··on Ask HN: How are you improving your use of LLMs in production?
I think the point is they already had those components or familiarity with them in their system hooked into existing data. You could add semantic search with text vectors 3 years ago within 1 hour and so the need for a service isn't there.

They didn't need a simultaneous second DB for records just matching by a foreign ID key based on a new service.

I have seen it too (as somebody who studied Information Retrieval and these systems and implemented aspects in production at 2 startups, including more recently these semantic search approaches) - you can store vectors (floats) of length 384, 512, 768 or 2048 like most of the current ones easily alongside existing data without typically making a dent in overall data storage, and load into memory for semantic search according to your needs (for example, for big data you can use more bucketing approaches with don't guarantee the nearest neighbour, but one of the roughly closest ones, which is often a trade off worth making).

It's not only not a hassle to do that, but overall superior to using the vector DB online tools for developer experience too. I suspect at 3 million+ rows you begin to have struggles, but barely any services will get there. And if you do, you may try other techniques (dimensionality reduction to 3x smaller often only loses a tiny portion of the data accuracy, but saves 2/3rds on costs), or employ cutting edge embedding techniques like Custom Matrix Optimisation, multi modal alignment, and others that these services don't necessarily offer.

Sure it's good for the beginner/a tutorial website, but it doesn't solve a problem for developers in information retrieval imo.

james-revisoai··on Ask HN: What OCR tool do you use in your project?
Explore different page segmentation modes and make sure you are using v4 (it's a massive step up)
james-revisoai··on Struggling to understand the value of non technical cofounders in tech startups
Value isn't a simple equation - it's often multiplicative.

A co-founder with distribution options allows you to hit the market multiple times (sometimes this is true for investment), greatly increasing the chances of something catching on. Often times, they can save you going the wrong path with the product too.

A co-founder with technical skills can make the first version faster and strategize product work later as compared to other options. Except in the case where you are doing Truly Innovative Technical Work in which case you probably want a pro-team of mostly or all technical leaders with perhaps a CEO-style visionary for PR and some strategy, that however is not a startup so much as a moonshot, a business which operates differently.

These two skills are much more multiplicative than additive for most startups.

Technical people also get unique benefits: they can also opt away from dealing with some stress, people problems, etc - it's a deal a bit like insurance in that sense. And the risk staked by the business size is larger: a technical person whos product does even slightly well can get hired into a management role at another tech startup working on R&D. This is rarely true for CEOs unless they make it big. So the failure/"just about ok" stage that 70% of ideas leave at, is much better a position for CTOs than CEOs.

In your position I would try to launch several projects alone as a technical founder - the value of distribution will be made clear, making you value your co-founder more, or you will succeed - either result is good for you.

james-revisoai··on Ask HN: Isn't GPT-2 enough for most tasks nowadays?
The quality difference does matter.

For a while I ran an AI app with summary features, with T5/BART in 2020 and then generative from 2021 (using GPT-3 API). It's an unfortunate truth that when I tried using Babbage (GPT-2 size), it was ok for happy path examples, but user retention crashed.

Even Curie was much worse than Davinci in terms of user satisfaction.

One interesting finding from my data was that shorter outputs had less difference in quality between models - longer responses, it mattered more which model you used (this was before Chat, and even before instruct/RLHF GPT models, so before the constrained output forms of verbosity conversational defaults)

For AI applications, users have a 1 strike policy in general for trust. So you kind of need to overdial things to retain users. Might be different for dataset work, but there absolutely is a quality difference over enough uses.

BART or T5 were "better" than barebones GPT-2 for summarisation initially, but that does not mean anybody really uses them for summarisation (in a sense, the task they were most optimised to do during creation).

I do think most of us feel summaries from latest models are getting to as good as is reasonably expectable. So sure if GPT-5 comes and is particularly great at remaining tricky tasks, why not stick with 4 for summaries.

That's not to mention context length is longer in terms of easy "do it today" ways to access a model for ChatGPT or GPT-4, vs GPT-2 models which are much shorter context lengths in terms of input.

james-revisoai··on Ask HN: How to do market research for product?
What's your advice on doing this for products in rapidly changing markets?

Example: Say you are an AI graduate who has worked in AI for a while before ChatGPT.

Suddenly a competitor spins up something you were also developing - faster with ChatGPT - no problem, you launch it too.

One month later, 20 more competitors have popped up, but none really gained too much share. It seems ok.

Two months later, a competitor with funding launches, but now the product is free.

You start losing users to FreeComp, who also buy conference attention.

Your users say well, if you add this advanced feature - then we'll pay - but not now we can go to FreeComp.

Can you trust that? For how long? How would you weight it?

james-revisoai··on Orthogonal Representations in NLP
Use different embeddings according to your need.

The mini-V2 (default sentence transformer ones) are far superior for these tasks than many others.

The query-document snippet style ones (trained on MS-Marco) are great for matching definitions etc or QA.

The implicative and other more semantic embedding types are ok but usually too slow to be practical, especially cross-encoders.

The Word2Vec ones are very fast, but not deep. The main usage of terms dominates in cases of polysemy which will obstruct your goals. So I recommend exploring cosine similarity of SBERT.

it should be said given the predominance of GPT in these tasks, besides for academic purposes, these techniques are not particularly useful these days - GPT will be much more useful for almost every end task (every task I've tried it has been, but I accept their could be exceptions, as somebody who used to compete well on Kaggle for sentence embedding/classification kind of tasks over the last few years)

You can subtract or add embeddings or pool embeddings are mentioned but these won't exactly remove concepts. What you would do better to do is remove the embedding which represents the pooled average of other animal nouns (Dog, Cat, etc) rather than "animal", because "animal" will be used as a verb, will be used as a collective, it won't probably get to what you're aiming for.

james-revisoai··on LLM Powered Autonomous Agents
The probabilities are fun (and ability to manipulate/mask/logit bias them), but small nit pick: Beam search isn't popular because it results in less human like text and takes more processing time, as a concept it sounds great in theory, but it's generally worked worse in practise compared to the next token with temperature, and results in text that is not human like, the gap is larger with larger models too, it doesn't get better:

https://blog.fastforwardlabs.com/images/2019/05/Screen_Shot_...

The way most GPT models are set up, they don't edit, but there was a lot of work (made redundant by OpenAI) on using gap filling tasks (masking) to iteratively improve or construct or otherwise edit and formulate logic using the encoder-decoder models of yesteryear - in many ways, I think it would have been more exciting if that approach powered the likes of 'think step-by-step' results.

james-revisoai··on Ask HN: How to Get Software Based Business Ideas?
Not great advice in a world of ChatGPT at your fingertips.

You generate race-to-the-bottoms like the ChatPDF tools by leaning into easy to copy areas that online popularity highlights, rather than your gut feel around problems - how many of those existing ideas competing with ChatGPT for example are still online 3 months after many launched in Dec/Jan/Feb/March? Check theresanaiforthat, and even though they remove dead pages, under QA(or I think it's now a new catagory depending on the aggregator tool) you will see many dead ones, either outright no site any more or they do not work. Second issue: From my experience, it doesn't feel good to do this so you'll lose motivation...

james-revisoai··on Ask HN: What does it take to be a CTO?
You likely have the skills to get a product out the door, especially if you find a way to step back or have someone ask you diligently "can we skip this", "can we not reinvent this", "is this overengineering?", "what's the one we should prioritise, and the one we should drop?'. Especially if you've been doing ML research and fiddling about - there is limited time for that.

The problems you might have are along the lines of when you get lots of customers. Scaling is horrible for tech, especially anything involving cloud quota requests or newer libraries. You will have the secondary issue of customer support, which at first you can deal with, but quickly can zap 50% of your productivity due to random requests. For CTOs, try and get a reliable dev on your team to take increasing support workload, or hire a dedicated support role.

The two problems you face as a succeeding CTO are quite different. So really do ask yourself how you would handle the customer support issues. The type of startup matters here. Is this urgent for every client, if it breaks? How would you feel if a ping came in at 10:40pm? If your great 5* rating drops to 4* because of bad support, who takes responsibility? Do you entrust your CEO to understand the importance of segmenting support duties from development to ensure your velocity can stay decent?

+ add on presentation and graph making skills if you are raising investment

james-revisoai··on The Secret Sauce behind 100K context window in LLMs: all tricks in one place
As far as I know, Firefox still allows this for any expired certificate which at least has correct domain details and authority (e.g. it once worked, which some dev should validate).

SSL version or cipher mismatch can be from other causes. For example, the server might be responding with a html page that your browser is interpreting as https or vice versa, such as if the developers run http for local dev and https for prod and something gets confused.

james-revisoai··on Ask HN: React UI Libraries – Share Your Experiences
1. MUI, Chakra, Bootstrap

2. Elderly appearing/Generic appearing, or PRO locks certain components. Inconsistent components between mobile/desktop. In particular the "scroll tags" on mobile don't tend to look nice on Desktop - it's hard to easily switch out for a desktop-suitable UI.

3. It's a minor problem I wouldn't pay for, but a library that overwrites font awesome v4 icon references to more modern icons would be great, the icons show aging. Likewise if there was any way to have fallbacks for icons (e.g. having developed for over a decade, I confuse icon names between frameworks.. would be great if there were more synonyms or sensible synonyms or something higher tech). Also, few support animations as components, which I feel should change soon as GPT is making it trivial to generate fairly good animations. Not sure if design will trend this way though.

4. Looking good. Not changing with updates.

5. Will it price us with a license? How different does it look (code) to what we usually use? Close enough or less code? Great.

james-revisoai··on Ask HN: Is the rate of progress in AI exponential?
Well really, what people in the field felt and saw was that on metrics for measuring tasks, what had happened in computer vision with giant jumps in performance for tasks previously considered hard to ever get to human level (like identifying cats in photos), then came to pass for NLP: Tasks and metrics we never thought we'd see even close to human level without some new theories or approaches to thinking and language, jumped up overnight. The metrics and applications of NLP before 2017 heavily relied on operating on words, tokens and adding in rules-based(GOFAI/expert system) modifications to do things like old-style chatbots. So many tasks now done by 355m tiny models locally had dismal performance, like repunctuation, translating faithfully, really - what turned out to be basically "easy" stuff for a network to learn.

And then add on RLHF and instruct tuning which is bizzarely more capable than anybody reading the RL/T5 papers that lead to these largely expected at the time.

It's the gap in researcher experiences vs assumptions as of 2017 that lead to this description imo. If you had projected the performance from 2012 to 2017 in NLP, you'd have about 3 major advancements, around word embeddings, and there wasn't much room for innovation there (as it turned out, although obviously semantic search and stuff followed to an extent, it kind of existed as "IR"). But from 2017 to 2023, we've had 30 of similar calibre, and they've stacked (GPT-2, then GPT-3, then instruct/RLHF basically).

The real point, as I see it, is that if we have the same jump - we outperform expectations by an order of magnitude - we'd be in trouble. Now maybe the expectations have adjusted, but I can tell you even as of 1 year ago, many NLP labs still disliked and ignored GPT models in favour of clearly relatively obsolete/now academic-curiosity projects.

james-revisoai··on Tips for better coding with ChatGPT
I did read that and find it interesting, but without knowing it was now filtering contextually (awesome!), I didn't consider it for the codebases I was looking at due to the context size needed. Seems like keyword matching vs semantic/index search at first glance but I see you are indeed pushing the envelope here.

Great to see, thanks again for developing it.

james-revisoai··on Tips for better coding with ChatGPT
Great tooling.

I was using CodeGPT but this looks better. And it's exciting to have those command line options to easily insert other contextual input like command outputs.

Can I ask about your plans? CodeGPT planned to add semantic embeddings of code, but I haven't seen any updates for quite a time. The idea being that, like walking an Abstract Syntax Tree of the code, you could include relevant functions and function definitions(/classes/constants/etc), to help improve output quality.

Alongside what you note with module/library versions and pasting in doc snippets, I have often found is necessary to include package.json files to nudge the results towards the right API calls. I think semantic code search might push it finally to being able to work even on comprehensive, multi-service style repos.

For MVC or services architected solutions, I think this could be more helpful than including whole files - though it may be overkill - do you have any thoughts on doing this?

james-revisoai··on Ask HN: Lots of AWS credits expiring soon, ideas?
absolutely, unfortunately you are right.
james-revisoai··on Anything can be a message queue if you use it wrongly enough
Did you detect this MixPanel or something?
james-revisoai··on Ask HN: A programming language designed to be generated by GPT?
That could be a good idea. It already works quite well the other way, to have a sort of universal way to distil requirements into many languages, as Fireship elaborated here: https://www.youtube.com/watch?v=iO1mwxPNP5A

There are ways to coax the models into more and more convergent output (for now existing languages, like JS or Python), especially with contextual text. I think the Fireship guy got it to 80% the same output each time with a slightly non-zero temperature prompt, but it was consistent across many tasks which is fascinating (he doesn't fully elaborate on the set up like temperature/top-k - which must have been > 0 because it wasn't always the same output).

In many ways we are building flows of information, and as humans, we like to think back from the output through processes to the earliest building steps. maybe a new language could capture that, with intermittent steps not defined in the code - rather executed by recall from models or generated on the fly or by some kind of code-generation semantic cache- it would make for so much easier reading, debugging and reading. Afterall the main blocker to people, especailly business people, learning code is typically how disconnected most of the lines are from the actual output. Defining a class and then a main function is so illogical until you understand the frame. Moving away from that paradigm could open programming up to a lot of people who like talking about outputs, and first steps.

james-revisoai··on Ask HN: What can we learn about human cognition from the performance of LLMs
This is a great point to illustrate the importance of the token representation of position, though I didn't find ChatGPT had any problem with the second example you showed with extra words though - it was correct.

I'm not sure the cases you reference work in practise how you describe exactly really nowadays, even though they did in the past (say, 2019) - I agree the wraparound effect could occur like that, though the way the position is encoded in the vector is usually built to work so that it will wrap around at different levels as such (the frequencies w) which include padding tokens as non-zero values (implementation dependent), so in essence for every sentence ending at 8 tokens, the model has many other encodings that indicate the loop, not just the one wraparound encoding. This should avoid that problem and the reason for this relates to spans of tokens and is a problem considered when devising training for earlier transformer models like BART and the MASK/NSP tasks. But in practise, position is learned these days, it does not use sin/cosine anymore.

There is a technical difference in positional embedding vs positional encodings (and the trend since 2020 to learn position) which is quite interesting.

james-revisoai··on Ask HN: Have any managers/team leads been asked to layoff staff b/c of AI?
(not parent but -) It depends on area, generally: Provider APIs(ChatGPT through OpenAPI etc), langchain, huggingface transformers, pinecone/vector DBs are absolutely taking off.

Lots of specialist ML models which required specific data collected carefully for a business task are no longer needed. Most ML roles until now, and research time, was spent collecting data and training specialist models.

General models like ChatGPT or pretrained image models do better than a fresh model trained from scratch or even a finetuned small/medium model (e.g. BERT/T5) ever will these days.

The special ML (pipeline of data --> train --> deploy --> manage/mlops/drift) used tech like PyTorch, Tensorflow, MLFlow.. and for more applicable levels (e.g. deployment), transformers, sci-kit learn, keras. However these are being replaced wholesale at many companies by langchain, huggingface inference API (for vision tasks) and pinecone/other vector DBs.

Langchain is just a smart way to wrap and order API calls to OpenAI/ChatGPT/other providers really, with some prebuilt use cases. Right now there is less on the metrics/output side than with lets say "bring-your-own-data" ML models, which you could measure things like precision.

Now, the old guard of ML (PyTorch, Tensorflow) is still used for training new models, open source replication attempts etc. But newer frameworks like JAX have not really taken off, as they have been entering the community as the community switched to using providers, rather than training their own models.

There remains a subset still powerful for communication with C-suite: Using simple models like K-means to show clusters with readable axis. They tend to use sci-kit learn or R. But this is more classic data science than ML.

There are also areas of AI so far relatively unaffected by ChatGPT etc - time series prediction (like OP, so it's less surprising they are using the old guard technologies), game engine AIs, non-discrete data, recommendation algorithms, some computer vision algorithms (especially Active learning). Some like HuggingFace (a commercial company running transformers Python module) are sort of inbetween given they serve both data-trained and the newer models.

james-revisoai··on Show HN: [Embedditor] - the Open-source Microsoft Word for Vector Embeddings
This is a great way to boost performance. Have you looked at implementing something like Autochapter/semantic chunking? I know your second point leans towards that, but I think such tech would be helpful here.
james-revisoai··on A college student reached $64k/mo in 6 months by being an AI first mover
I wrote more about this in another comment ("Postmortem"), as I developed a similar idea from Mid 2022 to early 2023, but made different decisions.

OpenAI had a serious focus on this that did see adoption; as early as Spring 2021 this was in their docs (linked by others), but more crucially, they quickly added an Answers endpoint(https://platform.openai.com/docs/guides/answers) specifically for this use case (query against uploaded files).

The creator of this project did almost everything perfectly, for sure. They had speed, UI, a converting page, found virality. But they also had timing: OpenAI had banned this open-ended use case till that time, they got lucky that people did not develop Explainpaper competitors, and ChatGPT improved quality while reducing run costs by 10x right when they came into this area, meaning many companies that fore-aware of these changes would have taken it's place, were not trying to enter the market (OpenAI approval was hard, and the costs of models before December 2022 were literally 10x as much for decent quality, making it unprofitable)

james-revisoai··on A college student reached $64k/mo in 6 months by being an AI first mover
Beware that most 2020-2021 OpenAI startups haven't survived to today, especially since ChatGPT allowed free use of most B2C use cases (article writing, paraphrasing, etc). I posted a brief postmortem of my experience from 2021 to 2023 when I did at one point launch a similar product to yours which, if you got into this more recently, may be worth reading.
james-revisoai··on A college student reached $64k/mo in 6 months by being an AI first mover
Here's a post-mortem and the true history of this tooling approach and being a first mover, or not...

Pre 2021: Useful semantic search widely deployed, used for recommendations for sales. OpenAI has an answers endpoint specifically for this technical use case, full docs, and many companies implement this internally.

Mid 2022: People like me experiment with GPT answering using semantic context on dynamically uploaded content, e.g. https://www.youtube.com/watch?v=7V3VkNj2bag - but cannot get approved for this use case by OpenAI.

Nov 2022: GPT had banned open ended responses and chat like interfaces until now. Explain paper is the first tool that allows you to chat with your PDFs with multiple questions, and to use it on many types of PDFs, released in November IIRC and viral on twitter: https://twitter.com/rauchg/status/1596220185727275008

December-January 2023: 15+ Chat with PDF tools are launched, including pre-hyped launches on ProductHunt from pivoting products January+ onwards: Almost daily launches, sometimes with USPs. I launched AnyQuestions.ai which was at the time the only question tool able to work with videos [1]. Chatbase catches virality on twitter and makes some other sound decisions [2].

Feburary 2023 onwards: No platforms since February have succeeded.

[1] But it took a long time to process with Whisper and the interface was more answer focused than chat-and-response focused; I also made bets on including a larger context and using non-typical embeddings (looking for entailment/contradictions of the sentence as well as semantic similarity) which turned out to not be the right commercial choice.

[2] Allowing embedding on websites, and sharing the Chatbase branding that way was powerful, as well as being able to easily work for new websites as a quick solution. It spread easily through competitor FOMO and was a fast product.

james-revisoai··on Show HN: AI Domain Genius
1. Yes, very. Loading indicator could be a little more clearer on time estimate, but great/outstanding overall.

2. No, it seems too direct and not creative. I wonder if the original non-instruct models (curie etc) would be better for this use case than ChatGPT. Because they make more divergent and creative results.

james-revisoai··on Ask HN: How can I improve how I Show HN?
Specifically, we used semantic similarity for shuffled answers (so they are close and not too easy), completely shuffled answers, and AI-generated distractors. We are now going to choose the most popular default (based on retention during A/B/C testing), and also offer the mode as a choice.

The quiz question (though not answer, since overwritten) from the demo you posted was generated in 2020, with T5 not GPT-3. Other and more complex questions were also possible with our finetuned tech on a bespoke carefully collected dataset. It was at the time much more impressive than it is now with GPT models, and we had it out before competitors even launched(the closest thing was old style GOFAI and a few online tutorials on finetuning T5 for similar tasks that had inferior data).

james-revisoai··on Ask HN: How can I improve how I Show HN?
Really good points on the angle there, thank you. And important point that it shouldn't necessarily come from you! (that's what's worked better for us everywhere but ProductHunt, reflecting on it)
james-revisoai··on Ask HN: How can I improve how I Show HN?
Thank you, this bug has been fixed. We recently changed from shuffled distractors to Ai-generated distractors, but as you identified, such a bug could occur.
james-revisoai··on Ask HN: Resources to learn about early startups engineering decision?
Can you expand on some good guard rail metrics you've used/seen please?
james-revisoai··on Venture Predation
A flipside effect to this is advertising and PPC which has trended up so high overtime, especially in education.

Strategies viable in 2019 are largely useless to bootstrapped businesses in 2023: the costs of ads even to acquire customers with low spending power is two to three times what it was (e.g. students)

As someone working on a NLP product for education, the pricing and subscriptions are also dropping rapidly, lead by the funded companies entering this niche (the founders almost entirely are non technical recently). The combination of the two is like a jaw - it's unclear what to do when your technology becomes a trend, other than grab and deploy your own funding and use marketing strategies and technical efficiencies to outlast your competition.

← PreviousPage 3 of 6Next →