ttt3ts

327 karma · joined December 21, 2022

ttt3ts··on XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

It isn't script kiddie level but it isn't hard I finetuned a 15B parameter reddit bot with an afternoon of time and a day of training on a 3090. Bot got a few thousand Karma in a couple of days before I turned it off (proof of concept done).

If all you have is an M1 or whatever, ya, you need a real workstation and depending on your use ChatGPT might be cheaper/better.

ttt3ts··on XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

Do you know a dataset for fine-tuning roleplaying?

ttt3ts··on XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

https://huggingface.co/blog/stackllama

You can easily finetune 7B or 15B LORA model with that on consumer GPUs.

ttt3ts··on r/startrek has just launched its own Lemmy instance

Join a different instance with a different color scheme?

ttt3ts··on Google Cloud Workstations managed development environment is now GA

Also, that isn't exactly a powerful machine at 4 cores and 16GB of ram. Probably have to double it or what is the point? Security, maybe?

ttt3ts··on Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

I own a P100, P4, M40. They either lack ram or speed. Also, unless you are putting them in a server you have to cool them.

If your goal is to learn ML don't tinker with very obsolete hardware. Rent or buy something modern.

ttt3ts··on Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

No clue but if you want learn/finetuned ML use a Linux box otherwise you will spend all your time fighting your machine. If you just want to run models Mac might work.

ttt3ts··on Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

Source? I own 3 and I would take a single 3090 any day. M40s are simply too old.

ttt3ts··on Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

You can finetune whisper, stable diffusion, and LLM up to about 15B parameters with 24GB VRAM.

Which leads you to what hardware to get. Best bang for the $ right now is definitely a used 3090 at ~$700. If you want more than 24GB vram just rent the hardware as it will be cheaper.

If you're not willing to drop $700 don't buy anything just rent. I have had decent luck with vast.ai

ttt3ts··on Windmill: Open-source developer platform to turn scripts into workflows and UIs

Thanks for the clarification.

Looks like you're a new y combinator startup. Kool. I would suggest you find away to allow existing code to be reused. Deno market share is very small and experienced/larger teams are not going to want to rewrite and learn that stack. They would have to depend on you, a small new company. Too much risk.

ttt3ts··on Windmill: Open-source developer platform to turn scripts into workflows and UIs

Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts tied to a weird sandbox runtime.

Any support for docker based workers?

ttt3ts··on Releasing 3B and 7B RedPajama

Finetuning which can easily be done on consumer hardware and can give these models a lot more power for specific applications.

Also, ChatGPT just can't do a lot of things because of their "rules". I was doing question answering about products on Amazon with ChatGPT and refused to answer any questions about underwear, certain books/videos, etc

ttt3ts··on Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds

Not a good comparison. CPU vs TPU. Also, whisper.cpp perf varies a ton depending on hardware. Avx512 or neural engine?

ttt3ts··on Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds

Yea, whisper runs fairly quickly on really old GPUs and leveraging CTranslate2 (underlying tech in your link) you need < 4GB of GPU RAM to run the large model.

This might be faster but an order of magnitude more expensive? Also, huge lockin to one provider.

ttt3ts··on IPFS-tiny: attempting to take IPFS to Space

+1 for academic torrents. Torrents are a lot more mature and has a better culture around seeding to ensure models and data stay alive.

ttt3ts··on IPFS-tiny: attempting to take IPFS to Space

No, was scraping TBs over it when NFTs we're a thing. It murders CPU, garbage cleanup is so bad you should just disable it, causes network issues.

Seems like it's momentum is gone and if you haven't interacted with it yet you probably shouldn't bother.

ttt3ts··on TPU v4 provides exaFLOPS-scale ML with efficiency gains

Nvidia's last gen...

ttt3ts··on Faiss: A library for efficient similarity search

Faiss supports FLAT aka not approx nearest neighbor. Also, all the vector databases use ANN too because nearest neighbor is hard.

Or are you referring to the ability to store data in addition to the vectors? In which case, you can pair any time tested DB with the index avoiding the hype of DBs that might be gone in a year.

ttt3ts··on Faiss: A library for efficient similarity search

Scale, Faiss supports a bunch more algorithms. A much larger scale.

ttt3ts··on Samsung's AI photo feature adds creepy teeth to baby photos

"Cheaping out" is not the best description. Apple has riddled the space with a ton of patents. https://insights.greyb.com/apple-facial-recognition-patents/

ttt3ts··on The teen mental illness epidemic is international – Part 1: The Anglosphere

So you can? (serious question not sarcastic)

ttt3ts··on The teen mental illness epidemic is international – Part 1: The Anglosphere

Or the world has always had issues and the lense through which we now view the world is an everpresent fear coaster.

Step away from the news & social media for a few months. You can't effect the outcome anyway and you might find you're happier.

ttt3ts··on The EV Transition Is Harder Than Anyone Thinks

Lol, not quite that jank yet. Maybe in another few years.

ttt3ts··on The EV Transition Is Harder Than Anyone Thinks

Don't worry the butt warmer will have failed long ago in your 15 year old EV

Typed while sitting in my 20 year old pickup truck whos manual windows still work but not most of the electronics :P

ttt3ts··on Vector database built for scalable similarity search

Redis uses too much memory and supports only two NN algorithms FLAT (very slow) and HNSW if you start indexing millions of large vectors you will quickly understand the problem.

That being said, many of these DBs are overcomplicated for most use cases. Redis HNSW will work for many use cases.

ttt3ts··on Vector database built for scalable similarity search

Don't start with mullivus if you're learning. Too much yak shaving. Try https://github.com/criteo/autofaiss.

Also, TBH, it is a lot cheaper to run a simple faiss index.

ttt3ts··on Vector database built for scalable similarity search

Learn faiss first. Mullvus is very complicated in comparison and faiss will work in many cases.

I mean look at their system diagram https://milvus.io/static/0bc2e74d0a1b20bbfb91bdbd03f77e5e/bb.... Not fun...

ttt3ts··on WhatsApp would not remove end-to-end encryption for UK law, says chief

I don't care what Meta does they are big enough to solve the issue.

I was talking about smaller messaging apps that can't just refuse to comply.

ttt3ts··on WhatsApp would not remove end-to-end encryption for UK law, says chief

Just block the UK and make everyone use a VPN this is already the case with age verification for adult content.

ttt3ts··on Facebook disclose the carbon footprint of their new LLaMA models

Who cares? Expected lifetime CO2 for an F150 is 100 tons. So, equivalent to 10 more pickup trucks. Seems like the progress is worth the cost.

← PreviousPage 4 of 5Next →