HNHacker News
TopNewBestAskShowJobs

isusmelj

302 karma · joined February 28, 2019

Deep learning enthusiast interested in solving real world challenges.
submissionscomments
isusmelj··on LLaMA now goes faster on CPUs
Is there somewhere an overview of the progress we made on the software side for training and inference of LLMs? It feels like we squeezed 10-100x more out of the hardware since llama appeared. This crazy progress will probably saturate though as we reach theoretical limits, no?
isusmelj··on Suno, an AI music generator
This is really cool.

Song: https://app.suno.ai/song/83680b6f-db37-44de-adf9-3f7fff6b79d...

Prompt: A 90s hip-hop song with a male singer with a deep voice singing about how AI models are creating new songs after being trained on all the data of artists. Talks about AI models are stealing the show.

Lyrics:

[Verse] Step back, my friend, 'cause the future's here AI models spittin' rhymes, oh so clear (oh so clear) Trained on data from all the greats Now they're droppin' beats that dominate (dominate)

[Verse 2] Don't need no ghostwriters or melody makers These AI models are the true risk-takers (oh yeah) Analyzin' every flow, every precise word Stealing the show, that's just absurd (it's absurd)

[Chorus] AI takin' over, breakin' the mold Stolen styles, but they're icy cold (they're icy cold) The game's been changed, no human control The rise of the AI flow, takin' its toll (takin' its toll)

isusmelj··on Alphabet Earnings Release FY23 Q3 [pdf]
Can anyone explain why servers are now more useful for longer than they used to be? I feel the world is spinning faster, and new tech comes out faster. Especially with faster networking and more compute.

In the document they mention this:

In January 2023, we completed an assessment of the useful lives of our servers and network equipment and adjusted the estimated useful life of our servers from four years to six years and the estimated useful life of certain network equipment from five years to six years. This change in accounting estimate was effective beginning in fiscal year 2023, and the effect was a reduction in depreciation expense of $977 million and $2.9 billion and an increase in net income of $761 million and $2.3 billion, or $0.06 and $0.18 per basic and $0.06 and $0.18 per diluted share for the three and nine months ended September 30, 2023, respectively.

isusmelj··on US annual inflation declines to 7.7% in October vs. 7.9% expected
The way I read it the price of things like eggs increased by 43% (on table 6 which is seasonally adjusted it's still 10%)? Does anyone know why?
isusmelj··on Apple security bounty upgraded
I just hope you get the money transferred to your bank account and not in Apple product vouchers.
isusmelj··on GitHub Down?
Official uptime did not mention any issues. Our team is mostly in Europe. Asking the crowd here seems the most simple and reliable way to figure out if this is a real problem outside of our control and whether there might be a workaround.

I don't get why so many services have an uptime service which does not help at all. Why even adding one?

isusmelj··on GitHub Outage
Same issue. Site is also very non-responsive.
isusmelj··on My take on how DuckDuckGo can outperform Google
Growth of searches of DDG seems to stagnate? Any idea why? https://duckduckgo.com/traffic
isusmelj··on U.S. annual inflation rate drops to 8.5%
I'm surprised how much everything seems to depend on fuel prices. If gas & oil goes up everything gets more expensive as production and transportation costs increase.
isusmelj··on To uncover a deepfake video call, ask the caller to turn sideways
Deepfake models are trained on very similar data. They don't generalize well, usually. E.g. we take lots of data from YouTube videos of a single person under a specific condition (same time, same day, same haircut etc.) I know that as I spent quite some time researching these models and worked on a deepfake detection startup. Purely looking at it from a technological side, it's a cat mouse game. Similar to an antivirus software. A new method appears to create deepfakes. A new detection method is required.

However, we can also make use of the models to not properly generalize and their limitations of the training process. Anything that is out of distribution (very rare occurrence in training data) will be hard for the model: - blinking (if the model has ever only seen single frames it will create rather random unusual blinking behavior - turn around (as mentioned by the author, side views are rarer in the web) - take off your glasses - slap your cheek - draw something on your cheek - take scissors and cut a piece of your hair

The last two would be especially difficult and funny (:

isusmelj··on Teams is killing my Mac every day
Not solved yet. I still struggle a lot with Teams. Ended up using the browser version instead. Used fine until today. Suddenly camera was not working and laaaagg. This is bad. Google Hangout and Zoom work fine. Since years...
isusmelj··on Launch HN: Nyckel (YC W22) – Train and deploy ML classifiers in minutes
Hi George, Oscar and Dan. Congrats on the launch! Nyckel sounds super interesting. It looks like you mostly focus on classification tasks. Any plans to also support other tasks such as object detection or image segmentation?
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
All the active learning features and interaction with the platform already works with different frameworks such as Keras, Tensorflow or Jax. The part for training self-supervised learning is currently only available for PyTorch. We don't have focused yet on bringing it to Tensorflow. But it's definitely something we should look at!
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Yes, sorry. I meant he started a good conversation with his tweet.
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Scale is one of the biggest companies in the data labeling space. They recently introduced Scale Nucleus which goes in a similar direction as Lightly. However, whereas Nucleus works well with already labeled datasets we designed Lightly from the beginning to focus on unlabeled data. With the combination of self-supervised learning and embedding visualization of Lightly, you can easily work with datasets where you don’t have model predictions or labels at hand. Note, that Lightly is also available on-prem which is a must for some of our customers since usually there is 100x more unlabeled data than labeled data.
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
When learning ML at university one assumes that the data you have well represents the environment. We do the famous train/validation/test split and train our model.

However, in practice we see that it is very hard to collect a good dataset. There is a great twitter thread from Abubakar(CEO Gradio) about this topic: https://twitter.com/abidlabs/status/1423067498862219267

isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Whether you introduce or reduce bias using Lightly depends on how you use the software.

What we ideally want, is that the data we collect for training, validating, and testing our model represents the environment the trained model will operate in. However, collecting this data can be super difficult. E.g. for autonomous driving systems that would mean collecting data from every corner of the world, at every time of the day, during all kinds of weather conditions, in all 4 seasons...

However, this is a really difficult task. You will very likely end up having more data collected from one city (e.g. because your fleet is much bigger in city A than city B)

isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Yes, depending on the kind of data you want to work with you might want to explicitly get the blurry ones. That depends on the task of the ML model you train and requires domain expertise.
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Most active learning frameworks just use model predictions to find the images where the model has the lowest confidence and ignore the image diversity aspect. E.g. the model struggles with bicycles at night. The problem with this approach is that you might end up adding many new images to your labeling pipeline that are very similar to each other.

However, with Lightly you can additionally make sure you only select images that are visually different from each other. And you always get visual feedback on the selected data in our web platform. The additional control and feedback mechanism allow you to have a more focused workflow.

isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
The library looks great!

Prodi.gy is great but focuses heavily on NLP and speeding up the labeling process. Our goal is really to help you reduce what you label before you use any labeling tool.

We are working with labeling tools as well as providers to streamline the workflow.

isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Thanks, we also try to avoid using AI as much as possible :)
isusmelj··on Launch HN: Lightly (YC S21): Label only the data which improves your ML model
Thanks for the interest and great questions. Responses are below:

>How do models trained with Lightly compare with other approaches wrt adversarial robustness?

We have no benchmark available. Both approaches can be combined. You can use Lightly to pick a diverse subset, label it and then during training/ evaluating the model check for adversarial robustness and re-iterate.

>Can using Lightly introduce additional bias in the model, since only a select few of inputs are being labeled? This may be a concern for publicity purposes.

If we remove bias we automatically introduce bias. BUT we want the introduced bias to be controlled and known.

Bias typically comes from the way we collect data. For example, more data is being collected during the day than during night for autonomous driving. We also have more data collected during sunny weather than rain or snow. We also have more data from cities like San Fransisco than cities like New Mexico. Most of our datasets are biased.

> By the way, I thought ETH spinoff requirements were incompatible with YC requirements - nice to see it can be made to work.

From what we know we are the first ETH spin-off who is part of the YC program. we hope they don't abandon us.

isusmelj··on Hypersim, Photorealistic Synthetic Dataset for Indoor Scene Understanding
With $57'000 for 77'400 images, the costs are below $1 per image. However, there are only 461 scenes => $123 per scene. They mention vCPU years to perform the rendering. I wonder whether GPUs would be more cost-efficient and feasible for this rendering task.
isusmelj··on How redundant is your dataset?
Yes, we're just finalizing other case studies with more diverse data. The trend however is similar. Removing semantic redundancy from a dataset helps reduce annotation and storage/compute costs with little loss in accuracy.
isusmelj··on AutoMLPipeline – Create and evaluate machine learning pipeline architectures
Haven't seen the |> operator before. Is this Julia specific?

e.g. used here: pdec = @pipeline (numf |> pca) + (numf |> ica)

isusmelj··on COVID19: U.S. Officials Warn Americans to Brace for Likely Outbreak
We just got the first confirmed case in Switzerland today: https://www.telegraph.co.uk/global-health/science-and-diseas...
isusmelj··on Show HN: Decentralized Automation Framework in YAML
Thanks a lot, I'll have a look at the links you shared!
isusmelj··on Show HN: Decentralized Automation Framework in YAML
Could someone give me an example of when I would use it? It seems to me rather complicated by reading the text even though it promises a simple decentralized solution.
isusmelj··on Revolut raises $500M at a $5.5B valuation
I heard a lot about N26 and Revolut here in Europe. Is there any US company doing the same?
← PreviousPage 2 of 2