Hacker News Reader

dnthon Cluster images at scale using DINOv2 embeddings

We released a free tool (fastdup) where you can use to cluster image using DINOv2 embeddings.

dnthon Find image duplicates and outliers – A free, scalable, efficient tool

fastdup is a tool that let you gain insights from a large image/video collection.

It lets you identify image duplicates, video duplicates, wrong labels, outliers, corrupted data, and image clusters.

fastdup is -

Unsupervised: fits any visual dataset. Scalable: handles 400M images on a single machine. Efficient: works on CPU (even on Google Colab with only 2 CPU cores!). Low Cost: can process 12M images on a $1 cloud machine budget.

dnthon Dedup-ing LAION (60M duplicates) and ImageNet (1.2M duplicates) with fastdup

The authors at fastdup ran an analysis on LAION 400M and Imagenet21K. Here's what they found.

LAION 400M

> 60M duplicates. > 962K broken images. > Various label discrepancies.

ImageNet21K

> 1.2M duplicate images. > 104K train/val leak.

fastdup GitHub repo - https://github.com/visual-layer/fastdup

dnthon [dead]

Even pros have dark, blurry & duplicate shots. But disorganization can make it hard to find those special memories. Let's fix that.

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Update - After some code optimization, I got the inference time down to below 100ms. The lowest I got on my Pixel 3 XL is 37ms!

https://dicksonneoh.com/portfolio/pytorch_at_the_edge_timm_t...

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

You're welcome! I'm not sure if I'm the right person to advise on this. But this idea is also known as federated learning right?

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

With flutter you can also build web apps.

https://flutter.dev/multi-platform/web

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Note yet! But I heard things are going to be a lot faster in 2.0. Have you tried?

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Thank you for the feedback! Let me know if you have questions :)

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Thank you!

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Thank you!!!

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Thanks a lot there! I was hesitating to write this piece actually, thinking it's not going to be valuable. I'm glad you find value in them!

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Media pipe looks really cool. I havent tried it. Have you?

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

The pytorch lite package also supports yolov5 models.

I posted on my LinkedIn awhile ago

https://www.linkedin.com/posts/dickson-neoh_deploying-object...

dnthon PyTorch at the Edge: Deploy 964 TIMM Models on Android with TorchScript

Model deployment is painful. Running a model on a mobile phone?

Forget it .

The frustration is real. I remember spending nights exporting models into ONNX and it still failed me. Deploying models on mobile for edge inference used to be complex.

Not anymore.

In this post, I’m going to show you how you can pick from over 900+ SOTA models on TIMM, train them using best practices with Fastai, and deploy them on Android using Flutter.