warangal

I started using Nim because i wanted to port some of machine learning models written in python with the idea of making them more portable. It was a lot of work as community is relatively small and a new user would end up writing a lot of code.

But Nim has a pretty solid standard library with clearly written code and an awesome community to help with problems. I generally read a lot of standard library code to expand my knowledge of language and discover common patterns which repeat themselves in a lot of real world problems.

C inter-op is really first class, and as far as i know it has one of best C++ inter-op as well, you can take a look at: https://github.com/jmgomez/NimForUE for a real world example.

I use Nim for my work in both professional and personal capacity and also have written about some of it at https://ramanlabs.in/static/blog/index.html

Really cool to see your approach, didn't know about svd2nim project. Hopefully i will able be to use your approach for some hobby embedded projects.

Thanks again for taking time for a detailed reply!!

Thanks for a detailed response!

I also use Nim quite a bit on linux/windows for machine-learning purposes. But even on these systems, It is generally a pain to try to trick Nim into using a compiler not directly supported, as some original hard-coded flags would be incompatible with the one i am trying to use,for example NVCC .

Follow up question, Do you generally use Nim on ESP like chips which run some form of a RTOS, or also on bare-metal micro-controllers ?

Interesting you use Nim, would you have some resources/pointers for using Nim on embedded devices. Do you use a native compiler or cross-compile it for this (RISC-V) chip ?

For what it is worth, we work on a tool[0] to index all local videos and images and later allowing query just using natural language. It is based on CLIP which has been trained on image-text pairs, but seems to work great for videos after applying some naive heuristics.

[0] https://github.com/ramanlabs-in/hachi

Hey, thanks for checking it out. We also started working on it for personal reasons. Any suggestion or feedback is welcome, you can contact me directly at anubhav@ramanlabs.in .

Some features like Face-recognition and improved video-search are available as premium features for now but working on to let users preview those features through a demo.

I think, it is really underrated how much faster numpy approach is for low-key workloads like personal images and videos where you would generally have a few K embeddings and it is also possible to hide some latency by sharding and stream results as soon as a shard is done. We work on a similar project[0] to make it easy to index personal images and videos. Before using numpy, we also looked at libraries like Annoy to index the embeddings, but being approximate sometimes such libraries would leave out most-similiar image/frame. For personal data, we found it is better to depend on exact-search rather than an approximate one even if it comes with a speed tradeoff.

[0]: https://github.com/ramanlabs-in/hachi

Interface really looks nice! We also develop a similar project but for indexing personal videos and images[0] based on CLIP. It is a Flask based application behind a caddy proxy. [0] https://github.com/ramanlabs-in/hachi

I write most of my code in Nim language, and it generally involves wrapping up some C codebase, or calling DLLs routines and working with raw pointers. Although Nim has it own GC or ARC to handle memory management for its native data-structures, i find it really helpful to define a `destructor` for any arbitrary objects/structs. Since language can track lifetime of any object/struct during compilation and can call corresponding `destructor` for that object, which generally is just a `free()` or `some dll exposed routine`. I end up using language to handle all the manual memory management even for existing C codebase. With new ARC, all destructor calls are inserted with compilation and hence no extra GC based runtime to deal with, and even works flawlessly with GPU programming stuff too ! I haven't used many programming languages, but personally find it really useful to use any language as an assistant to help solve issues outside of their intended context.

I also have written about patterns i find useful when dealing with existing C codebases at https://ramanlabs.in/static/blog/raw_memory_management_with_...

Compiling extensions to python from another language also provide opportunities to fuse two or more operations, resulting in speed up even for cases where fast but independent implementation of operations are already available in python. And may be really worth it if that functionality is needed frequently. I have been writing python extensions[0] using Nim language for a while now and it has been a very smooth experience every time.

https://ramanlabs.in/static/blog/Generate_Python_extensions_...

Videos encode so much information, and it looks pretty cool when such projects extract higher level information to play with. And recent models like whisper and clip are amazing to help make sense of that information even for personal projects.

We are also trying to do something similar[0], but still a lot of work remaining. Idea is to allow real time processing of any video using just a CPU. [0] https://www.youtube.com/watch?v=E7UPj9blnWc

We are two friends working on a native app to analyze videos using computer-vision [0]. We have been developing it for more than 2 years, to make sure it works in real-time for all CPUs. Not sure about impact but it is really cool to play and analyze videos in real time and help unlock some of the values stored in lot of video libraries. [0] https://ramanlabs.in/static/videointel.html

Video player looks cool! Even though this is based for badminton, i have found this one <https://cvit.iiit.ac.in/research/projects/cvit-projects/badm...> quite interesting for sport analytics projects.

I think interesting moments bookmarking is more of an open-ended problem and would very much depend on the game, but large computer-vision models like CLIP have proved to be really useful in recognizing general-purposes activities. You could sample frames uniformly from a game video, index them with a text-captioning model, then find/curate a subset of those captions based on your definition of interesting, and then use those curated captions to look for those moments in future videos, i think!

Thanks for suggestion! Haven't worked with Nix ecosystem yet, but definitely going to check it out and see what works in the end.

Yes, we should be able to containerize it easily as we already tested it for virtual linux machine. Is there a way to contact you, in case we add this and more features in the future?

Still a lot of work pending on user interface side, but this is definitely on the TODO list. We are also working to allow users to search for a lot more categories of objects in the upcoming releases. It has been a lot of work, so thanks for feedback and checking out!

Thanks for checking out. We are also working on SDK to make it easier to integrate with existing applications. Linux version is almost ready, just looking at ways to distribute dependencies a bit more easily.

Yes, ffmpeg is one hell of a library and documentation doesn't make things easier. You have to ask a lot of questions to understand the low level details, and also have to answer them yourself by doing experiments.

For the point about packet freeing, i think ``av_packet_unref`` is supposed to de-reference the ``encoded video/audio`` data being read in the RAM, so that internal ffmpeg code can deallocate it. It has to be called for every ``av_read_frame``. ``av_packet_free`` is supposed to release the generic ``packet`` object/struct allocated by ``av_packet_alloc``. I learnt this while making a simple video player based on ffmpeg library.

Even though Nim by default works with a GC to manage memory, I regularly use Nim to work with direct-memory/raw-pointers and can still have increased memory safety by using HOOKS and ARC. I have written about my experience here[1], in case somebody is curious.

1. https://ramanlabs.in/static/blog/raw_memory_management_patte...

We started a company[0] to make it easy for developers to integrate ML models in their existing applications.

[0] https://ramanlabs.in