The code is available in the repo: https://huggingface.co/spaces/osanseviero/draw_to_search/blo...
It first uses a simple CNN to get one of the 100 classes. It then uses that class to retrieve images using CLIP. The original CLIP Space that helped for inspiration is https://huggingface.co/spaces/vivien/clip.