pippy360

Please do post to GitHub, I have a Samsung projector I'd love to root

The thing that got me back into pirating is having to pay extra for 4k. When every bit of extra enjoyment is taxed it just kills my joy. Like a lot of people in this thread I really would much rather just pay for a good product than pirate. Thanks to steam I haven't pirated a video game since I was a teen.

Do you have a good/entertaining source for this? I'd love to read (or watch/listen) more about it

I really enjoyed this deep dive on how far video games have come. Also with ~900 views this video is likely to be buried too quickly for the HN crowd to see it otherwise.

That was a very higher quality YT video. It's clearly written by someone who knows when they're talking about even though it's mostly non-technical

yes! couldn't agree more with your long post. Especially this part: "(This is exacerbated when components of the automation require internal-only tooling—the poor data scientist now needs to go read through a bunch of half-written, out-of-date documentation about tools they simply don't care about to do a task that is not a core responsibility for them.)"

"Vertical integration" in my experience has just been turning a group of simple tools into a complex monolith that no one understands and is extremely difficult to debug

Sorry to pile on but, the google sign on also turned me away. I have a google account and usually use it to sign up for sites I'm interested in but when it's used to gate content that could be accessible without a login I just feel like my data is being harvested.

If you really want to keep the login then maybe a message like we won't spam email you unless asked and only want your to log in to protect against bots would help.

It looks great. Quick question: Can i search through my entries? And is there any plans to use semantic/embedded vector search?

Archviz (architecture visualization) is very good for this. Just search “ue5 archviz“ on YouTube

Can you comment on how difficult this was? and how much time it took?

I've thought about doing something similar with some open source games but have just never had the time.

The work is fantastic! I hope you enjoyed making it!

I think for handling truly/non-affine arbitrary transformations we will have to resort to ML. Then we could have matching very similar to how humans do it (where we really don't care if the transformation is affine/non-affine we just care if it's a huge transformation the makes the image unrecognisable). But I really don't know much about ML.

Okcupid talked about this in an article about image hashing [0] and they have a nice quote:

"The end-to-end approach of training a Convolutional Net to embed images such that the positive / negative case distances separate nicely is probably the cool new hypebeast way forward. This would have the added advantage of being able to handle whatever transformations you can think of at training time."

[0]https://tech.okcupid.com/evaluating-perceptual-image-hashes-...

It would be great if I could remove overlapping triangles then I would have near linear growth of triangles for larger images. But it's very hard to come up with a technique which removes the same overlapping triangles (and leaves the same one) and is invariant to 2D affine transformations.

For example if you have 50 overlapping triangles you have to decide which 49 to remove and you have to remove the same 49 on the query image and the matching image. But because we want to be able to do our searches in sublinear time we can't compare the two images and decided on which triangles should stay/be removed, you have to do all that in preprocessing before inserting into the database/querying. delaunay triangulation looks almost perfect but isn't invariant to 2D affine transformations

possibly, I'm not sure what implementation you have in mind?

I find it's the combination of a poor keypoint finder, noise and needing to be able to match partial images is the problem. If I could guarantee that 99% of the keypoints in the query image would be in the matching image and that the extracted hashes wouldn't be affected (hugely) by noise and that most of the query image was present in the matching image then I could simply discard a huge amount of the triangles and only query for a manageable number of them. Obviously that's a lot to ask but the better I can make each part the more options I will have for culling the hashes. For the moment I need to query as many fragments as possible to have a good chance of finding the matching image.

>Why are we transforming the triangles to equilateral triangles?

This is so that the extracted image fragments/triangles are always very similar and so have an extremely similar hash. If I didn't transform the triangles then, for example, a fragment of a query image might be stretched compared with the matching fragment in the database. Intuitively it makes sense that it would be easier to match image fragments if they have the same scale/aren't stretched versions of each other. The transformation to equilateral triangles is what makes the algorithm 2D affine-invariant.

>what if you use ASIFT (Affine SIFT) which is Affine transformation friendly

I only discovered ASIFT while reading this thread, it looks very interesting and useful but I'll need to look into it more. I did test out SIFT as a keypoint finder and it performed really poorly for what I wanted to do but I didn't do a lot of testing. My current method of finding keypoints is a load of crap that I threw together for this demo, a better one would seriously improve the accuracy and speed.

>like if too many keypoints are within some 2D window/I propose that let us do Delaunay Triangulation

I looked into both Delaunay Triangulation and a 2D window as a way to decrease the number of proposals/triangles generated, but had problems with both. I will still try later to implement a 2D window. As for Delaunay Triangulation, if you play around with it on paper you can see that it's not 2D affine invariant, it might still work well for small transformations but there are affine transformations where it fails. Still I think it would be worth looking into more.

Unfortunately it needs a huge amount of work to be production ready. I'd love to see it wrapped up in python though and I'll be continuing to work on it. The main issue you will run into is that the number of triangles computed from the keypoints is n^3 (where n is the number of keypoints), So currently it will take far too long to match large images. I have a few ideas on how to solve this but none of them have lead to quick and easy solutions...yet

I'm currently working on an improved method for getting 2D affine invariant keypoints because the current one just barely works. After that I will be working on handling the scaling issue.

>The big problem is, a moderately complex image can yield n³ triangles

This is a big problem I have been facing. You will notice that all the images I use in my c++ demos are small for this reason. I'm trying to solve this scaling issue but I wasn't able to get a solution quickly so I decided to release this and continue to work on it and hopefully release an improved version at some point (or let other people work on it and hopefully find solutions).

>I was wondering, how did you determine your criteria for matching

Yeah, the query image is broken into a number of fragments and then the hash of each fragment/triangle is checked against the hashes stored in the database using a nearest neighbor search (well you find any neighbors within a certain distance/threshold, I found a hamming distance of 8 to be a pretty good threshold).

Currently only one fragment of the query image needs to match a database image for the images to be considered a match. You could also put some threshold on the number of fragments that need to match but this is only a proof of concept I haven't had time to find the perfect combination yet.

>could this be extended to colour transformations

Yeah, actually after you have re-transformed the triangles to be equilateral you can apply a lot of other techniques to future improve the search. You can even use NON-affine transformation partial image matching on the triangles to match fragments that only partially match fragments stored in the database.

Hey guys, I'm the guy who made this. Let me know if anything isn't clear or if you have any questions.

*I developed this after I saw how poor google (and any other reverse image search engine I tested, like bing/tineye) performed on rotated images, for example google doesn't match this image [0] to it's original [1] (google's neural network will find that it is a cat but won't match to the original). After playing around with trying to make my algorithm (uniform/nonuniform) scale and rotation invariant I found that I was able to make it fully 2D affine invariant (pretty much by luck)

I developed all this in my spare time actively for about about a year. I also have a c++ implementation that let's you reproduce all this but it's just a proof of concept and so it's quite slow. You can check that out on my github here [2]. There are loads of ways this can be improved but I wanted to get the idea out quickly.

[0] https://github.com/pippy360/transformationInvariantImageSear...

[1] https://github.com/pippy360/transformationInvariantImageSear...

[2] https://github.com/pippy360/transformationInvariantImageSear...