UnrealCV
unrealcv.github.io
unrealcv.github.io
(nevermind that the last Unreal I played was in 1998.)
My concern is how many recruiters would actually download and run a 1 or 2 GB application simply to see my resume, when they instead could visit my website and watch some quick videos.
My guess? None.
In the Deep Vision world we as a group are trying to segment, classify and reinforce our NN training on labeled real world data. The challenge is, it's a very manual process to label data - specifically images. The more that we can do inside the computer, for example automatically labeling pixels inside an image, without having to acquire and label real world data (or making it easier to do with real world data) the easier and faster training becomes that can be applied to real world use cases.
The trick is making the virtual world match the real world as closely as possible so that the nets we make are accurate representations of real world scenarios.
Synthetic datasets have been used in the past too, but this makes it much more accessible.
However CV is becoming more and more Deep Net based.
In other words, you would train a NN on input from a footage of a walkthrough of a real building to output rendered footage from the same path (3D data from architects is already there). The middle layer of this quasi autoencoder then is the basis to train fully simulated tasks, e.g. autonomous vehicles. In a way, it would be similar to colorization of b/w footage. Would that scale training data?
The challenge is labeling - or autolabeling pixels.
One thing we are trying to work out is how do you label and build nets on volumes - rather than just pixels? I'm thinking it's going to be a magnitude harder.
https://www.technologyreview.com/s/602317/self-driving-cars-...
Here are two examples of some of the research done in this area:
http://www.cv-foundation.org/openaccess/content_iccv_2015/pa...
Though, at the end of the day, there is no reason why you can't use both.
I can think of a few algorithms that have changed or been created since the 1990s so I have to assume there's something that can be done to optimize it.
There's also a severe lack of "in-browser" CV solutions (which could be fun), instead opting for binding OpenCV in the server-side.
Is it so that computer vision researchers can create scenes in UE4 that can then be fed into their application?
Some similar efforts (though not with open source engines)
[1]http://link.springer.com/chapter/10.1007%2F978-3-642-33786-4... [2]http://www.doc.ic.ac.uk/~ahanda/VaFRIC/iclnuim.html [3]http://robotvault.bitbucket.org/
For example, in security applications you have basically zero data on the sorts of anomalous events you want to trigger on. If you can simulate example cases of those things that means you can actually start applying supervised machine learning.
If not, is there a way of generating levels with code?
The engine supports generating meshes dynamically at runtime: https://wiki.unrealengine.com/Procedural_Mesh_Generation
It is also fairly easy to generate random or pseudo random levels using blueprint (Unreal's visual scripting API) or C++
If you have any other questions I'd be happy to help.
The reason I ask is that I have a large dataset that I would like to visualize.
The ideal situation would be: I could use python to create a giant flat plane, and then iterate over the dataset to generate the level (which initially would just be quads on the 2d plane).
The dataset would be too large to do in-engine in realtime, and the quads are static - I would like to generate the level as a 1 off process, and then have Unreal do any optimisations that it can by "compiling" the level (e.g. frustum culling, quad tree calculation etc) I might have some of those optimizations wrong since my knowledge of engines is still stuck in the Quake 1 era, but you get the gist :)
Do you think this is possible? If this tech doesn't exist, I would pay someone to develop it for me....
How many total datapoints are you trying to represent at once, ballpark? My first instinct is that it would probably be most efficient to feed the data into Unreal and build the physical representation there VS importing a large monolithic 3D model, because then you could make use of instancing, distance culling, etc. This would also mean you could animate the objects separately as well, and manipulate scale, color and the like in realtime.
Again, probably depends heavily on the specifics. In general though, modern gaming GPU's can handle just an insane number of polygons so either way, you should be able to represent a ton of datapoints.
As far as getting data into the engine, it supports .csv importing where you can define your own row types etc: https://docs.unrealengine.com/latest/INT/Gameplay/DataDriven...
I believe there's also a SQL lite support module for the engine as well, if you want to deal with really massive sets of data, though the documentation there is sparse.
My contact info is in my profile, feel free to hit me up if you have any other questions, happy to help.