Hacked Kinect is now a 3D video capture tool | Hacker News Reader

Hacked Kinect is now a 3D video capture tool

adafruit.com

222 pointsepiphany4742 comments

Don't miss:

a) His other video of the system where he shows that measurements of 3d objects exactly match real counterparts: http://www.youtube.com/watch?v=f1ieKe_ts0k

b) His homepage of other experiments: http://idav.ucdavis.edu/~okreylos/index.html

Neat. I did something similar for my graduate thesis project a couple of years ago, but to play pool/billiards on the computer using real cue sticks (for practice and feel) http://mixedrealitybilliards.com

It's implied in the measurement video that he's feeding his captured data through a realtime 3D modelling/rendering tool. Does anyone know what that tool is?

On his page[1] it says "The 3D reconstruction code is entirely written from scratch in C++, using my own Vrui VR toolkit[2] for 3D rendering management and interaction."

[1] http://idav.ucdavis.edu/~okreylos/index.html (frame navigation?? ugh)

[2] http://idav.ucdavis.edu/~okreylos/ResDev/Vrui/MainPage.html

Here it is on the creator's website instead of a lame blog post: http://idav.ucdavis.edu/~okreylos/ResDev/Kinect/

I'm not in this space or planning on investing time into this but I just wanted to say Kudos to Microsoft for creating something that developers are excited about again... even if they didn't indent to do that. I'd start hacking on this if I wasn't invested in a different direction.

I wouldn't give MS too much credit. They bought the hardware from the company that created it.

Yup, here's the reference design in case anyone is curious:

http://www.primesense.com/?p=514

Please keep in mind that the folks who did this did so while being threatened with prison by a cowardly anonymous Microsoft spokesperson.

If money is no object, commodity stereo equipment exists with better range than the Kinect. From Wikipedia:

   The Kinect sensor has a practical ranging limit of 1.2–3.5 metres 
   (3.9–11 ft) distance when used with the Xbox software.

The Kinect was limited by cost, size, and the need to work in poor lighting conditions. But by spacing your higher-quality cameras out (increasing the baseline), accurate depth at 10 meters is a reasonable goal.

One such device: http://www.ptgrey.com/products/bumblebee2/index.asp

Accuracy chart: http://www.ptgrey.com/support/kb/data/stereoaccuracy.xls [XLS warning]

Money is always an object. Any idea roughly what these go for?

Not only bad lighting conditions but repeating patterns, low-contrast backrounds and large areas of a single color. Kinect is far more robust than any kind of stereo vision, and at the same time far less CPU intensive.

I hope he links up two or three of these to form fuller 3d models of a room.

The Kinect works by projecting an infra-red grid on to the room so you can't use 2 of them in the same space (unless you modify the hardware somehow)

Here's a nice video, if anyone's curious about what the infrared pattern looks like. http://www.youtube.com/watch?v=nvvQJxgykcU

Could you synchronize two Kinect devices on one clock and then toggle the display/capture of that grid every odd or even 10ms span? So, only one grid is displayed and processed during its 'slice' of time, with the grids strobing rather than continuously projected.

Software-only, or would that result in too jerky of a motion capture?

10ms seems like a reasonably small period of time. I believe it is considered to be effectively instantaneous in UI.

Edit: Although, I suppose the "refresh rate" of the entire system would then be 20ms, which might be noticeable. Chop that down to 5ms per Kinect, and it might be viable.

Only one "eye" [Kinect] has to notice/process the movement; the other can infer, during the integration step, that the "rest" of a moving object comes with it, rather than stretching out.

Absolutely you can. That's exactly how LED matrix displays work (look up charlieplexing).

Unless the 3d environment is rapidly changing, you don't need that many frames per second from each device to capture a good image.

25 fps is one frame every 40ms.

How about slightly different wavelengths of infrared?

How about projecting different patterns?

Couple this with shutter-glasses and time-multiplexed display. Now you can make a "3d" display that works for multiple viewers.

Of course, the frame-rate would suffer.

Maybe you could turn one on, capture the positions of the dots, and then turn the other on, and somehow guess which one was which when the 3d mesh changes (from motion)?

Last week, a guy from Microsoft Research said that they got multiple cameras in the same space working, but that others in the lab had problems with it. Seems to depend on the setup, but is obviously possible

You could also just use pretty standard scanmatching algorithms to build a model from a sequence of scans from one camera (i.e. waving it around at a reasonable rate).

each unit could broadcast the grid at different wavelengths, so they don't interfere.

Wow, this reminds me of the holographic home movies in minority report. We have the camera, now all we need is the projector.

Exactly what I thought. Also, someone's made a side-by-side video: http://www.youtube.com/watch?v=da3iq9fnyWs

For the frequency of the news about Kinect hacking I get the feeling that the homebrew software for Kinect will soon boost with much more professional features than the official software, esp. since it's pushed with games.

As Microsoft sells these things highly subsidized to claim profit with on the games I see lot's of conflict potential.

I can see indie game developers using Kinect to do realistic motion-capture in games.

I recall seeing this issue pop up in the 'making-of' DVD for Alan Wake by Remedy. When they made their first big game, Max Payne 1, they did the storytelling in the form of a comic due partially to budget constraints. Using a hacked Kinect and cheap PC hardware, game developers could capture a huge range of motions and use video editing software to overlay the required graphics onto the captured motions.

The problem there, I worry, is once someone starts selling an indie game with third party support MS will go after them legally.

They don't have much standing with the hackers atm, but a commercial indie game is another matter.

Which is concerning :(

The point of the parent post was using these things in creating games, not as a game interface. That is, to make animation more realistic. Then again, given commercial motion capture systems are in the 5-10k range for pretty good stuff, I don't suppose that's a major concern for a game studio.

Bah, my comment was something I had in the back of my mind... so I looked at the parent and misread it as being on the same topic, sorry :)

> As Microsoft sells these things highly subsidized to claim profit with on the games I see lot's of conflict potential.

http://www.eetimes.com/electronics-news/4210649/Kinect-s-BOM...

Of the roughly $56 BOM, about $17 is attributed to the cost of the PrimeSense reference system, including the cameras, microphones and processor, UBM TechInsights said. Microsoft is planning to retail the Kinect system for $149, and stands to make a healthy profit on each unit, UBM TechInsights noted

It's rough but still pretty damn cool. Software that does this from footage can easily be in the thousands of dollars. Image the results if you pair a kinect with this: http://www.youtube.com/watch?v=vEOmzjImsVc

Idea: Please, someone somehow need to attach via software a Kinnect and a 3D printer.

I have a 3d printer, but not a Kinect. Range data is not very good at all at creating 3d models. What you really want is that plus a turntable so you can get multiple views. To get a printable model, you need a mesh with no holes. A single range image has lots of holes, so you can't really print the image without filling them by taking multiple images and matching them. That said, I do think I'll make a 3d scanner based on this thing.

Hi, I'm thinking of getting RepRap. Do you find your printer brings you any business opportunities ?

Well, I have been selling some parts, but mostly to other Reprappers. I'm looking for ways to develop something more extensive with it, but it's one of a stack of fabrication-related ideas I'd like to play with, each more attractive. The main problem is that printing is slow, and linear. You can't easily scale it. Some people are building botfarms, but even then their supply is linear, and limited by space.

Pop over on the #reprap channel on freenode and meet the rest of us if you want to know more.

Take 3 cameras and you get full 3D. Then project on a 3DTV, and you have a much more immersive video conference!

While that would be very very cool I don't think it would work with the current hardware. It works by projecting a bunch of dots on the scene in IR (so you can't see them) then tracking them with a camera. So multiple Kinects are going to confuse each other (probably, I assume so anyway).

Just got my Kinect in the post this morning, very excited about what I can do with it!

I expect to see a lot of homebrew robots using Kinect for vision in the near future!

Reply on news.ycombinator.com