We trained it on a large dataset of gifs created by our users. The first thing the AI does is break the video into scenes and then scores them based on how similar they are to the gifs int the training set. We then take the top scored scenes and put them in the highlight reel.

The machine learning neural network is loosely based on https://people.ee.ethz.ch/~gyglim/video2gif_cvp16.pdf