DOM manipulation in a controller is something you can't do and who says controllers, says services too.
On the other hand, the double ng-repeat could be a hot spot for the bad performance.
Don't get me wrong, the app is awesome, awesome idea but the implementation doesn't respect the angular way so it is really slow.