Nice one! I don't remember all that much from reading the Mask-RCNN paper last year and have not seen many implementations so it's nice to be presented with this Pytorch implementation.
From what I recall about Faster R-CNN, the Regions Of Interest (ROI) are pre-determined via Selective Search, right? So I presume you would need to do the same thing with Mask-RCNN? I think this is the part I am the most confused with since I have never implemented Selective Search myself. Could you point me to introductory material on it?
Lastly, I can see the author of this work has read my blog post on understanding SSD MultiBox - glad it helped in some way :).