Building a drawing tool in the browser is complex, when one has to begin to calculate various stuff(do bounding boxes intersect, perform a affine transformation).

Could you share some insights?

Do you use an external math library like math.js(which seems to be very hewavyweight)? Do you have a model with objects and other objects/functions that draw to the canvas, or draw the objects themself on the canvas? When the user moves a object, do you update the model immediately, or only when the move has ended?