Do you plan to release this? If not, can you discuss your approach?
In terms of the approach, the whole page is rendered in a headless browser, we extract the whole page screenshot, text and other features, and feed them into one neural network where all modalities are joined and which handles the extraction.
Speed of the AutoExtract ML part is not a concern (many pages per second on GPU) - the bottleneck is in the browser rendering.