As someone who has been recently optimising fpnge, Adler32 computation is pretty much negligible regarding overall runtime.
The Huffman coding and filter search take up most of the time. (IIRC fpng doesn't do any filter search, but Huffman encoding isn't vectorized, so I'd expect that to dominate fpng's runtime)