Just a leftover test function inside the main app referencing this library would probably be enough to consider it a derivative or combined work, no?
(Not sure how HTTP comes into play regarding the linking and combining of a project and this AGPL library?)
I think the terms are difficult to understand. In the general case who knows what a library author intended or how think they understand the license??
(Also, claiming GPL only pertains to binary software sounds really quite imprecise too. Just because you receive source code does not mean you can use it or distribute it in any way you see fit. You would still need to abide by whatever license accompanies the non-binary software)