I've been interviewing recently and my opinion is more or less the reverse. I preferred the interviews where I was asked algorithms - only got asked on graphs once, but my last job did involve a lot of those so I was pretty comfortable with them. They seemed more useful to me than asking "trivia" type questions which you could learn in fifteen seconds - for example, I got one elsewhere about access to members of nested classes in C++, which I just couldn't remember at the time. The thing is, you can solve that immediately if given a compiler.
Personally I'd rather have a coder who understands the concepts behind these things, even if they don't necessarily know the answer right then, than one who is simply able to rattle off definitions by heart. In the example you gave, I agree that it's valuable to know that an Integer is significantly bigger than an int, and that a Double is bigger, and that your TreeMap<Double, Double> might be bigger than you expect, even if they didn't know the exact sizes of any of those things - but I guess I'd want them to be able to take a stab at working them out.