>
From a computer science perspective it is formally 'meaningless' because the performance improvement was made by changing the requirements, not by a novel algorithm or data structure.I've studied computer science and I don't recall seing any such "formal" definition of meaninglessness.
Nothing in computer science tells you you have to have the exact same behavior in 2 APIs in order to measure some common subset of functionality. You can always restrict your measuremnts to that. Computer science is not that concerned with APIs anyway.
So you can use very formal computer science to compare the performance in big-O of X vs Y implementation of the common supported cases (namely, contiguous arrays).
>This is always possible, but it ignores the original constraints of the problem and hence doesn't shed any additional meaning on how to solve the original problem.
There's no "original" problem that one has to take wholesale. There are use cases, and some care about X, others about Y.
Scientific papers in CS compare implementations of things like filesystems, memory managers, garbage collectors etc ALL THE TIME, despite them not having the same API and one having some more or less features. They just measure what they want to measure (e.g performance of X characteristic) and ignore the others.