We are trying to build a data access API that can provide a functional and flexible view into data where we know some of the requirements and expectations of the initial users, but we don't know how it will be extended and transformed in the future.
If we don't provide things such as customization of the returned fields, desired date ranges, and ordering, then I believe we would have to rely on waiting for feedback from users/devs that we did not satisfy and attempt to build new end-points to correct that deficiency. That seems to be almost guaranteed to get us off to the wrong start for a large portion of users and they might just decide it isn't worth asking us to make changes since we didn't appear to be catering to them in the first place. I understand that we can't meet all the unknown needs, but if we can build features into the API to meet general classes of needs, that seems better than ignoring them until the specifics become known and then supplying further endpoints that answer only those specifics.
How would you extend this philosophy to a large data portal such as data.gov?
Even in the FBI example, if there is no extensibility or transform-ability in the API, how are the unmet needs of users handled? Do you have a system that gathers complaints from users about the data they suspect is possible but not available and then build new endpoints for each distinct need? Do you just ignore the potential transformative uses that aren't directly applicable to the needs of the customers to which you are paying attention?