The author of REST, Roy Fielding stated that only the entry URL and media type should be necessary to know in advance. Swagger would be considered out-of-band information, if it's necessary to have this documentation beforehand then it doesn't follow this constraint. Interactions must be driven by hypermedia, an idea which has a very unmarketable acronym.
The alternative that is suggested is to document media types, not APIs. If HTML were not a standard and every web page had to define its own technical specifications, the web wouldn't have taken off, because it wouldn't have been interoperable. Interoperability is key, it reduces friction from transitioning between web pages on different servers. How HTTP APIs are built today is wasteful, there are vendor-specific tooling costs up front and it doesn't scale.