You should do a deeper dive into QUIC, it has a number of interesting ideas.
1. Having multiple congestion control algorithms is already standard in both QUIC and TCP. As deploying Homa requires you to control both ends or at least understand both ends, in a comparable QUIC deployment you would know they are in a datacenter and can just link the corresponding implementation. Or you could use the configuration parameter handshake mechanism to switch at runtime for certain flows.
2. QUIC is stream-based, but they support fixed/final-sized streams which are logically equivalent to messages. You just generate a disposable fixed-size stream for each “message”. So, it is trivial for the Tx side to know the “message” size and thus do SRPT prioritization. The thing that would be more annoying to support is the Rx side knowing the stream size as size finalization is detected by the Rx side via the FIN bit when transmitted the terminal segment in a stream. You could transmit the terminal segment first, but that is awkward and Homa is superior in that respect. However, Rx size finalization would only really be important for avoiding over-allocation of Rx buffering in QUIC since they do not rely on MAX_STREAM_DATA for flow control whereas the equivalent concept in Homa, GRANT, is used for flow control and thus precise Tx sizing awareness by the Rx side is more important.
3. As mentioned above, Homa GRANT is very similar to QUIC MAX_STREAM_DATA which are basically just extensions of TCP window size to more than one data flow. The main difference is how they are used.
TCP and Homa both use it for Rx resource availability and bandwidth limiting. Homa uses it additionally for data flow prioritization as it has more than one data flow. QUIC uses it for Rx resource availability and implementations can use it for data flow prioritization. QUIC uses MAX_DATA or just plain congestion control bandwidth estimation for bandwidth limiting.
The fundamental usage is Rx resource availability. If you have no Rx buffers available to take the data, then no amount of data sent is meaningful. That is a hard cap. However, using that as a bandwidth limiting mechanism like TCP and Homa requires continuous tight feedback to avoid the Tx from determining there are no Rx resources available while not over-provisioning allowed bandwidth.
To illustrate, suppose I have a 10 GB/s NIC and generate a 1 GB request per second, but my end-to-end is a 1 GB/s link with 1 ms one-way latency. So, this link only handles 1 MB in-flight without buffering. For the sake of simplicity say we pack only 1 KB per packet, so the link can only have 1,000 packets in flight without buffering. How should we manage the GRANT flow?
If we do a initial GRANT of 10 MB, then the Tx will shovel 10 MB into its 10 GB/s NIC which will spit out into the network in 1 ms. This will clog up at the 1 GB/s bottleneck requiring ~10 MB of buffering at the bottleneck. The Tx will then wait for more GRANT as it just bursted it all. After 1 ms the Rx will get the first byte, what should it GRANT and when? If we wait 1 ms, we have received our first MB, should we GRANT 1 MB? Our steady state then looks like bursts of 10 GB/s with 10 MB of buffering at the bottleneck until the request completes.
If we do a initial GRANT of 1 MB then GRANT 1 MB when we get it all, then we stall.
If we do a initial GRANT of 2 MB, then 1 MB every ms then we get 2 MB of buffering at the bottleneck until the request completes.
What is the optimal flow? The optimal outcome is a continuous 1 GB/s flow with one 1 KB packet entering every 1 us. This results in minimal buffering as a result of the transport protocol. For the GRANT mechanism to do this would require a new 1 KB GRANT to arrive every 1 us.
Absent a separate point-to-point bandwidth control system to pace the DATA flow you are not getting that. But once you have that, the GRANT is no longer the bandwidth limiter, it becomes a Rx resource availability and flow prioritization mechanism, just like QUIC. It no longer becomes meaningfully distinct.