mpt-7b and the falcon models have longer contexts available. They aren’t trained on longer contexts so, the results might not be very good.
Isn't this true for any model. I always thought the context window exists on the training side and on the query side but not in the model itself and with query side bigger than training side not useful.