Sure, a framework that insists on building BoringSSL (on the first build, but not on incremental builds) has no impact on the long initial compile time.
Users shouldn't be met with compile times that long on a "Hello, world" program.
For comparison, a "Hello, world" server in Rust with Axum and Rustls takes 25s and 16s in release and debug mode respectively.