Why did you opt for PGP? If I understand correctly, your system inherits the lack of PFS.
The whole thing is generally more secure then many of these file transfer things due to the use of the passphrase. The idea of strongly protecting the secret key against local attacks is a PGPism that tends to be forgotten these days.
We wanted to use PGP because it's a well known and heavily tested solution. Also, the openpgp.js library is great and makes it really easy to encrypt/decrypt the messages to the client side without the need to really mess with security stuff. Finally, the users would want to have access to their old messages in the future so using ephemeral keys for each transmission wasn't really possible.