If neither a hardware component nor kernel key management is involved, crypto should be done in userspace, end of sentence.
The more I think about it, the more I think it should be behind CAP_SYS_ADMIN, or a new CAP_KCRYPT (better name TBD. CAP_CRYPT_OFFLOAD?)