Are you using both a master password and the key file?
> Any suggestions to overcome this reduction?
Not and continue to use KeePassXC, as this is inherent in KeePassXC.
Ultimately, your method here relies on:
1) the security of the KeePassXC dbx file's encryption
2) whether there are any known exploits of KeePassXC files
3) whether there are any unknown (except to the attacker) exploits of KeePassXC files
4) the complexity of your 'master password' (if you use one)
5) the complexity of whatever password is used to unlock this 'token' (which is just a KeePassXC "key file").
If, by chance, you used "Password1!" as both your master password and token password, then an attacker is all but 100% certain to open your vault.
If your "password" is available in any of the breech lists, or can be deduced by a hashcat "try variations" run, then an attacker with the hardware, time, and desire would be able to open your vault.
If there is a known, or unknown (except to the attacker), CVE for KeePassXC that results in opening the vault, then you are at risk of having your vault opened.
Ultimately your resistance to attack here comes down to how resistant the KeePassXC vault is to attack.