If the delay is long enough, the output does not just feel delayed, but entirely unrelated to the input.
A latency perception test involving a switch can easily be thrown off by a disconnect between the actual point of actuation vs. the end-users perceived point of actuation. For example, the user might feel - especially if exposed to a high system latency - that the switch actuation is after the button has physically bottomed out and squeezed with an increased force as if they were trying to mechanically induce the action, and later be surprised to realize that the actuation point was after less than half the key travel when the virtual latency is removed.
Without knowing the details of the experiment, I think this is a more likely explanation for a perception of negative latency: Not intuitively understanding the input trigger.