That's what actually happens everywhere. For instance undo is usually CTRL+Z. On QWERTY that will be CTRL + the key at the left side of the X, and on AZERTY that will be CTRL+the key at the left side of the E. Therefore, that's the behavior people expect and it has advantages. Having to write KEY_Z to actually have KEY_W is also jarring and very counter-intuitive. That also means UIs don't have to translate key bindings, unless they want them to correspond to some word (for instance word processors have CTRL+B for bold in English, but CTRL+G is allowed in French (Bold = Gras).
Now I also understand the benefits of your choice, and I appreciate that nobody is entitled to make you change this in any case. At worse everybody should be thankful for your choice to release this work as open source and spend time on it to the benefit of everyone. I'm also not a user so I don't have any stake in this.