I use mididings[1], which is a set of python bindings for receiving, transforming, and sending MIDI.
Using it xdotool[2], I've done things like used various MIDI controllers to send keystrokes, mouse movements, and mouse clicks to various applications that I use.
You could potentially use this setup to control GIMP, Inscape, Krita, or other drawing applications.
[1] - http://das.nasophon.de/mididings/
[2] - http://www.semicomplete.com/projects/xdotool/