Yes, apps can change terminal mode at any time. The solution the author went with is to have a list of "visual program" names, and when a visual program name is detected, launch it in an external terminal (xterm or Emacs vterm). That is making no attempt to handle mode switches during program execution, it is making a permanent decision before the program starts based on its name.
I think the solution here is to have a visual terminal "widget" that can be embedded into the graphical REPL, and the REPL constantly monitors the output and terminal mode of the subprocess, and when the REPL detects initiation of visual mode, start an embedded visual terminal widget, and then stop the visual terminal widget when it detects visual mode is completed.
On Linux, it is possible for the pty master to be immediately notified of any mode changes in the slave [0], using packet mode and EXTPROC. I don't think that facility exists in most other operating systems.
[0] https://stackoverflow.com/questions/21641754/when-pty-pseudo...