Supervisor allows you to have event listeners (e.g. for processes quitting/crashing), so you can use those to achieve that and kill supervisor itself. Here's an example of people doing just that:
https://gist.github.com/tomazzaman/63265dfab3a9a61781993212f...