Just configure your CI to re-run all PR jobs when master updates. E.g. Bitbucket is updating the refs/pull-requests/pr-id/merge git ref automatically and I'm sure many other tools do, too. We are doing exactly this at my workplace and I am surprised that it seems to be unknown to so many people on HN.