I'd like to actually do this, but how do you tell your CI server what it needs to merge where? In our case we have Jenkins and BitBucket. It's not in my power to change that, and these tools aren't integrated that well (to my knowledge) that you could say which PRs are open and which branch merges into which.

Maybe if you have a workflow where it's always the same target branch (master) it works, but we often have long standing feature branches that get multiple smaller merges from other branches.