That would be a modern trojan horse, smart to refuse it. Even if it may have look tempting back then (wars have been fought for meaningless land grabs before), as we see now with Ukraine it would be just inviting worst kind of problems down the line
Forced migration was a part of the Soviet playbook for generations by then, so Poland and Lithuania (and I would guess most countries in the Soviet Union) were all too familiar with the tactic.