The official rules say that one of the requirements is that the recipients "Forgo other employment or educational enrollment during the two year Fellowship except as approved by the Thiel Foundation."
Thus, they can go back to school after two years if they really want to.
