The Founders envisioned an extremely weak criminal justice system, especially for "their class of people." Defendants were given extremely strong protections, and convictions were the exception, not the rule.
The Founders were more concerned about facing a duel than a criminal conviction.
So they added other mechanisms for presidential accountability: impeachment, elections, and the weakness of the office.
These other mechanisms have become weaker and weaker, while the criminal justice system has become stronger and stronger.
Impeachment's happen, but not Senate convictions. The political parties have created a duopoloy on power which allow them to run weak candidates. Congress is less and less willing to hold presidents of their own party accountable. Dueling is prohibited not just criminally, but constitionally in most states.
At the same time the criminal justice system is becoming more and more powerful. Convictions are in the high 90%. Juries are very weak and at the mercy of powerful prosecutors.
The Constitution simply didn't envision a situation where the criminal justice system is more likley to hold someone accountable than an election or Congress.
Impeachment, elections, and duels no longer deter bad conduct. Convictions do.
So we have an edge case: a system that can only hold an ex-President accountable via a criminal charge.
Edge cases are weird. They create "sometimes it works and sometimes it doesn't" situations. And that's where we are now.