Do you just mean the ballots are filled out at home where a government authority is not looking over my shoulder? Because everything else is controlled by the government. The ballots and booklets are printed by the government (who authorize what can be on the ballot and in the booklet), are mailed by a government agency, are checked by a government authority, etc.
Imagine constructing a system that can thwart a abusive, tyrannical father who insists that his wife and children vote for a particular candidate (to make it concrete.) If you can get past him, your voting system passes the first test. Now imagine someone is offering $50 if you vote in a particular way. If there's no way to figure out how someone would claim it, it passes the second test.
The abusive father can literally just fill out all of his family's ballots, and the $50 could be claimed by filling out the ballot in front of the buyer. You could thwart this with allowing multiple votes but only accepting the first, but then the father or buyer could just have the ballots filled out immediately at the first legal moment.
I don't know that it's a thing that can be done without totally private environment around the voter and the record; meaning that the actions of the voter cannot be observed.
I don't think this is currently happening, so I don't think it is a major issue.
