Not a CEO, but I knew one who did this...
The main benefit is that you can get many more contracts with the government. SBIR contracts are great for getting a startup going, and some of them will require a clearance.
The main cost is probably that the style of financial accounting required for government contractors is nearly incompatible with non-government business. If you want to do both, you almost need to have two divisions as separate companies owned by a holding company. Having separate divisions may also help you avoid making criminal mistakes with any overseas opportunities.
An interesting difference is your employees. This is both a benefit and a cost. Normal companies are prohibited from refusing to hire because of greencard status, but you would be required to do so. You will not face much cultural discord in your workforce. You can't get an H1B or do foreign outsourcing. You don't have to deal with background checks because the government handles that for you, and they do a better job than you could ever hope to do -- even if you grow to be a large employer, you'll see very very little misbehavior from your employees. Except as required by the federal security rules, you can do everything on the honor system.
Get the contract first, probably a SBIR. With that in hand, ask to get clearance stuff sponsored. (you must have a government sponser; they "hold" your clearance) At minimum you need people cleared, letting you visit the government site to talk about classified things. You might even choose to work at the government site; they can provide a space for you to work. If you want to do classified things at your own facility, you'll need to get your facility sponsored too.
