He has to fill the court with federal judges that have been appointed by a President and approved by the Senate. That's a somewhat higher standard than whoever.

That real estate court was headed by someone more like an administrative law judge. Some states allow that sort of thing. In New York, towns and villages have elected justices who go to judge school for 60 hours. They preside over minor cases like traffic and "police court" type matters.

One of the more serious issues with the FISA court is that they appear to be only nominally distinct from the Executive. The staff seems to come from the DoJ, for example. I think this reflects the fact that this was until recently a sleepy backwater of the court system.

I wouldn't focus too much on the Chief Justice's ability to appoint the judges. Federal judges are very powerful people, and that power (and the permanence of their position) gives them more independence in terms of judgement than just about anyone.