I'm going to assume you don't have a lot of familiarity with conducting. It often takes an entire season to rehearse with a group in order to produce music that could accurately be 'judged'. And a conductor is about far more than the music -- how is their rapport with the orchestra? What are they like to work with? What kind of an artistic vision do they have for the group, and how do they communicate that? Ultimately, what kind of a leader are they going to be, along 20-odd different dimensions?

Even with world-class orchestras, where performances are regularly put on with guest conductors after only a few hours of rehearsal time, no permanent conductor would ever be hired on the basis of merely listening to their music. It's a leadership position. (Unlike orchestra players, where it really is more directly about musical proficiency.)