> But there is no evidence of such minds, right?
Well, I am a mind, and so I have direct evidence of the mind that I am.
I don't have direct evidence of the existence of other minds. But I do have indirect evidence of their existence. And I believe I am justified in believing in their existence.
> We have no ideas about where or what they are
Why must a mind have a where? Must entities have a spatial location in order to exist? (Anyway, my sense experiences are from a particular varying spatial vantage point, so maybe that vantage point is the present location of my mind.)
As to what a mind is – according to idealism, minds are fundamental/irreducible/basic concepts, which cannot be explained in terms of anything else. Hence, to ask for an idea about "what a mind is", if that question presumes minds can be explained in terms of something more basic, is a mistaken question. But, the very same point applies to materialism, with respect to whatever it proposes to be the most fundamental concepts of reality–particles or waves or forces or fields or strings or branes or whatnot – whatever physicists ultimately settle upon as most fundamental, materialists will accept as most fundamental.
> or actually anything anything at all about them?
Well, it seems we can say lots of things about minds. They have a succession of thoughts, feelings, beliefs, desires, memories, habits, character, sense experiences, consciousness, unconsciousness, dreams, hallucinations, etc. We can describe their contents in great detail.
> which I assume is more like the 'mind' of a monistic panpsychism
When I say "mind", I am not intending to use that word in some special sense fundamentally different from the everyday one. I am simply proposing that mind, in (more or less) the everyday sense of that term, is a basic/fundamental/foundational aspect of reality rather than a non-basic/non-fundamental/non-foundational one.
"Monism" is ambiguous between type monism and token monism. Idealism and materialism are of course both forms of type monism, compared to substance dualism which is a type dualism. But not all idealists are token monists. Some idealists do of course say that only one mind really exists, and the existence of multiple minds is some sort of illusion. But, it is possible to be an idealist and yet insist on the existence of a real plurality of minds. I favour that later view.
Idealism needn't involve panpsychism either. It is open to an idealist to say that humans have minds, and even some higher animals have minds, but at the same time deny that rocks and trees and planets and stars and bacteria and atoms do. I observe correlations between my inner experiences and my outward behaviour – when I feel sad I will cry – and so observing similar outward behaviour in other humans, and even in some animals, I believe I am justified in concluding that those same inner feelings exist for them, even though I can't directly observe them. But I observe no such outward behaviour in bacteria or plants or rocks or planets or stars or atoms, so I don't have the same justification to conclude that they have minds.
> but for the purpose of understanding how our universe works, you want to engage the materialist perspective
I don't agree. Materialism is not part of the natural sciences, it is a metaphysical interpretation of the natural sciences. One can adopt a different metaphysical interpretation of the natural sciences – such as an idealist one – and then carry out the practice of the natural sciences just as well as the person who adopts the materialist metaphysical interpretation can.
> 'knowledge' is something that I believe to be totally mundane and amenable to scientific description (same with 'justice' and the best way of governing), whereas philosophers have a tendency to elevate it
To go back to my earlier point – if ethics/morality is "totally mundane and amenable to scientific description", then it cannot truly be objective in a transcultural and transpersonal way. I believe I have a moral duty to promote the idea that certain acts – for example, state violence against LGBT people – are gravely and objectively wrong, in a way that transcends cultural differences and personal opinions. Adopting your view undermines my ability to fulfil that duty. (In principle, if there was compelling rational evidence that your view was true, I would be forced to concede that evidence as overriding that duty – but I don't believe any such compelling evidence actually exists.) Hence, I cannot adopt your view.