I’ve of two minds on this. On one hand I agree, because the more functionality a music player has the more likely some of it will miss the mark, but on the other hand I find file managers as they currently exist are somewhat inadequate and incapable of fully replacing a library management system in a music player.
Without going too far out into the weeds, lack of integration between filemanagers/filesystems and music players is the main problem. File managers aren’t conducive to sorting by audio file metadata - even those that support it force the user to manually enable those columns in list mode and support is spotty across file formats, meaning the user has to fall back on “hacks” like modifying filenames to sort properly when sorting by name.
Additionally, the browser-type design that’s dominant in file managers doesn’t lend to versatile use with other programs.
This is one area where I think BeOS had the right idea. There, audio metadata was accessible by way of the filesystem which made it easy to access by applications and meant that the file manager more robustly supported sorting by that metadata. Additionally, the file manager was similar to that of Classic Mac OS where windows were dead simple and each represented a single folder, which made it easy to use a file manager window as a playlist window.