Pandas is one of those for which poetry alone doesn't do the job. So I have nix install pandas alongside poetry so that when poetry tries to install pandas, the non-python dependencies are all lined up.
Here's a (too verbose, sorry) example: https://stackoverflow.com/a/74944857/1054322
It works well on all of my linux boxes. On my mac it takes forever because it wants to rebuild the linux universe. Given that the system architecture is a parameter, I think there should be a way to rewrite my flake (or update the nixpkgs numpy) to just rely on bsd tools in those cases, but I haven't found it yet.
That's what's holding me up from recommending it to my team. I'm happy to just use linux everywhere, but they aren't.