Debian and Fedora provide serial, OpenMP, and pthreads versions of lilbopenblas. Are you sure OpenBLAS doesn't detect nested OpenMP? I thought it did, though I'd normally use the serial version outside something like R, but if you mix different low-level simple pthreads with high-level OpenMP, you can expect problems. OpenBLAS is fine generally -- competitive with MKL on Intel hardware and infinitely faster on ARM and POWER. For PyTorch, presumably you want libxsmm (which is responsible for MKL's current small matrix performance). On AMD hardware, I don't understand why people avoid AMD's support, which is just a version of BLIS and libflame. (BLIS' OpenMP story seems better than OpenBLAS'.) The linear algebra story on GNU/Linux distributions would be less of a mess without proprietary libraries like MKL. It's fine if you take the Debian approach, in significant experience running heterogeneous HPC systems. Fedora has cocked up policy through not listening to such experience, but you can do the Debian-style thing with the approach of https://loveshack.fedorapeople.org/blas-subversion.html (and see the old R example refuting the MKL story). That's one example of the value of dynamic linking.