I find that this happens when you enter folders that have media files like audio files, video files and so on. One way to fix it is to enter one such folder, then remove all columns (like file name, date modified - those columns) and remove all the columns that are media metadata columns. Things like track length, artist, contributing artist or whatever else, then click in the File explorer menu on the 3 dots icon (**) and select View tab, then click 'Apply to folders'. This will apply the column and view settings that you just applied to all such folders.
Now all folders with media files open immediately. Also if you want no wait for video files folders, right click in the folder and select 'View -> Details or View -> List or some other option where it doesn't create thumbnails and it'll load even quicker.