As I understand it, the complaint about the "administrative state" is that administrative agencies have the ability to regulate and assess fines or other judgments that should properly be part of Congress and the court system.

For example, the Federal Reserve can assess fines without going to court, like: "Federal Reserve Fines Deutsche Bank $41 Million for Deficient Anti-Money Laundering Program" - https://www.wsj.com/articles/federal-reserve-fines-deutsche-...

Did the First Bank of the US have regulatory abilities and the ability to access civil fines?

https://en.wikipedia.org/wiki/First_Bank_of_the_United_State... says it "did not set monetary policy, regulate private banks, hold their excess reserves, or act as a lender of last resort", which would suggest the answer is "no".

If correct, that would mean it's not part of the "administrative state".