I really wish more systems would adopt the "copyindent" pattern, where indentation is simply copied from the existing line. I've primarily developed in vim for 15 years and have similarly minimal `vimrc`s (no plugins), but some of the first configurations are always:

  set autoindent
  set copyindent
  set preserveindent

This should be compatible with basically any language (other than some esolangs like "whitespace") and any tab/spaces scheme. There's no need to have per-file/project configuration, as long as you're okay with pressing space 4 times instead of tab in the projects that use 4-space indentation.

The ftplugin for the make filetype includes setlocal noexpandtab, which should take precedence over what's in your vimrc.

  % vim Makefile
  :verbose set expandtab?
  noexpandtab
    Last set from /usr/share/vim/vim90/ftplugin/make.vim line 15
  
  % vim file.c
  :verbose set expandtab?
  noexpandtab
    Last set from ~/.vim/vimrc line 56

but I think that may not work as intended for OP as they don't have "filetype on plugin".

It’s been a while but I seem to remember being able to configure different tab behaviour specifically for make files.

It surely can. If I have to casually mess with a Makefile, I'll just remember to use control-v TAB as needed, or vi yank/put another line with a proper tab and edit. In the very rare case I'm doing more serious work, I'll temporarily turn it off.

You can change it per filetype. For example, I have a similarly minimal vimrc that defaults to sw=4, ts=4 and expandtab, but with the following additions:

  autocmd FileType go,make,sh set noexpandtab
  autocmd FileType js,json set sw=2 ts=2

Edit: I think you might also need the following to make it trigger the FileType event when you open a file in the first place:

  filetype plugin indent on