No, you can put them anywhere, it just defaults to putting them in a single folder for ease of convenience. There's even a setting, "Default location for new attachments," which allows you to choose:

- Vault folder - A specified folder in your vault - In the same folder as the current file - In subfolder under the current folder

But again, that's just the default location; you can put it anywhere and reference it wherever it is using relative paths.