Our solution for a similar situation involving semi-structured data (in postgres) was to double it up: put all the json we send/receive with a vendor into a json field, then anything we actually need to work on gets extracted into regular table/columns. We get all the safety/performance guarantees the database would normally give us, plus historical data for debugging or to extract into a new column if we now need it. The one thing we had to monitor in code reviews was to never use the json field directly for functionality.

That way you're throwing away 50% of the reason you use a relational database in the first place. Has it occurred to you that MongoDB exists?

Also I don't understand why you're afraid of migrations, especially since you're the only developer on your own SMTP server.

> And SQLite's `json_extract` makes adding indexes after-the-fact super-easy.

That's a migration.

> Of course, these additional fields need to be explicitly nullable, and I need to skip processing based on them if they're absent

That's an effect of not migrating - having to process null and absent fields instead of just null fields. After doing more of these, you'll run into the same thing that made people stop using NoSQL databases: with no schema, your code has to parse all previous versions of the data format and they probably aren't even well-documented. While an RDBMS can just set the new column to null in existing rows.

I do the same thing with serde_json in Rust for a desktop app sqlitedb and it works great so +1 on that technique.

In Rust you can also tell serde to ignore unknown fields and use individual view structs to deserialize part of the JSON instead of the whole thing and use string references to make it zero copy.

Recently I’ve been tempted to make an SMTP server that translates emails into a web-hook. Please tell me more horror stories so that I might be convinced not do it.