But to understand the specification and how it can be used to do a programming, you need to have at least a cursory understanding of turing machines and related theory, which isn't necessary to learn Java or python.

Under your definition, the conceptually simplest language is something like SUBLEQ, (the specification is only a single line!) but in this case, being able to implement the language, and learning the language aren't the same thing. Learning the language generally means, like, useful for given purposes.