SQLite, perhaps the most widely deployed software system, takes this approach.
> The Lemon LALR(1) Parser Generator
> The SQL language parser for SQLite is generated using a code-generator program called "Lemon".
> ...
> Lemon was originally written by D. Richard Hipp (also the creator of SQLite) while he was in graduate school at Duke University between 1987 and 1992.
Here are the grammars, if you're curious.
But I do think the wider point is still true, that there can be real benefit to implementing 2 proper layered abstractions rather than implementing 1 broader abstraction where the complexity can span across more of the problem domain.
SQLite isn't some kind of universal template, I'm not saying people should copy it or that recursive descent is a bag choice. But empirically parser generators are used in real production systems. SQLite is unusual in that they also wrote the parser generator, but otherwise is in good company. Postgres uses Bison, for example.
Additionally, I think that Lemon was started as a personal learning project in grad school (as academic a project as it gets) and evolved into a component of what is probably the most widely deployed software system of all time shows this distinction between what is academic and what is practical isn't all that meaningful to begin with. What's academic becomes practical when the circumstances are right. Better to evaluate a technique in the context of your problem than to prematurely bin things into artificial categories.
Sure, but adding the complexity of a parser generator doesn't help with that complexity in most cases.
[General purpose] programming languages are a quintessential example. Yes, a compiler or an interpreter is a very complex program. But unless your programming language needs to be parsed in multiple languages, you definitely do not need to generate the parser in many languages like SQLite does. That just adds complexity for no reason.
You can't just say "it's complex, therefore it needs a parser generator" if adding the parser generator doesn't address the complexity in any way.
Creating abstractions does decrease complexity if one (or more) of the following is true:
- The abstraction generates savings in excess of it's own complexity
- The abstraction is shared by enough projects to amortize the cost of writing/maintaining it to a tolerable level
- There are additional benefits like validating your grammars are unambiguous or generating flow charts of your syntax in your documentation, amortizing the cost across different features of the same project
It's up to you as the implementer to weigh the benefits and costs. If you choose to use recursive descent, more power to you. (For what it's worth, I personally use parser combinators to split the difference between writing grammars and hand-rolling parsers. But I've used parser generators before and found them helpful.)
The only time I have used this myself was an expat style transformer for terraform (HCL). We had a lot of terraform and they kept changing the language, so I would build a fixer to make code written for say 0.10 to work with 0.12 and then again for 0.14. It was very fun and let us keep updating to newer terraform versions. Pretty simple language except for distinguishing quoted blocks from non-quoted.
I hear stories like this and I just wonder how we got here. Like, did this work provide any monetary value to anyone? It sounds like your team just got way too lost in the abstractions and forgot that they were supposed to make a product that did something, ostensibly something that makes money.
I mean, I guess if you can persuade people to give you money to do something, it's profitable. :shrug:
This work made it easier to keep terraform in sync with aws and so maintenance (e.g. adding a new policy to existing S3 buckets was just edit the S3 bucket module and re-applying to every account. The parser was way easier than manually editing the files manually (especially since I had so much terraform as a test bed of data).
