As long as you have low standards, LLMs will not disappoint. They are toys, and this is a toy example. This article is a good example of a certain pattern:
"I tried to do X with AI. Look! It seems like I succeeded!"
If you don't understand or have deep experience with software, then you have no professional basis on which to trust it. Neither I, nor has the author of this post, nor anyone else has subjected this way of using Claude to systematic testing. Is it reliable? Does the output fulfill the brief? Nobody knows. At the end of the piece, the author invites us to test it. The author expresses his satisfaction.
At one point the author says that LLMs are great at synthesizing data. This is something I have tested. The truth is: NO. They are not great at synthesizing data. They are systematically biased and quite error prone, especially for larger data sets. Only if you have no strong need for a particular kind or form of data, or if you have no list of attributes that must be true of the data, are you going to be consistently satisfied with LLM data generation.
