Trump's "Big Beautiful Bill" likely created with AI – what does this imply? (opens in new tab)

(jonathanbennion.info)

24 pointsrooftopzen8mo ago24 comments

24 comments

Compares emdashes per page in the bill vs the same from the average bill sent to Congress

Incredibly stupid how this has become a leading AI “tell”.

Different models have different quirks. I use 4o at work and it has an annoying habit of taking things absurdly literally and using emojis and formatting too liberally instead of giving me a useful answer. Claude is much more subdued and helpful because it's conversational when you ask something open-ended, succinct when you ask it something data-driven, and almost always asks a follow-up question.

The sci-fi trope of being able to tell "AIs" apart from one another is absolutely coming true in real-time.

anakaine8mo ago

Might be stupid, but it works, for now. Over indexed tokens is another (common inclusions). Some grammar constructions, too, where over descriptiveness is present - though that's easy to read and probably a bit harder to code for.

rsynnott8mo ago

If it works, it works.

It's a crude example, but pattern analysis to figure out who wrote a thing is an old, old technique; people have been doing it with Shakespeare stuff for centuries, in particular.

IAmGraydon8mo ago

Why is it stupid? As long as people are too lazy to find/replace them, it seems to work.

inverted_flag8mo ago

This administration is too stupid to write coherently without the aid of AI. It’s a way for them to dress up their low IQ ideas.

IAmGraydon8mo ago

It’s probably also an attempt to inflate the bill to the point that it’s difficult to read. It came in at over 900 pages.

rooftopzenOP8mo ago

Exactly the point of the post (900 pages falls in line with “flooding the zone” mentality), but we don’t have data on the intent.

bglazer8mo ago

First, I’m almost certain that this article was also partially written by AI. See for example this paragraph obviously copy pasted from Deep Research

“Overall, a more nuanced view of AI in government is necessary to create realistic expectations and mitigate risks (Toll et al., 2020)”

What a unique and human thought for a personal blogpost. Also who the fuck is Toll et al, there’s no bibliography.

Second the authors used Gemini to count em dashes. I know parsing PDF’s is not trivial but this is absurd.

rooftopzenOP8mo ago

First, see below for Toll et al 2020 and I used autocorrect for grammar. Sorry you were dismissive before looking it up, is more a reflection of your bias.

https://liu.diva-portal.org/smash/get/diva2:1591409/FULLTEXT...

Second, I noted all caveats with an LLM counting that - I actually presumed I undercounted, but it had been noted that a simple ctrl-f found 3.8 per page rather than 9.8 per page (counting only single emdashes not double). The actual number doesn’t matter so much, since low bound is absurd difference from baseline bills I checked from earlier this year and 2024, where they do not exist outside of the table of contents.

4.x emdashes per page (low bound) is absurd, and the implication of this is the point you (respectfully) missed.

umbra078mo ago

And how do you know it wasn't just edited by someone who loves em-dashes?

comparing it to the average doesn't matter too much. Better evidence would be proving that there has never been a bill with anywhere close to the number of em-dashes used in this bill.

1 more reply

bglazer8mo ago

CamperBob28mo ago

Add it to the ever-growing pile of "Things that Would Get an SF Author Laughed out of the Industry."

rooftopzenOP8mo ago

Lol can you elaborate

CamperBob28mo ago

Author: "A crowd of intellectually-challenged legislators is swept into Congress to serve an equally-dimwitted but far more devious POTUS who has hijacked their entire party. Their election campaigns were backed by endless free news coverage by media moguls and hundreds of millions in cash from an erratic Hugo Drax-like billionaire, all of whom demand a quick return from their investments. They immediately start passing laws they haven't read and don't actually understand, handed down from AI models that are still under development and highly error-prone. Hilarity ensues when..."

Editor: "Don't call us, we'll call you^H^H^H security."

1 more reply

elijahwright_8mo ago

this article doesn't make any sense. the bill has a lot of em dashes because that's how bills are expressed and it's a large bill. bills in Congress aren't written with em dashes because it can be confusing with the bill syntax and there's not a reason to do it that way

anakaine8mo ago

The author compares it to the average bill going through congress, where you expect 0.1 emdash per page, where this bill has 10. So 100x the historic average.

elijahwright_8mo ago

well, for one, it's more more than 0.1 em dashes per page. the SHARE IT Act has 10 on each page[0]. I don't know how many the 2017 tax cut bill had but it's more than 1,000 and that was over 185 pages[1], and obviously that was before LLMs like ChatGPT. so I don't really know why this is the measure of AI or not, especially because bills have always had a lot of em dashes to start. if you're not analyzing the text of the bill then it's just not going to be accurate

[0] https://www.congress.gov/118/plaws/publ187/PLAW-118publ187.p...

[1] https://www.congress.gov/115/statute/STATUTE-131/STATUTE-131...

2 more replies

rooftopzenOP8mo ago

Not following exactly, so apologies if I'm misinterpreting, but I'm the author and updated this post (transparently) with nuance I'd recently learned about that explains this (somewhat) - the larger bills contain entire pages with only headings that contain emdashes - removed the headings from analysis so that the emdashes per page are only from the legislative text itself. For the conservatively / minimal difference, we're still looking at a 30% increase from a decent baseline.

j / k navigate · click thread line to collapse

24 comments

rooftopzenOP8mo ago

Compares emdashes per page in the bill vs the same from the average bill sent to Congress

jeffgreco8mo ago

Incredibly stupid how this has become a leading AI “tell”.

alexjplant8mo ago

The sci-fi trope of being able to tell "AIs" apart from one another is absolutely coming true in real-time.

anakaine8mo ago

rsynnott8mo ago

If it works, it works.

It's a crude example, but pattern analysis to figure out who wrote a thing is an old, old technique; people have been doing it with Shakespeare stuff for centuries, in particular.

IAmGraydon8mo ago

Why is it stupid? As long as people are too lazy to find/replace them, it seems to work.

inverted_flag8mo ago

This administration is too stupid to write coherently without the aid of AI. It’s a way for them to dress up their low IQ ideas.

IAmGraydon8mo ago

It’s probably also an attempt to inflate the bill to the point that it’s difficult to read. It came in at over 900 pages.

rooftopzenOP8mo ago

Exactly the point of the post (900 pages falls in line with “flooding the zone” mentality), but we don’t have data on the intent.

bglazer8mo ago

First, I’m almost certain that this article was also partially written by AI. See for example this paragraph obviously copy pasted from Deep Research

“Overall, a more nuanced view of AI in government is necessary to create realistic expectations and mitigate risks (Toll et al., 2020)”

What a unique and human thought for a personal blogpost. Also who the fuck is Toll et al, there’s no bibliography.

Second the authors used Gemini to count em dashes. I know parsing PDF’s is not trivial but this is absurd.

rooftopzenOP8mo ago

First, see below for Toll et al 2020 and I used autocorrect for grammar. Sorry you were dismissive before looking it up, is more a reflection of your bias.

https://liu.diva-portal.org/smash/get/diva2:1591409/FULLTEXT...

4.x emdashes per page (low bound) is absurd, and the implication of this is the point you (respectfully) missed.

umbra078mo ago

And how do you know it wasn't just edited by someone who loves em-dashes?

comparing it to the average doesn't matter too much. Better evidence would be proving that there has never been a bill with anywhere close to the number of em-dashes used in this bill.

1 more reply

bglazer8mo ago

CamperBob28mo ago

Add it to the ever-growing pile of "Things that Would Get an SF Author Laughed out of the Industry."

rooftopzenOP8mo ago

Lol can you elaborate

CamperBob28mo ago

Editor: "Don't call us, we'll call you^H^H^H security."

1 more reply

elijahwright_8mo ago

anakaine8mo ago

The author compares it to the average bill going through congress, where you expect 0.1 emdash per page, where this bill has 10. So 100x the historic average.

elijahwright_8mo ago

[0] https://www.congress.gov/118/plaws/publ187/PLAW-118publ187.p...

[1] https://www.congress.gov/115/statute/STATUTE-131/STATUTE-131...

2 more replies

rooftopzenOP8mo ago

j / k navigate · click thread line to collapse