This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal.
ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further.
All signs point to this strengthening the value of curation and authenticated sources.
This is what people do collectively, long before any GPTs were in sight. Lots of strong convictions people hold today and publish all over the place, are re-processed excrements of long-gone mental viruses of past civilizations.
Security cameras have existed for a long time, but storage cheap enough to keep years of footage and algorithms capable of processing thousands of streams in real time create massive privacy problems that didn't exist even with the richest companies paying humans to watch.
When you lower the friction of an action sufficiently, it causes a qualitative change in the emergent behavior of the whole system. It's like how a little damping means the difference between a bridge you can safely drive over versus a galloping Gertie that resonates until it collapses.
When a human has to choose and put some effort into regurgitating a piece of information, there is a natural decay factor in the system where people will sometimes not bother to repeat something if it doesn't seem valuable enough to them. Sure, things like urban legends and old wive's tales exploit bugs in our information prioritization. But, overall, it has the effect of slowly winnowing out nonsense, misinformation, and other low value stuff. Meanwhile, information that continues to be useful continues to be worth the effort of repeating.
Compared to the print and in-person worlds before, things got much worse just with social media where a human was still in the loop but the effort to rebroadcast was nil. This is exactly why we saw a massive rise in misinformation in the past couple of decades.
With ChatGPT and humans completely out of the loop, we will turn our information systems into galloping Gertie and they will resonate with nonsense and lies until the whole system falls apart.
We are witnessing the first cracks now. Look at George Santos, a candidate who absolutely should have never won a single election but managed to because information pipelines about candidates are so polluted with junk and nonsense that voters didn't even realize he was a con man. Not even a sophisticated one, just a huckster able to hide within the sea of information noise.
But also, this is an "AI", not human thought. Why conflate the two as if they are equivalent? We are not at the point where machine learning is smarter or produces better quality content than humans.
Very insightful of you to say that. (Though I must say that it is not a recipe for happiness, AKA ignorance is bliss and all that... )
This is what they said about Wikipedia viz. Britannica… alas, it’s a brave new world out there… nowhere to run to nowhere to hide, see that Wiezenbaum post also on the homepage now, as another commenter quotes[0]:
> Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it”
> a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.”
> This is what they said about Wikipedia viz. Britannica...
And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick free crap over nutritious food.
Edit: Another thought came to mind: stuff like ChatGPT may contribute to killing off Wikipedia: Wikipedia is currently the cheapest and fastest way to find information (that's often crap). However, if something like ChatGPT can get information to people faster (even if it's crappier, just as long as it's minimally acceptable), Wikipedia will become much less popular and could end up just like Britannica.
>"11:59" is an episode of the science fiction television series Star Trek: Voyager. The episode originally aired on February 9, 2000 as the 11th episode of the sixth season.
On the other hand:
$ Are there Voyager episodes titled “10:59”, “11:59”, or “12:59”?
There are no episodes of Voyager titled "10:59", "11:59", or "12:59".
I wonder if we'll get a "dead sea effect" with AI, I've seen some stuff saying they've basically run out of high quality training data and now the training pool will get poisoned by AI generated shit. Basically garbage in, garbage out and these large language models might not be able to improve
Of course, some of those books will definitely be AI generated or garbage quality, and we all know many of those journal articles can be worth less than the paper they're printed on.
Yet even if we cut it down to 100,000 books and half a million scientific papers, that's a lot of training data each year... And that is just considering print media, there are other ways to get more content too.
For example, there is also transcription of video/podcasts/tv-shows/movies, etc. along with descriptions of the scenes for video, which could be used to generate a lot more stuff.
With people speaking to their devices and using text-to-speech more often, that's another source too--wouldn't be surprised if some devices just start recording conversations, and transcribing them.
Seems like a ton of potential data sources to me, although it will certainly get more difficult to cull AI generated stuff to prevent feedback, I'm sure the tooling will evolve to enable easy AI content detection and exclusion.
It will be like low-background steel, steel that has somehow been isolated from atomic fallout form the mid XX century onwards, and must be used for radiation-sensitive equipment: https://en.wikipedia.org/wiki/Low-background_steel
Except somehow worse, because it's just steel, this is culture.
And it's not like you can assume an indiscriminate crawl of the net is all human generated currently, anyway, let alone accurate. There's always cleaning involved.
[0] https://www.poeticous.com/shel-silverstein/hungry-mungry
The way I see it is that ChatGPT isn't the only tool out there that can create spam and junk content. The only difference is that ChatGPT produces something that's of a high enough quality that it's not as easy for a human to easily classify it as spam. And something you can't easily classify as spam arguably isn't spam.
If you assume that those incentivised to create spam today are creating spam anyway and all ChatGPT will do is allow spammers to create better spam then I don't see why the quality of content online would necessarily drop because of ChatGPT - you might actually find that what was once just spam is actually kinda interesting all of a sudden.
But it's not just the quality of AI spam that will increase with ChatGPT... Consider BuzzFeed... Arguably they're just paying people to write trash content today. And this is very common. Most companies have a blog where they pay someone to write mostly junk content just for SEO. I think ChatGPT might actually produce higher quality content that what is currently being written at places like Buzzfeed and on junk blogs. Or at least these workers now have a tool to write something that's higher quality.
I think the only way you're correct is if ChatGPT were to greatly increase the incentive to publish spam, resulting in a much greater amount of spam that counteracts the positive improvement in spam quality. And although I think it probably will increase the number people producing spam content to some extent I doubt it will have a net-negative impact.
Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is. This spam singularity that some are predicting, where the prior generation of spam bots produce the content that trains future generations of spam bots makes no sense given these companies are trying to create AI that doesn't just spit out spam and inaccurate information.
It has to drop. ChatGPT can not source new truths except by rare accident.
I bet a lot of you are choking on that. So, I'd say this: Can you just "source" new truths? If you just sit and type plausible things, will some of them be right? Yes, but not very many. Truth is exponentially exclusive. That's not a metaphor; it's information theory. It's why we measure statements in bits, an exponential measure, and not some linear measure. ChatGPT's ability to spin truth is not exponentially good.
A confabulation engine becoming a major contributor to the "facts" on the internet can not help but drop the average quality of facts on the internet on its own terms.
When it starts consuming its own facts, it will iteratively "fuzz" the "facts" it puts out even more. ChatGPT is no more immune to "garbage in garbage out" than any other process.
"Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is"
Even if authority is perfect, that just slows the process. And personally I see no particularly strong correlation between "authority" and "truth". If you do, expand your vision; there are other "authorities" in the world than the ones you are thinking of.
Something that's not obviously junk but is entirely wrong is even worse than something that is obviously junk. It'll waste more time and probably convince more people of falsehoods.
[...]
> I think the only way you're correct is if ChatGPT were to greatly increase the incentive to publish spam.
Arguably it still is spam, and consider the incentive to hide advertising (or generally to push any agenda), when using a program is orders of magnitude cheaper than paying people to do it, but now is hard enough to recognize, I cannot say any more whether your average hn comment has been written by ChatGPT or not as long as I am not specifically looking out for it.
> All signs point to this strengthening the value of curation and authenticated sources.
This is the solution. Knowledge is a web of trust. The only root authority is you the individual. "Experts" and "authorities" are just heuristics. The widespread error that many are making is this: If there is a single objective reality, then curation can happen globally/objectively, not individually/subjectively.
What we need are more mechanisms for individual curation. A user should be able to inspect and understand the chain of believability, from one of their own highly vetted one hop experts, to a distant influencer, public official, or other source of (mis)information.
I am not sure, if the situation is that dramatic, but just wait, until advertisers finds a way to get their "data" into ChatGPT results (or alike).
Then things will get really ugly.
So yes, this is what we will have to do:
"All signs point to this strengthening the value of curation and authenticated sources. "
It’s just a new chapter in the arms race of spam techniques vs detection. Lots of money presumably made selling both sides.
Prediction: this is going to end up like CocaCola taking public water (wiki and LLM) and bottling it ("selected sourced") at a couple bucks a pop! "But there will be premium brands!"
that sounds like a positive and greatly needed outcome
Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer to a simple question.
The only thing that's being threatened is companies like Google who are responsible for the current state of the web.
ChatGPT as a service that can be used to mislead us to trick us out of our money on the behalf of megacorps, like the entire rest of the Web has become? To "promote" things to us against our interests? Meh. Call me when it's mine and will obey me and will never lie to me or serve someone else's priorities over mine, and I'll be interested.
[0] https://pluralistic.net/2023/01/21/potemkin-ai/#hey-guys
Can it, though? The whole point of the article is that ChatGPT "makes shit up".
$ curl -I https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/
HTTP/2 302
server: nginx
date: Fri, 27 Jan 2023 16:53:41 GMT
content-type: text/html; charset=iso-8859-1
content-length: 282
location: http://google.com
Maybe an outcome will be knowledge will be better structured, ala Wolfram Alpha, and just depending on random text documents to encode knowledge won’t be as much a thing - similar to how we no longer use oral traditions to encode and disseminate knowledge. Who knows. But I doubt AI will fundamentally destroy society or knowledge or the internet or whatever. We will adapt, not just to preserve the way we did things, but to use the new things in a way that controls its side effects. It’ll of course be imperfect and some side effects can’t be controlled for. Some things just will cease to be a thing, like phone booths. While you can never have a bill and teds with a smartphone, it’s actually not the end of things. It’s just different. That’s how it works my friend. Life is impermanent, everything changes at all times, you can never recreate the past, and our suffering stems from our inability to let go of the way things are.
Edit: I hate to bring it up, but maybe this is what the semantic web was waiting for.
There is also strong negative reaction to technology that does incredible damage to society. This is just an excuse to ignore people pointing out issues.
So my first take is that people querying it for research are doing it wrong.
Then again, if there’s a large economic incentive to use it in that way, we are very well may end up with the kind of feedback loop that the author describes.
If it can't verify, it just won't answer/tickmark check the answers (happens 16% of the time... and ... always for maths). This is a feedback loop stopper, in the sense of only relying on your documents as the base, and being able to operate entirely without OpenAI (still though using other GPT models)
It's Fragen.co.uk - we believe that more answers formerly missed by CTRL+F will be found with this technology, than false answers taken as true. And if that's true, you are enbettering knowledge. And if not, you're enshittening it slower than the higher-hallucinating alternatives.
If you can wield an ephemeral and verifiable token which asserts your humanness, hierarchically derived from a certificate privately issued to you by one of hopefully a healthy number of well known authorities, you can participate within circles of the human web without revealing anything else about your identity (name, a/s/l, etc)
But, outside of this enclave you can also interact with AI without revealing whether in fact you too are AI.
In this way the internet can develop a more sensitive immune system where it is difficult for human systems to be perverted by Sybil attacks.
I think community managed lists would also be a great way to filter out people with annoying or problematic communication styles or opinions. If you publicly support Jordan Peterson, I can add you to the list of Jordan Peterson supporters, and as a community we'd never have to read anything you post anywhere, ever.
On my personal experience, ChatGPT is a precious tool to refine knowledge and thinking, but it depends on how it's used.
The same painting brush can be used to paint Monalisa or a meaningless blurred thing. It depends on how it's used.
To use ChatGPT and other AI tools properly, we need to build a solid foundation of knowledge and critical thinking skills at first. Then triangulate the answers given by the AI with other sources, our own previous knowledge along with new evidences through experimentation.
Since the brain urges for cognitive relief, most people will be lazy, while just a few will extract the true potential of ChatGPT.
I don't see the value in GPT models for this kind of use. I see the value in like, translation work, or summaries and other transformations, and that GPT works best when in concert with search engines, document embeddings, or knowledge databases like Wolfram Alpha. It's good at predicting more text which is useful in many different kinds of contexts but not in all contexts. It's pretty clear that it is untrustworthy for factual responses like quotes and population counts but that problem goes away if the actual factual data is included in the prompt!
Isn't that where ChatGPT would actually be worse overall? It's been shown to give semi-accurate data very often while sounding very confident about it.
When people require precise answers and it gives almost correct answer, it seems the general feeling is of amazement. I don't share that feeling. I hope nobody is using it for serious work without a human vetting the output.
Yesterday I asked it "what are the chords to 'A Foggy Day in London Town'": it made up a whole new Gershwin song!
======
"A Foggy Day in London Town" is a song with music by George Gershwin and lyrics by Ira Gershwin. The chords for the song are:
Verse:
C C7 F C C7 F C C7
Foggy, foggy dew, you can't see a thing
F Fm C C7 F C C7 F
The visibility is zero, and my eyes are dim
Chorus:
C C7 F C C7 F C C7
It's a foggy day in London Town
F Fm C C7 F C C7 F
The buildings look one hundred feet tall
C C7 F C C7 F C C7
It's a foggy day in London Town
F Fm C C7 F C C7 F
And the people look like ants at all
Bridge:
Am D7 G C C7 F C C7
Just like the feeling you get when you're down
Am D7 G C C7 F C C7
The sun can't break through the London Town
If the preponderance of human writing/creation/art says the sun revolves around the earth, then AI not only nails that down as infallible, but INFERS from it and comes up with new justification for stuff that it made up.
Mandela Effect is all about what 'should' be the truth. Some of the most interesting parts of reality are where bits of reality don't neatly fit into the narrative.
AI exploiting its own inferences and magnifying the Mandela Effect means it steamrollers inconvenient realities on the grounds that 'most people' wouldn't think so. But we don't even think of AI as 'most people', we imagine it as some kind of super-set of humanity, the ultimately wise and skilled overseer.
Boy, is that a mistake.
That's the main problem from my point of view.
The trick to that is, how do you take away ChatGPT's inclination to build upon the wrong premises or the wrong final message, and instead confine it to the role of verbally adroit assistant -- so it only expands a skeletal design into an well written final product? Now THAT would be a use for ChatGPT I could get behind.
Alas, that wasn't the goal of its designers. By giving it almost total control over the plot line and requiring no fact checks or bibliography, the spewing of unchecked drivel was its only possible mission in life.
Congrats, OpenAI. You've automated spam.
Archive link: https://web.archive.org/web/20230127104933/https://castlebri...
Unfortunately with the current state of machine generated content and SEO spam on the internet, that is not likely to happen as there is no real incentive to do so.
And it's not just being confidently wrong about a single person like this author noted, it's being confidently wrong about anything and everything.
The other day I asked ChatGPT about the reasons behind the connection between Italian filmmakers and Western movies. It gave a few reasons that were semi-plausible if not easily verifiable, and then made the bold claim that Italy and the United States were great allies in World War 2. It said this confidently and with a tone of authority.
And while we may all go "ha ha that's wrong" now, WW2 was nearly 80 years ago and not everyone has a strong history background. There aren't that many people left with direct experience and that number shrinks every day. With growing distrust in actual authorities on matters and rejection of observable fact--based at least in part on massive piles of internet BS--this is pretty concerning to me.
I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook.
ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)
In the near-term, AI will just accelerate the winner-take-all nature of our economy.
Ultimately this is where ChatGPT and similar projects seem most likely to hit a wall. Text generators are statistical machines, but the relationship between statistics and truth is merely probabilistic and not identical. There is no way to train ChatGPT to say things that are true, because the number of true and false statements in the world are far too numerous and too specific to ever reliably delineate. At best, we will be stuck fact-checking ChatGPT using enormously complex knowledge bases and asking it to try again until something doesn't raise a red flag, at which point it may still not be true.
For most business purposes this is unacceptable. We will have machines that can generate untruths in a way that contravene common sense, so long as those untruths don't obviously contradict whatever local database is being consulted for grounding. And the mechanisms required to re-ground the GPT output even to that unhappy state will be so complex in their own right that businesses might as well just implement a similarly complex system to handle the issue directly, without calling out to a complex, licensed GPT. Not sure whether there is a way around this.
Say you ask "What is the height of Everest?"
1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer
2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet
3. do a second pass to rewrite the answer with the LLM using search_snippet in the prompt
Basically, the incorrect phrase candidate_answer is very good at matching the correct answer in search engines. It is like a template tuned to extract the desired facts. A search engine check could also verify when the searched fact has no references.
Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work.
Also: Is the generated response really "very good at matching the correct answer"? I suppose it would work because the search engine's language processing cancels out the useless parts that were generated by the AI (sort of a "human ABI", analogous to the C ABI?) but a more direct query (e.g. "height of everest") would likely be just as effective.
The question you should always start with is why and if you can’t come up with a why then you should start from a more neutral position on the behaviour.
Spam isn’t free to engage in, especially on platforms like Facebook, and spam is rarely an irrational money-losing act, it’s just profit generation on the wrong side of our view of what is ethical. Why would someone spend money to unleash chat-bots on a Facebook group to discuss a piece of furniture?
If you can’t explain why, your theory is probably wrong. I would be absolutely shocked if it’s anything more than just some real people who love furniture and use a localised phrase that you don’t recognise.
That said an interesting advance on real game bots would be ones which use AI to chat and respond to other players actions or messages.
Tangentially related: does anyone else feel like ChatGPT's default voice is just... super bland? I read the "op-ed" the author linked in the article (https://www.thecrimson.com/article/2022/12/9/porios-chatgpt-...), and the writing style lacks the kind of variation in structure (or content, really) that keeps opinion pieces interesting. It feels like a polished middle-school essay.
Legend.
OpenAI's PR likes to preach about ethical AI, but it's a total farce. There's no way that OpenAI isn't fully aware of all the harms their AI is going to cause.
I only see one of two possible ultimate outcomes. AI like OpenGPT is going to flood the internet with false or inaccurate information, that I have no doubt. The first outcome I foresee is we simply lose all trust in the vast majority of information sources. The other outcome is we're going to have to sacrifice our privacy and anonymity online in order to distinguish real information from real people and not the AI bot generated nonsense. For example, Social Media may likely require you to prove your real identity, and will only allow you to use trusted hardware devices (i.e. phone or tablet) to post things to your social media account. It'll become increasingly difficult to share information anonymously online in a way that can be trusted. Any information that can't be tied to a verifiable source (a real person) is immediately going to be deemed untrustworthy.
Did you ever have it though? I'm old enough to remember newspapers, and I always took everything I read in them with a grain of salt and I'm absolutely positive they were written by a human being. (disclaimer: didn't read the article because it's down right now).
You’d be shocked at how good people can be at deluding themselves when their livelihood is on the line. Execs at Facebook still think their product is good for society.
When there is a conflict Mountain goats just knock heads against each other (presumably so that both goats suffer concussions … until the one with the weaker knowledge graph suffers a loss of memory).
No such process in AI.
Unfortunately this implies that free information will be largely unfit for consumption.
htthttps://twitter.com/khorasandiary/status/1618682313528975360
Is it viable to constrain conversation to human voices this way or is the cat out of the bag? Is it desirable? Should we just go back to in-person conversations?
Monoliths / Centralized services are the ones that can't adapt to the coming technology.
https://wiki.p2pfoundation.net/Counter-Anti-Disintermediatio...
Great implementation example: https://ssbc.github.io/scuttlebutt-protocol-guide/#follow-gr...
I think ChatGPT is great but it does have a tendency to make things up. All output needs to be taken with a grain of salt. So what? It is still a useful tool.
> quality increase
We must have seen very different enterprise code.
My worry was it would completly destroy the SEO industry and indeed, SEO itself.
The kind of things I do (building thousands of content sites to feed a master money site) are super niche and require millions to get into the game, all it will take is a handfull of non droped high authority domains from auction and a decent server to churn out crap, scale that on a few hundred domains and it is game over.
Boy, I hope "enshittening" isn't gonna be 2023's Word of the Year. The concept of "stuff gets worse" is really old, and a buzzword for it just feels tiresome.
It's only January, so there's plenty of time for this to fade. And I suppose it can't be worse than "gaslighting", a word that was plenty relevant in 2022 but was hardly specific to last year. But surely we can do better.
Awful at precise recall and facts, but amazing as artists, poets and making things that roughly look right.
For generating ideas, they are absolutely wonderful tools. I spend at-least an hour interacting with a combination of ChatGPT and MidJourney.
Wolfram Alpha is different, but it's a great tool I consult for any mathematics/finance questions.
LOL Thank you, I needed a good laugh!
"Did you write this and if so when?"
That would solve like 80% of issues.
chatGPT by itself is amazing. How people (keyword) will use is going to determine outcomes.
Guessing based off the `wp_footer` class you have in there. Much better solution than 302 to Google.
- - - -
Schimidhuber says that his task is "to create an automatic scientist, and then retire."
Not long ago it was mildly insulting for someone to suggest that your writing sounded like the output of GPT, already (for most of us) it has become mildly complementary. GPT may be hallucinating, but it writes well.
So what if you connect it to empirical feedback? Make it a scientist. I don't think it will be long before these machines think better than us (at least most of us.)
The thing is, they don't have glands. They don't have historical trauma in their flesh. What I'm getting at is that these machines won't have human hangups. They won't be neurotic. They won't be HAL. They will be sane.
The interesting thing here is that life on Earth is actually pretty straightforward. In video game terms Earth is very easy and all the walkthroughs and cheats are known and available. The only reason it seems hard is that people are kinda messed up. Once we have sane intelligent machines to make our decisions for us things should get better rapidly.
> If you argue with a madman, it is extremely probable that you will get the worst for it; for in many ways his mind moves all the quicker for not being delayed by the things that go with good judgement. He is not hampered by a sense of humor or by charity, or by the dumb certainties of experience. He is the more logical for losing certain sane affections. Indeed the common phrase for insanity is in this respect a misleading one. The madman is not the man who has lost his reason. The madman is the man who has lost everything except his reason.
[0] https://www.pagebypagebooks.com/Gilbert_K_Chesterton/Orthodo...
Side note - your main point seems overly optimistic. How would we recognize, value, or design a 'sane' machine when by your argument we dont have access to sanity? Seems way more likely to generate a distillation of our neuroses.
Anonymous content will be generated content, real discussion will shift to gated communities with either paywalls or proofs of identity.
Your AI assistant will just take up so much of your time that you won't be bothered with what other people think. Communication with other humans online right now is largely a form of entertainment. The bar is just so low to be more entertained by future AI assistants than a random human.
I am personally certain that I will not be reading this board at some point in the future because the AI assistant will just have far more profound things to say than what random people are going to post on here.
It is not like the assistant only has to be one personality too. Could be a whole group of assistants having a group chat. Lamda can probably do this right now considering it can talk as a planet or national park.
“It knows everything. It’s smart. It’s smarter than you!”
What a crackpot, talking about stuff he doesn’t understand for clicks. Sadly it works for the lowest common denominator
The average person already consumes a low-signal, high-noise info diet, since the advent of modern propaganda in the 1920s.