"The more it eats its own output, the less variety the output has"... sounds exactly like the UKposts recommendations algorithm.

This reminds me of when I was playing with play-doh when I was young. You start out with many different colors, and somehow always end up with a big brown ball.

I remember as a kid I used to record my voice and play it back on my speakers, which I then proceeded to record once more. By repeating it, I could hear how it slowly degraded until it was nothing more than a weird, synth-like sound.

My biggest problem with AI is that it needs to get the information from somewhere, and sometimes these sources can be slightly dodgy. I did an experiment where I asked ChatGPT about some very narrow subjects: The Danish organplayer Peter Erling, the trio Klyderne, and the artist Jørgen Fonemy. These are subjects that I have some knowledge about and actually have written Wikipedia articles about. I could see that most of the answer that I got from ChatGPT was based on the exact Wikipedia articles that I wrote! I have tried to write the truth in those articles, but if I didn't care if things were correct - or worse; if I deliberately wanted to mislead people, then AI would base the answers on wrong data, if there wasn't multiple sources available. The problem I see with AI is that we trust it too much. Already now there are people who believe that it is an omniscient trustworthy source of all answers and that it will always be more correct than human knowledge or just knowledge that we have googled or looked up in an oldfashioned book.

3rd possibility: AI learns how to gaslight us, and we forget how many legs elephants have.

Apparently in the original Matrix movies storyline, the reason why the machines needed to keep those troublesome humans around was not as an energy source (“batteries”) but as a source of creativity. But the writers thought that this idea was too complex so they substituted the battery idea instead.

That plastic analogy is probably the most succinct depiction of AI generated content contaminating the environment and why I always thought that human intervention in the use of computers is always necessary. We can fake human thinking to a degree, but getting the full complexity is still a pipe dream.

I love that this underscores how complex human intelligence really is.

This could lead to a .. Nightmare on LLM Street....

As someone who used to play with photocopiers as a kid... A copy of a copy of a copy is always much worse and weirder than you might think. Small flaws amplify until you all you get is a smudged blur.

I recognized this as a possible problem when I learned that they were training AI by allowing them to converse with people on Reddit. AI developers can now apparently pay a fee to be allowed to plug their AI into Reddit and have it learn by having conversations there. It occurred to me that, "wait a minute, wont then the AI's end up conversing with each other and training each other??? Won't this cause problems?"

to me the scariest thing is how quickly people advocated against themselves the moment they realized the potential with AI

A friend in the UK is a graphic designer; he says that over the past few months, more and more clients have been saying 'NO!' to AI-generated artwork - "it's too samey". They'd rather pay more for something original. Trouble is, AI has pushed down the rates; so while designers and artists are noting an uptick in requests for proposals, the money is much worse.

My experience with chat GPT is that you can ask it 2 or 3 questions, get it to contradict itself, and when you point out the contradiction, it starts to ask if you are angry, and/or says IT'S taking a break from YOU to let you relax...😊😮😂

Glad you started this conversation. There is also the theft component of generative AI. A UKpostsr like yourself will get automatically copyright struck for using 4 seconds of a clip in a 20 minute original video. Yet these generative AI companies can use entire social media platforms with content painstakingly created by individuals across decades to create their data sets. This is peak hypocrisy in which, as per usual, corporate "big money" is protected while the individual is left with no means of defending their content. Generative AI is 100% theft in my opinion.

A friend of mine who has been a musician since the late 1950's explained tp me a few years back how there is very little originality in music these days as much is dependent on preset rhythms , chords , etc much due to "electronic " devices . It can only get worse with so called AI

1) AI is trained on data from the Internet. 2) AI outputs data to the Internet. 3) Goto 1 ... haven't anybody acquainted themselves with the topic of "inbreeding" ?

This is exactly what I was telling people the other day. Our greatest danger with AI isn't that it'll take over but that at the moment we begin relying on it most, the more it will collapse because it's going to end up cannibalizing itself.

Sabine, as a professional in the application of machine learning in medicine I would like to thank you for making this video! It’s understandable and it reaches a lot of people! There is the AI hype (which people should not believe because it comes from executives and rookies) and there is the machine learning reality that veterans understand. This technology will be useful in automating some drudgery and common simple tasks…. It’s dogshit at doing anything truly valuable. What’s most worrying is the very real threat, without laws, that this nonsense will create such a firehouse of bullshit that we can’t get through our email, find what we need on the web, tell the difference between fact and fiction, and generally think for ourselves!

That’s really interesting.. because comparing it to ‘bad cinema’, most bad cinema is bad in the same way, if that makes sense. Overused tropes, predictable storylines, cliche characterisation. Is there someone that can expand on this thought?

It's true, when you've worked enough with ChatGPT you can immediately recognize a ChatGPT text. It just always has a certain vibe that makes it distinguishable from human text.

You reminded me of Pandora, the music recommendator that provided you with music according to the 👍 and 👎 that you gave to the songs proposed. No matter if you started with Black Sabbath, Chopin or Yunchen Lahmo. Eventually, after a couple dozen songs, you always ended up in a Coldplay loop.

Short, to the point, informative. Thank you, Sabine.

It's kinda like Google search engine. It started out sucking, then there was a time it was pretty good to find stuff. Now it sucks again....

Adding random variation probably isn't as easy as it may sound because the randomness still has to follow certain rules. For example, no one is going to believe that elephant with two trunks.

Garbage In/Garbage Out. I've been saying the same thing about both AI and Analytics for the past decade and a half. People only want to look at processes, algorithms, ease of use, speediness, raw power, TCO, design and pretty UI with both AI and Analytics. You rarely hear people talk about things like bias, data integrity and context. Those three things only come into conversation when AI and Analytics produce horribly incorrect results.

Thank you for clearly articulating what many have been trying to point out.

This is already true for me when writing. In the first few rounds it appears as if the AI is very creative but soon after some repetitions it becomes clear that the AI keeps repeating itself over and over again. This is one of the reasons I am not concerned too much. It appears that the human still has to input A LOT of guidance to make sure it doesn't repeat itself and actually gives you more interesting "mixes" instead of repetition.

You make a good point. Already I can usually pick the ‘style’ of AI generated images. They have a certain ‘style’ because they are in a sense too perfect, too smooth, too balanced. It is not something one could define in some cases, but the human brain is good at recognising patterns.

There's a third option: there may be soon a deliberate attempt to poison the content to make it unreadable for AI. There are already tools out there that scramble images just enough to make them confusing for AI to use as a training set.

Unfortunately it will only be a matter of time and they will either get better with the parameters that introduce random variables that simulate creativity or they will achieve singularity which they are moving extremely fast towards. I believe we just have to keep creating original works of art. Ai can never replace that. Draw and paint more originals and let Ai exist for the people that don’t want the joy and fulfillment that true creativity brings them.

After the autotune-as-an-effect started, with Cher singing "Believe", you have multiple generations of singers who mimic the sound of autotune artifacts quite naturally and spontaneously. Also, young people having plastic surgery to mimic the look of phone camera filters. We are also trainable, it turns out. I think that is an important element to consider when predicting the future of this stuff.

So basically, if you keep feeding the output back into the input, you could get a feedback loop.

As an artist, this is a known issue in HUMANS Thats why the art solution is to look at the "old masters" as mentors before learning how to draw from more modern artists

Makes me think of the 3 body problem issue/chaos theory. The more variables, the faster chaos erupts.

I think Frank Herbert might have already told us the eventual solution in Dune.

My experiance with ChatGPT shows it to be a regurgitator, the test questions were in an area of X-ray physics that I know well and it spewed out all the usual stuff with no insight, no deep understanding, no creativity, nothing that would indicate any form of curiosity.

The one thing people should know about machine learning is: a machine learning trained model will only be good as its training data. It's just learning (in theory) the pattern behind the data leading to a host of problems. The main issue is that it doesn't actually reason about the data. Let's say I train a model where I have several examples where I have pi as 3.14 and then one where it's 4. The model doesn't reason "you know.... this one example seems to be wrong" but rather it updates the model to make it slightly more likely it will give the wrong answer. So how do you prevent models training on information generated by another machine learning model? The current approach is to stick to information before generative AI become dominate but most of that information (for better or worse) is probably considered or part of the training dataset. The main problem is that there's a popular opinion in machine learning (and sadly AI) that, as an AI researcher, I have had to deal with. This opinion is the key to all AI problems is that we just need to use larger models, with more training data, and train it in the "correct" way. "Look how far LLMs have come. Just imagine how much better they will be in a couple years". But you run into the 90-10 principle: 10% of the effort for 90% of the results and vice versa. It's why self-driving cars are taking a long time: there is nearly an infinite extremely rare cases that the car needs to make the right decision in. As such, it should be expected for the current LLMs to plateau performance wise unless new smarter methods are found.

This actually is not that suprising, when you think about it: these AIs are basically using huge amounts of data to approximate averages of various things, and with more iterations they extract more and more core features until they just have the same set of features they are using all the time. It's like taking data scores and continually averaging them until you are left with one value.

Worst case scenario, it'll be like Walmart moving in to a community, destroying the mainstreet businesses, everyone takes Wal-jobs, and then the Walmart closes because the local economy is crap simply leaving a ghost town with people who don't have the capital, business acumen or consumer base to reboot mainstreet.

Every email in the future will start with: "i hope this email finds you well" 😂

Wow... this is like ultra-high-speed "Groupthink"

I was using an AI image generator to come up with rug designs. The first few were great but it soon deteriorated to regurgitated images similar to what it already produced.

In my industry gives vague answers when you ask about laws. It keeps saying that things vary. They don’t vary, there’s just more than two answers. That’s not the same as varying. Some customers go under option A and others under option B. You can’t just tell them it varies

As an AI scientist, we've been talking about this for years. Once the AI starts eating its own tail it will quickly optimize to a singularity of stupidity in its own echo chamber. The only way for AI to continue to work is to automatically label all AI output and ignore it for training. Or to manually post label it by humans. Humans are necessary for AI success in any case. It would be interesting for you to discuss both the labeling servant culture and its injustices as well as the impossibility of AGI if AI depends on human labeling.

2) More randomnes in AI output might do away with the problem of repetitive AI output, but it might increase the mistakes. Instead of elephants with big heads or two heads, we might get elephants with two big heads.

Doctor Frankenstein used snippets from a whole bunch of people to make his monster. I was told the experiment didn't turn out well for him either. 😂

The problem is oldest than your video suggests. Automatic translation and even grammar correctors have been distorting human language (and reducing language variance) for decades, and we have incorporated their language quirks in our day to day language.

I think a big part of the convergence is that people often are attracted to certain tropes and conventions when it comes to what they like, so AI produced images are actively being 'pruned and purified' by our preference of our existing cultural paradigms. What I think is really interesting is the feedback, where people's tastes of which conventions they like are in turn influenced by AI art.

This is exactly what I've been thinking since all of this exploded into popular awareness. It's like a giant ouroboros eating it's own tail. I'm glad to see that people are talking about this. Editing to add: Will you critics please lighten up? I'm not anti-AI. I'm just agreeing with Sabine that this is a potential problem that should be studied. All new technologies have potential problems that need to be studied and understood. Pointing this out does not make me some kind of neo-luddite.

I really like this take. Thank you for all the information.

I called this a few months ago, good to see I'm not the only one who can see where this is heading.

Look how people are more and more hating CGI in movies to the point that some movies refuse to do any. If you have ever read anything written by AI you know it has the ability to make the most exciting subjects boring.

Consider what generative AI actually is. It's designed to produce data which most closely resembles its training data. So distinctive and idiosyncratic ideas are actively selected against.

I hope it boots universities a bit to start bringing individuality back into academia. The amount of students I heard that said "whatever I'll just chatgpt it and work over the bumps" is staggering.

I’m not surprised by the finding and I dare say that it is obvious. I dare say that because such a thought has crossed my mind and I have often tried to convince others who are willing to listen beyond the hype; AI is “artificial” but not “intelligent”. There are different ways to assess and critique AI - philosophically, linguistically, psychologically, biologically, etc, which many thoughtful experts, beyond the industry, have challenged the claims of AI, in particular AGI - but, for me, it comes down to creativity and novelty, the latter AI lacks completely and the former AI can only mimic. If you want to be impressed, see a human child.

Great presentation, Sabine. I have always maintained that AI is like students cribbing exam answers. One student just has to copy one thing wrong, once. From then on it is a done disaster. When scientists hypothesised robots making copies of themselves - they never saw this far into the mess.

The popular image generation models prior to Stable diffusion were GANs (generative adversarial networks). The way they worked was to have two different networks - one trained to generate images, and the other trained to classify images as real or fake. This forced the generator to learn to avoid the most identifiable characteristics and to generate a diverse set of images. Stable diffusion was more effective and scaleable for higher resolution images, keeping the whole image globally coherent. But it's likely that reviving some adversarial techniques could help with the diversity issue.

I've been drawing attention to iterative decay of information in these systems for some time. Eventually we will just end up with a 'grey goo' situation similar to that suggested for uncontrolled nanobot replication.

Thank you for the video! I am not an expert on AI just used to work with big data. It has never occured to me that AI collapse due to low quality of input might be an issue, but it actually makes sense. I would like to point out that adding more randomness might now be even viable solution to this problem since all the existing random number generator are in fact pseudo-random (the random result can be predicted if you know the algorithm) Guess we will see how it will play out in the future

Surprised we arent already forcing watermarks on ai content. Actually blown away. Like giving a kid a staw house and fireworks and not expecting a fire😊

A third potential is that we decide to move on from brute forcing LLMs to work and get more efficient or different learning models. A human does not need to look at a billion images to learn how to draw. Even if we don't have AI that are capable of what we can do, it does demonstrate that there are better ways to design AI. Right now it's kinda brute forcing and incredibly inefficient

I was just called by an AI telemarketer. I am sick & this was reflected in my voice when I answered the phone. The AI tried to relate to me by matching the quality of my voice. It sounded like a drill sergeant or concentration camp guard mocking me.

AI as it currently stands will become a generalization of people. If it doesn’t become a unique individual, or become better at imitating us, we’ll just learn to recognize them as we grow up and live with them.

I kind of wonder if this problem actually started with the widespread use of the internet. We went from communicating with books, which had to meet a certain standard before the expense of publishing could be justified, to zero-cost sharing of opinions on the internet, to having machines lap up these opinions and feed them back to us. Each of the above steps involving less rigour than that which precedes it.

AI can learn from itself when there is an objective outcome to measure. For example Chess, Go and Poker AI engines can improve by playing against themselves (though they also benefit from historical game records and playing against humans). Where there is no objective measure, such as art or creative writing, it's difficult to see how AIs can improve without human input.

So basically ai need a human brain. Take note matrix.

Great video. This problem may be a teething problem, though. After the computers started beating the top human Chess players, Go players like myself felt smug. We said things like "Chess is about crunching through possibilities. Go requires real intelligence." For years we annoyed chess players with this sort of talk, citing the inability of computers to beat the top human Go players as proof. This lasted about 18 years until AlphaGo came along in 2016. Important point: AlphaGo does not play like humans, except faster and more accurately. It came up with some genuinely novel moves. So, I would not take any problems that AI is now experiencing as an accurate predictor of what AI will be like in another two decades or so.

I think that if we program in randomness, they'll introduce wacky and impossible and obvious problem elements. The problem is in explaining how to adjust and add randomness to a program which doesn't understand the original state and how it has simplified and made things uniform. It doesn't understand what it does, so how can it introduce some oomph. How can it know when it's introduced too much? There are way to many parameters which can be tweaked.

Please, would you put under the video the references of the sources you use? A study from Japan, another from France - please, give us the links! Thank you.

I agree. I also know that the amount of input required to create accurate images that are on par with illustrations, photos or graphic design, especially if it needs to fit within a preexisting brand visual language, and need to be otherwise visually descriptive to achieve a specific goal, is ridiculous. Try entering meeting notes, and text describing a brand visual language into an AI generator and to visually convey a concept and you won't get anything that works anytime soon. Give a 250 word creative brief to a qualified professional an you will get something back right away. In this way AI is not close outpacing the human brain in this way.

Than you for covering this! I've been saying this since I saw the 1st interest in AI art

I’ve been saying this to friends for about a year now. I’m glad I’ve finally run into an expert identifying and addressing this potential problem.

Very interesting findings there and suggest that while initial models create biases, refined models may also create average bias.

Some call this phenomenon "dead internet theory", where some companies (barracuda networks) show that only 36% of the modern internet is actual human traffic, with the other 64% being automated traffic (eg. bots, GenAI, spam, etc.). There is only so much human original created content you can train these huge AI models on.

This is something touched on in Halo regarding Cortana, and other “smart” A.I.s. This is known, in the Halo universe, as “rampancy.”

This issue has occurred to me for quite a while. I have worked with Big Data extensively and had brief real world experience with AI development. AI’s reuse of AI-generated data seems highly likely to result in a “creativity asymptote.”

its already happening on youtube. the same videos with different thumbnails usually one or two words are changed

This is probably a reflection of the filtering used to keep AI presentable to the public. Filtering limits the bandwidth of the input data and successive runs with a filter would narrow the scope even further. It's like running noise through a feedback loop that also has a filter on it. Eventually you may get a sinewave tone with enough passes through the filter. There's also something about the limiting rule sets that act a lot like quantization with generative sound design when used with random input values. If you change the rules to the point of quantizing down to one note, that one note is all you're going to get regardless of the input values and how random they are. Weird analogy, but I swear there's some very basic AI behavior that can be observed with something like generative music - and certain rules like picking scales or rhythm patterns will make a melody fit into a genre just by letting a computer do its thing. Sure images or language are more complicated, but it seems that similar nuances in emergent behavior are there.

This is why I've never had any real fear of what they deem singularity. I will never get to a point where it can improve on itself because all things break down over time. Nothing gets better.

Exactly! I've been wondering about the potential for this problem. Thanks for pointing it out.

One potentially positive side-effect of this "averaging effect" of AI output - if it continues - is that creative people who want to distinguish their output from the common generative stuff will be forced to be more individualistic and idiosyncratic to be distinct and valuable. Of course if generative output is then trained on their later output this becomes an "arms race".

AI is geared to come up with a solution using the smallest amount of computations possible. It means that ignoring diversity and details is part of its basic make-up. It can't capture interesting quirks and details and spurt them out once in a while, in some outputs but not others. It also means that if it found one way to get to an acceptable solution, it will only try to get there quicker the next time around. If you've experienced getting stuck in a rut with ChatGPT, that's why.

It's basically positive feedback: the circuit always rails. Analogous to if you only eat fast food and consume soft drinks - junk in, junk out.

This reminds me of something happened in bycicle industry, sales were declining so "creatively" they "invented" 29" wheel and stopped producing 26" .

As a 80-90s kid, I just wanted to say your thumbnail looks like artwork for a gameboy game with that left stripe.

To go further Companies will have to start paying for people to create high quality data. Art, photography, text like stories or reviews, solving logic and math problems. Rather than just scraping the Web.

In Computer Science, the consensus i feel is that it CAN help you considerably, but students shouldn't be given access to it as they need to learn to work without it to not become dependent on it. And second, that AI puts much more stress on the code review process which needs to be done more carefully now than before. When we do use it, we typically use it like Google, and are always reviewing what it outputs carefully. In regular science i think this will soon also become the norm, it has the potential to aid researchers, but it puts stress on the peer-review process (which was already problematic before) and the wave of students using it is worrisome because they're not really being forced to really know the subjects deeply on their own. In regular science also, topics can be more complex to review, i can easily tell when the code the AI generates is invalid, my tests, compiler and other tools help me catch errors from the AI, scientists don't really have these guardrails and need to rely on their own parsing of the information. So they need to be doubly aware of the fact AI can hallucinate pure nonsense out of the blue.

I learned about these problems a LOOOONG time ago! I was raised on the acronym "GIGO". . . then again, my father was a computer programmer with Monsanto back in the 1950s and could program computers in MACHINE language. Eventually, he was able to learn the brand new "high level" languages of FORTRAN and COBOL. I've since turned my attention to Sociology and can see where mutually constructed social realities have crept into compute programs -- and those social constructs are especially strong in AIs.

I absolutely love reading comments & knowledge shared by people. Seems like a computer or program doesn't really know the world, discernment. Still pretty interesting & useful.

one thing I saw someone bring up in reference to an assured AI model collapse, is that these AI models require more and more training data forever, in perpetuity, and there just isnt ever going to be enough unique data in the world

Business are going to realising they’ve been sold a faulty product based on hype.

I wonder if the generated elephants look so similar because usually generated images try to match the sample sizes (512x512, 1024x1024) which only leaves so much room for good compositions and wonder if in future with larger models we might see this change a bit more.

It's becoming a real pollution issue now, with candidates titivating their CVs and students bolstering their theses with AI generated crap, which is already showing signs of becoming increasingly generic. It will inevitably settle down. Most people are developing a very good nose for AI, and as Sabine's examples show, it's starting to look increasingly like all those annoyingly garish CGI Marvel movies.

There’s a chaos setting in Midjourney. I imagine that modifier will be available in everything. It works well for randomness although the weirdness is a bit much

I have been using gemini's free version since it's release, and I have been noticing a sharp decline in reliability and a massive uptick in refusal to answer questions, i.e. ignoring questions on some very public information such as laws etc.

I love listening to your channel, you explain the most complex subjects in a clear easy and simple way for easy comprehension Thank you and keep up the great channel Love it

1:10 Why so surprised? It's well known that in echo chambers all differentiating opinion/perception gets eliminated. And when AI is following the input data given, it will convergence to a consensus in order to establish its rules. This is also why "learning the rules" works, the randomness is just to be less sensitive to small input variations.

Thank you for telling the truth! Most people seem to be really naïve about it

Indeed, thank you. Chris