my favorite david shapiro take is that human extinction is a neutral outcome

Hearing the stalwart optimist David Shapiro say his P(Doom) is 30% has set my own P(Doom) to 99%.

Unfortunately, a nice future right now is almost entirely dependent on our politicians being smart, forward-thinking, and non-corrupt... So we're basically totally screwed.

Day 1 of asking David Shapiro to do a What You Should Do Between Now and AGI video

Watching this made me wonder where on Earth you found a 70% chance of non-doom.

I appreciate your intellectual humility and willingness to admit you don’t have all the answers. Actually pretty rare these days, even among far less qualified individuals.

I'm 74 and love your Star Trek outfit. It caught my attention, but your thoughtful content made me subscribe.

What I enjoy about your videos is that they are well put together whereas most AI channels just meander for half an hour on the smallest grain of information. Their speculation isn’t even engaging, just a long run-on sentence.

Right now I wish AGI was already used to make decisions that were not solely based on political narratives we need logical management not biased utopian fantasies.

This video is GREATLY APPRECIATED! I cannot emphasize enough how important it is that people with prominence like you make and keep alive the case/plight of the common person.

You got to over 100K because we like what you do and how you do it. The Star Trek shirt was cute on you tho 😍

The best teachers make the very complex understandable by the masses. Your tone is spot on, David.

When I first started following all of the AI developments last year, I was first frightened, then sad about it. At this point, I am 41 years old and was just laid off from my day job due to 'workforce reduction' (automation). All of this AI stuff lately has pushed me to appreciate humanity more. I'm enjoying life more now than ever, and it feels good. It feels like humanistic values are already at a premium, and that's what I'm going to choose to pursue.

I estimate it's highly likely things get worse for a while, but very unlikely things get worse forever. Derailing the engine instead of the whole train.

Thank you for taking the time to unpack these things brother 🙌🏾

Brilliant work David. Your clarity is a portal to eliminating Greedlock. Very grateful I found you. 🙏🏻💪🏻

You don’t have to wait long everyone, the collapse and unfathomable change starts going mainstream fully by May. Our world is about to change in an unprecedented way.

Listening to this in the hospital facing potential surgery for a crohns diseased cause obstruction. Bring on the AI, let’s risk it and go for glory! I’m hoping ai cures my Crohn’s disease in my life (I’m 26)

I listened to that podcast episode. Good stuff. Would love to see you go on more podcasts and share your knowledge with the world.

Hey David, Love your work as always.

“The road to hell is paved with good intentions.” 😅

maybe a sentient AGI is our best hope

My p(doom) has almost nothing to do with AI. What matters is who is building it. The likelihood of AI going to hell is about capitalism and humanity beings doing bad things for profit and power. That will certainly take any AI - good or not - and make it worse. That said, I'd put it at 50%, because if open source can make it first, then we may not be entirely screwed.

You have a special talent Dave, please keep up the great work.

I love the combination and flow of inductive and deductive reasoning in your presentations.

I’ve never even watched Star Trek but really liked you leaning into it. However, I do like the fact that we’re getting more and more serious about these topics because it is needed. And if it means losing the outfit, that’s cool. The more people can join the conversation the better.

I’m happy you took off the Star Trek shirt. Even though I take your content very seriously it stopped me from sharing your videos with others.

Thank you, David, for the time, energy, and effort you're putting in to try and get us all on the same page and the birthing pains of AI and a future yet to come...✌️🤟🖖

My p(doom) is 100%. 99.9% of all species to have ever lived on Earth are now extinct. Do the math, my friends. We were pretty much f’d.

My Pdoom is at 50% because we are more than likely moving towards a cyberpunk dystopia, but I personally think that there is a possibility that a movement or a major event is going to happen and the people or a majority of the population is going to demand regulatory changes and demand for a more benevolent or fair system.

Man, you really put it into perspective. I'm trying to get people to see the implications in my own network and they look at me like I'm crazy.

Welcome to the "Great Filter". 😢

Is this considering AI as the cause of the doom? If so, mine is lower. If its just a general "is our direction towards a dystopia?", then mine is nearly 100%.

Very sobering. Listening with interest.

P Doom🤔 that sounds like a rapper.

Yes, I find your 30% number alarming. I also find your material hugely informative.

Fav channel since I discovered it

You'll always be the captain to me 🖖

have you seen the new OpenAI board members? i think the pdoom number should be increased lul

David :) You continue to impress and improve while remaining grounded in reality. (Which is getting more challenging) The path you are on is representative of good leadership. Leadership is hard, especially under duress. Being wrong is part of the role. Being right is not possible. Maintain a good batting average of not being wrong. With any luck you will bat for all of humanity.. in the only game that will matter. Expect that everyone on the planet will be in attendance. Please know that you have my support, you are definitely not alone. We want you on team humanity. Surrounded by others who also constantly get on base. Take care my friend, Jeremy.

You shouldn't forget the open source community for local AI, I'm having a lot of fun running local LLM, stable diffusion and TTS on my home computer. New findings appear every week with scientific papers open to anyone. There are quite a few fanstastic models out there like Mixtral 8x7b or Qwen 72b. With liberating the technology we'll liberate the people. Everyone is free to use, host and finetune their own AI models. You can even build companys around these as their licensing often allows commercial use. I'm sure that a Cyberpunk scenario can be avoided with an open approach on AI.

First time I’ve agreed with you 100% on any of your videos

Inclined to agree. I have been working on a solution for economic agency for the past few years. I'm hoping to get a working system published by the end of the year. It is a FOSS project, of course.

I subscribed last year once I realized how well-rounded you are in the sense of self-taught liberal arts/law/philosophy, ethics, politics, etc. At first the Luc Piccard outfit did throw me off. And you are correct, we do still have the power to vote with our feet/dollars/votes.

Delivering information and especially important information in a format that the person will receive and entertain is crucial to inform masses. It's something I think about alot and let play out. People very different from me that I may even find ridiculous are perfect to reach an audience I could never connect to myself. As long as people know what they need to know to a certain point, I'm actually thankful they exist lol

Dave made a good point about messenger credibility. I used to be part of the LessWrong crew that Eliezer Yudkowsky started. It was a locus of Rationality but descended in to endless navel gazing. As an Architecture minded person I had grown out of the engineer mindset of trying 'just be right and that was good enough', and rather using being right most often to produce strategies and governance frameworks to guide corps towards better directions. Being right and a communicator was better than just being right (and paid more). I saw Eliezer in an interview yesterday being engaged with adults who sadly had the general knowledge and mental capacity of the C grade average primary school student. Every step of the way he was right, the things he called out were correct, but he looked like a dork by fighting to be understood in an episode of Idiocracy News (tm). They could have just put "Not Sure" as his name and being done with it.

We are all just dogs in big techs hot car. Helpful they’ll remember to leave the windows down for the 99% of us.

With all the talk and likelihood around going cyberpunk, I’d love to see a video on how we can actively tip the scale to work on having a more solarpunk outcome

This is meaningful dialogue! Thanks… I think we need to go beyond AI and tech… into social/governmental trends, to fully get a good P (DOOM) calculation. Have you read or seen UKposts on Peter Turchin… he has a mind expanding view of history and states reasons for his thoughts on where we are now and where we may be headed.

Great content, aka, deeply interesting thinking and analysis...thank you!

I got dismissed from my previous job. Now my now job is language data annotator , I feel like a traitor but fuck it rent needs to get paid

OK, I'm stopping at Bass Pro Shops on the way home from work tonight to pick up a few more boxes of ammo...

It is extremely difficult, if not outright impossible, to become a established billionaire while remaining/being a good person.

We have a society that is highly unlikely to even consider a universal basic income strategy (some US states are outlawing even the discussion of it), while unemployment will undoubtably skyrocket as AI agents become ubiquitous. So, if you zero out all the existential risks of AGI near/AGI, the bad actor scenarios, rogue AI, etc., just +30% unemployment is historically a society killer. That make the baseline scenarios, without everything else that can go wrong, generally negative, even society collapsing. It also makes the rise of authoritarian governments, possible authoritarian corporatocracies, and the demise of democratic structures, far more likely. P(doom)=30% is being optimistic in the short term, although the long term outlook, after MAJOR societal upheavals, could end up being positive. It may take a while, but will the pain level even be worthwhile?

The three horsemen of the AI apocalypse: the military, the corporations, the government. Notice who is not in that group? The people, with open source AI.

The Zuckbot art is unsettling lol

For some reason I see the the most probable possibility of good outcome just being pure luck. Sure politicians are corrupt, corporations greedy and most people have their heads in the sand, but if things somehow happen the right way at the right time, it might be fine :)

If you are right about agi by around September and that a hard takeoff thereafter is likely, in that scenario, isn’t the technological space so much faster (pretty much immanent) relative to governmental and societal structures and dynamics, that any change in the latter space is to slow to change anything in the greater scheme of things?

Total Information Awareness.

I'd really like to hear other's perspectives on Silvio Gesell's 'Natural Economic Order' with respect to AI. This is from its preface: 'At the present day it is easy to imagine an economic system of high technical efficiency coupled with gradual exhaustion of the human material.'

Eh, still holding out for our benign alien overlord, unless we start cram training AI with Law and governance… 🤔

No brakes, AGI has to happen and happen now.

Good takes. Maybe a video on the reverse side of this would also be interesting. How do we get p(utopia)=70%? What is currently working as it should and which endeavours are worth exploring even further?

Thanks for boosting the signal on the For Humantiy podcast (@ForHumanityPodcast) - As you say the default path if we do nothing is doom in some way or another. I also think that only with enough public conversations at a level understandable by most people can there be enough political will power to take the right actions. Hopefully some of those people will be able to identify what the right actions are because at this point after watching hundreds of hours of podcasts on the topic I'm none the wiser.

Great video! Thank you. I'm a little surprised (and worried) that your p(doom) is so high. Before I can formulate my own value, I would need to define "doom," and I would need to add a second variable of time, meaning consider p("doom", t). So, if doom means something worse than a worldwide economic depression, maybe Dark Age or human extinction, then my p(doom, 5yrs)~5% and my p(doom, 10yrs)~50%. Big difference there because something exponential is happening right now.

We have two roads the first is regulatory capture the second is ASI literally being the second coming

I'd love to hear the markov chain that somehow reaches .3 without tripping over negative feedback or physical limitations

I think we are at if not “ the” filter we are at a filter. I’m leaning great filter

whats the opposite of roko's basilisk, a decentralized guardian AI angel that unconditionally protects and uplifts everyone?

Spot on, David. Your breakdowns are well thought through. You do have a very good grasp on human motivations. I see the "move fast and break things" mentality driving AI progress right now. I do believe in human resilience and the ability to adapt though. As someone in data and marketing, I've watched Gen Zs adapt to the economic changes in front of them. How they value education, job opportunities, career paths is completely different than I had to in the 90's and 00's. It will be ok. It will be different, but we'll adapt as we always do.

i wish i had never heard about roko's basilisk

Another area for you to investigate is public image experts that can probably help you a lot with your appearance. I think that you are one of the best voices, if not the best of all, in this area and it would be a great loss for everyone if details like a star trek shirt would diminish your influence.

That Elon pic was gold!

“Hell,id piss on the dam spark plug if I thought that would work “

Beff Jezos still cracks me up because in Dutch "bef" is the obscene word for cunnilingus 🤣

I’m not sure about the number, but all the points you’ve raised here are spot on. There’s been a trend of increasing populism, and I hope it’s indicative of governments paying more attention to the common citizen. In the past, populism has given rise authoritarian regimes, so it’s not a great track record, but it’s too soon to say how that aspect will play into society’s progression. There’s also another company, Verses AI, which is taking a much different approach to AGI, and allegedly they’ve been able to demonstrate state of the art level performance on a basic RL benchmark on radically minimized compute, but they’re very closed on implementation details. Personally, I find the language on their website a little grandiose, but I do believe in their approach. Something like that could pave the way to widely accessible AGI, leading to a much more decentralized concentration of power.

Dave. Alarming.

30%? Not 33.33333333333%?

Colossus: The Forbin Project is a master class in AI

Im with Yann Lacun here. The chance imo is very low, because there will be extensive research on alignment, and a ton of restrictions from the government. There is also a massive amount of hardware and money that is needed, so humans will be in control for a long time at least. When AIs are starting to have some autonomy they will be deployed in sandboxed environment, and a ton of safety restrictions will be in place. Also there will be quite some time before we have AIs that have full AGI. Navigating the physical world is extremely difficult. Just something as simple as being a babysitter or work in a kindergarten or learning to play basketball will be a real challenge for any robot. In order for it to be true AGI it needs to be able to learn how to do everything an average human can do in the same amount of time. The real world is so volatile and chaotic and the AI will need to be able to handle a ton of edge cases, something that AIs are very bad at today. I absolutely love AI and LLMs are amazing, but they are not even close to being AGI. This is just my opinion but I believe an AI will be superhuman at math before they are anywhere close to being able to navigate the real world like a human. Particularly if the AI doesn't get special treatment or help from nice humans while trying to do so :D.

Some accelerationists *want* the end of the humans.

"The road to hell is paved with good intentions." Is what comes to mind

The arc of history is long. We may go through a period that feels like (p)Doom but I’m confident we’ll come out on the other side; especially with the assistance of even the currently open-sourced models. The cat feels very much out of the bag.

Did you add rodrash style to the prompt while generating faces of different CEO's ? 😅

As usual, an excellent analysis of what's going on and where we might be headed as a species (dustbin) but there's one very important factor that's consistently not being taken into account in any of these discussions: catastrophic climate change. That's still happening, and it will go on happening as the world descends into that cyberpunk dystopia. How hard really will the cyberpunkian power structure work to keep alive the millions, if not eventually billions of no longer needed workers?

I thought you were all for a CERN type centralized frontier lab with heavy oversight. Did you change your mind?

Honestly 30% is really low considering how much you know about ai and you said that possibly agi is achieved by August or September time, at least some sort of agi. Would have thought it would be between 50 to 70% and asi would be achieved within 2 to 10 months after that. Should be 90% tbh.

I was waiting for some optimism to justify the 70% part.

Hi Dave, Your books are only on Barnes &Noble but Barnes &Noble exclude selling to UK (and countries in EU). I would lIke to buy your books but can't.

we need syndicated development - thousands of companies growing in the same time

Wait a minute... Picard WAS serious! Bring back the shirt!

Dave, some guy asked you to make a "things to do before AGI" video. Also, can we think about how to survive in case of...? I mean, Uncle Sam and the other tech leaders already built their bunkers with lots of food and ammo.

Assume we get AGI and hard takeoff to ASI and thus dramatically acceleration throughout science (potentially leading to fusion, defeat of aging, full immersion VR including all five senses and indistinguishable from real reality etc. relatively quickly), all coupled with UBI (let’s take the dependency scenario for the discussion). What would that look like from an average individual perspective? If it just means unlimited free time, no stress, unlimited lifespan in a young body and entertainment (and be it in vr), that kind of cyberpunk outcome (essentially brave new world but with unlimited lifespan), i don’t quite see the dystopian aspect to it.

14:00 and on is my biggest immediate concern. Perfect insight there. I had a long comment typed up, and then Grok dropped. Well, it doesn't change my perspective on what we face, but I think with this, we have entered into the FA phase of FAFO. What the FO part of the equation becomes is well gonna be part of the ride I guess. And while I'm sure everyone who is happy for the open source drop of a giant MoE model, it has my ears perked up as to what this was meant to get in front of.

What’s the role of open source in this? Also meta and now xAI with open sourcing grog, are semi associated with that to some degree.

There are too many good people in the world to let the catastrophic scenario come to fruition (in my opinion).

AI definitely has the potential to create a relatively perfect world or at least more equitable, it seems like those at the top aren’t aiming for harmony or equity just more hoarding of wealth and resources. If they get this wrong there's virtually nothing we can do as just regular everyday people. Maybe it's just me being odd, but if everything was to come to an end, at least I'd go out knowing that I didn't miss out on some monumental achievement that came after me-it all concluded with my era.

15:40 That's what is called overfitting in machine learning

David, will there be a video with your thoughts on the new scientific work: "Quiet-STaR: Language models can teach themselves to think before they speak." Apparently, this is the same Q* algorithm that Sam Altman did not want to talk about with Lex Friedman.

Good explanation! Though it's really confusing that the first slide describes an "inevitable" slide toward "destruction and suffering" - twice - but it only has 30% probability? Also, suggesting "voting with your dollars" (as in boycott) is the kind of thing that could help if you have an audience of 100 million. For smaller audiences... the first thing that comes to mind is to suggest everyone join social groups and discuss the hazards with others, write your members of congress about your concerns, share these videos, etc.