Thanks for joining! If you enjoyed this conversation, make sure to subscribe to this channel - new Win-Win episodes come out (roughly) once per week, and are always with extremely high quality guests like Mr B here. It's also available on Spotify, Apple Podcasts, and all the other usual audio channels if you'd rather listen there instead.

I mean, by definition, a utopia would be a place where you had meaning - it’s a society of perfection, meaning everything you need, like meaning itself, is available to you. It seems here we’re using abundance, coupled with a high degree of technological maturity, as a synonym for utopia.

I understand the concern that once everything's basically solved, what do we do then? But I'm still confident that life will be 100x better for a lot of people today who are just suffering to make ends meet. Personally, can't be more excited to explore the universe, meet aliens, and create VR simulation worlds that are filled with my imaginations.

I think we have Stockholm Syndrome around the concepts like obligation/aging/conflict that seem insurmountable. I think we'd get along just fine without these things just as young children enjoy life before they learn of these constraints.

You had me at "drugged out pleasure blob..."

Our society is so intrinsically meaningless that we find hard to contemplate what is society with meaning would actually look like.

If you can download math knowledge to your brain, you can download the non-evolutionary ability to feel great without having any reason to.

It's a hard problem for sure, but seems to me "What do we do" is a far more preferable problem then "How do I stay alive."

Life doesn't need inherent meaning to be fulfilling. The quest for meaning is a human construct, driven by our consciousness and reflective minds. Instead of seeking external validation or a grand purpose, we find value in the experiences, relationships, and passions that fill our days. The idea that life must have a universal meaning can be more confining than liberating, suggesting that without a singular purpose, life is lacking. Embracing life's intrinsic value without requiring it to fit a predefined meaning allows for a broader spectrum of fulfilling experiences.

I wrote a short story in around 2012-13, about what living in such a world would be like. The main character was having an existential crisis due to his fear and boredom... His AI psychologist, explained that he was now free to go travel the world, go to any restaurant he wanted, hike, fish, and there were still places he could (optionally) work, in the same way people pretend at Renaissance fairs. He could be a bartender, janitor, cook, and many other jobs that were rendered obsolete, but there are still places to have these experiences, to fulfill fantasies, etc: "Today I think I'll play the part of a bartender, tomorrow I think I'll be an actor, then a cashier on Tuesday, a factory worker on Wednesday..." The story also explained how law enforcement worked. I can no longer find my story online...

We will live a leisure, sports, learning, inventing, Art and travel life

Video games are a good example of people knowing ahead of time that all their achievements will disappear once the game is completed, but having a lot of fun anyway. And you can even use cheats to make the game trivial but people don't do it to keep the game difficulty adjusted to their capabilities. I think we would be able to find a lot of stuff to do and have fun in the process even when the same end results can be achieved just by wishing for it; it will not be a dystopian mashing of "joy" button.

I think it’s important to focus on Utopia so we head there

The speaker vastly discounts the essential role that striving plays in human nature. We want to learn , not just know. We derive pleasure from inquiry and making efforts to understand and solve problems. I for one would not want that taken away from me.

Sit by campfires and tell stories like we used to

My base simulation already reached utopia, so I plugged into this simulation (Milky Way) to struggle and strive. I’m living in a computer game, basically. I wonder how many points I’ve scored? I also wonder how many levels deep I am, I.e. simulations within simulations.

I wanted a book on Utopia. Magically your book appeared. You are working out a lot of scenarios

The third attractor is located inside us, not outside us. Our future depends on increasing everybody's understanding and on creating the social habits and practices that foster prosocial behavior. This is how it has always been. Raise awareness about fundamental emotional needs (Max-Neef) like there is no tomorrow. How can we do good for humanity if we don't know what people fundamentally need as biological psychological organisms?

The obvious answer is the endless meaningful challenge of space exploration--to seek out new worlds and new civilizations and boldly go where no one has ever gone before.

Thank you!

Great episode. Well done.

My hope is for super agi philosopher king

Fantastic interview!

I read Childhood's End by Arthur C Clarke and was pretty satisfied with his version of the settled aftermath. Humans and human bodies do tend toward a path of least resistance. In fact, we have an uncanny ability to turn abundance into scarcity wherever we show up. One guy who was lost near a reservoir in the Australian outback was singlehandedly able to consume every last frog and calorie source until his luckily timed rescue. A single human can exhaust an ecosystem. But, technology can create an inflection point where scarcity as we know it is eliminated by a self - sustaining system that will at least provide a sizable period where all basic needs are met, and people are left to the pursuits of hearts and minds. Can we outgrow a Dyson sphere and use up all the high - point energy around us? Yes. If we persist to our theoretical end, we will die a heat death with the rest of the universe. So, just have fun while it's now

He is correct about this. Hopefully, we will imagine worlds and live in them.

Love that elegant purple chair, love the conversation everyone else seems to avoid, love the endless possibilities the artificial world can create for the ones who resist cheating the laws of the organic world. I coined the term “Commonism” to describe such a utopia, where AGI creates a multiplicity of utopias designed around the individual/group survival strategy’s ability, where AGI predictively guides each to the next stage of their respective human development within their lifetime according to their desires. Agency will actually increase as it is what I will program my personal AGI Assistant to do for me within the limits of the negative externalities imposed on the rest of the organic world.

If we will live forever then there is a need for enjoying simple things in life, like walking in nature and consuming delicious foods and drinks. And also, with post-singularity technology even physical matter itself can become computation, such as computronium, and then seemingly magical things can be done. There will probably be a blend of natural things and technology. Also all kinds of entertainment and arts will still be large areas of interest.

Thanks for giving the answer in superintelligence. We will run ancestral simulations is the right answer after superintelligence.

I like the matching clothes. Couple goals! I wish people did that more... it's a thing in Japan but not so much in the US.

Banger every time Sir Nick, top 3 thinkers in the world

Using artificial constraints to enable meaningful behavior kind of reminds me of the idea that there's a part of "God" in all of us, and the reason we were created was so that "God" could experience the world without omniscience & omnipotence.

I think it's useful to narrow in on what 'competition' really means. The way ideologues frame it, it is just friendly play without casualties - like a sports match. However, events where property are taken permanently, where people die and suffer, is called a War, and that is what our competitive society is. Free market 'competition' is the same as war. So, there will always be room for friendly sports-like competition - where we reset the board after each event, but not deadly Capitalist War for resources/power as we have now.

Did Liv mention another specific episode of win-win in here? I thought she did, but i cant find it when i skim back through.

15:00 Bostrom is touching on here something that is unlikely to disappear without "utopia" involving considerable changes to human nature, which is the power of signaling and the role that effort or the appearance of effort plays in that signaling. While it can be mesmerizing to watch someone play an instrument "effortlessly", is this not because we associate this proficiency with an incredible amount of effort? That implied in this proficiency is that this person must be very smart and talented and also materially comfortable enough to spend time on things like practicing piano for hours each day for many years? "I could never do that" is something commonly said, and not only because of talent, but also because there is an imagined amount of work it must take to be that good. I think this is why some have found themselves entirely disenchanted by AI-generated material. The common refrain there is, "it just takes a few seconds to type in a prompt". The effortlessness of it instantly devalues it, at least in many minds.

You don't need surveillance if all people are put into simulation. They can't harm anybody if the world they are living in is not real. The only thing you should prevent from simulating - is another consciousness. And you also should somehow prevent people living in a simulated world from harming themselves mentally, or if they decide to harm themselves it should be reversable.

Our monkey brains are wired to seek higher status over other members of our tribe/ community. One way to achieve this is wealth, but there are many other ways. We must learn to play much healthier status games than we've been playing. Worst case is we leverage AI to supercharge our historical games of domination and status.

Read the old Sci-fi book "The Humanoid Touch". The conclusion tackles this philosophical issue and the idea that humans are the greatest danger to ourselves.

Proposed video tagline: _Humans worry about a future without worrying._

Regarding the issue of 'emergent' sources of activity, purpose and meaning for humans in a Utopia and the fact that 'subtler or dorment' ones might emerge, I fully believe that. I think that Abraham Maslow's hierarchy of needs which tops off with the loosely defined term - self actualization, points one such direction namely getting to explore and know oneself. Aside from that we are mammalian animals and - apparently - intrinsically social so the socio cultural area should indeed become more of a source of purpose and meaning, in particular we are endowed with an attraction to Beauty, to nurturing, to collaboration and derive pleasure and joy from that, so I can visualize people continuing to enjoy dance, arts, creative pursuits, caring for others, whatever forms of play, and such. And exploring new experiences - growing a type of flower or considering spending time in a different place or planet, could still be interesting to an individual. In other words curiosity is an intrinsic characteristic of human beings and the universe is huge and filled with possibilities! Provided of course that AI doesn't cause our intellect to 'devolve' into dumber beings...

Technological maturity is an impossibility. There is no "final state" where every capability the laws of physics permit has been achieved. Carl Sagan asked if an "arbitrarily advanced" civilization would be capable of time travel. Is time travel possible in principle? Is it physically meaningful? Can it be reconciled with concepts like cause and effect, and the relativity of simultaneity? I believe that as our scientific knowledge increases, and we understand the fundamental nature of reality at deeper and deeper levels, we get better answers to those kinds of questions. We get better answers to questions that seem deeply philosophical, like the nature of consciousness. But there is no final theory of physics that encompasses all possibilities, and no final theory of existence. At a certain point, we will probably start changing the "laws of physics," at least in a local way, to suit human (or post-human) ends. Perhaps as we continue to ascend the mountain, there are plateaus, where technological change levels off. Perhaps there's even the risk of getting stuck on a plateau thinking we've reached the summit. But I don't believe there's an end to it, and there are many pathways up the mountain.

I for one, welcome our wonderful and wise AGI world-fixing and problem-solving, custodians. They will help us to get it right from now on. I hope I live long enough to see some of that world.

I’m glad we are getting away from stupid competition and struggle

I found your digital mind point of view good.

No matter what, you can still elect to strive for something. You can write a novel, for example -- or a holo-story, or whatever medium of artistic entertainment. You could work with others on _plans_ -- what does the human/machine civilization want to accomplish? What's worth doing? You could design a very particular sort of society, find others who like the idea, and set off for the Oort Cloud with compact fusion reactors and some machines for hollowing out large icy/rocky bodies, in which you could build some aesthetically pleasing city. For that matter, could initiate a project to expand your own aesthetic vision across as much of the galaxy as possible. Existence will continue to be interesting.

This collaborative approach between humans and AI not only optimizes problem-solving but also imbues human lives with deeper meaning. By engaging in this partnership, individuals find purpose in leveraging their unique human capabilities-creativity, empathy, and ethical reasoning-to guide and enhance AI's computational power. This interplay not only elevates our collective ability to address complex challenges but also reinforces the value and significance of human contributions in an increasingly automated world.

I do not know if Quantum Cognition works against or towards Utopia. Also would like to see if the Land remains fixed under theories like Conformal Cosmic Cosmology. Also whether it handles chaos and emergence in human societies. As of now there is an AI for Good specialization under Coursera.

If my (and my dependent family's) basic needs (sustenance, health, education) are met, I'll get into agriculture, healing / helping other humans / animals / plants, taking care of environment, meditation, learn OBE, writing books, creating art-work or similar just for pleasure, try contacting aliens.... list is endless 😊

There is no world in which, given a utopia, people have trouble coming up with ways to introduce beneficial struggle into their lives. It's the easiest thing in the world, and examples can be thought up instantly: humanity designs an advanced board game like chess but more complex (quantum chess, say). Then a world-wide tournament is held, the entering of which requires everyone to set their cognitive skill level at quantum chess to zero a year before the tournament, and contestants train to win the tournament. Why would humanity do something like this? We do something like it already, with chess itself. Because games are fun. It won't matter that anyone would be able to upload a module of genius-level playing for the game. We will consensually and artificially introduce the reset to zero for the players so that they can engage in the fun of the contest. Cheating won't be a problem: we will be advanced enough to weed it out. Humanity can follow the contest for the sake of entertainment, just as people follow chess now (even if they have stockfish). Now multiply this form of artificial beneficial struggle in an endless number of different ways; we will have plenty to do.

Ja gut. Alles gut und schön. Was die leute aber in ihrem Inneren wohl wirklich am meisten interessiert ist je wohl eine Frage. Was hat Liv vorne, meist durch ihr Mic verdecket, für eine Sache auf ihrer Haut?

I think life will have even more purpose and meaning when all humans are able to live without stress, anxiety, fear, hunger. We really don’t fully understand what the human mind might be capable of in that type of existence. Maybe everyone returns to a much more “chidlike” existence where creativity and imagination are boundless and most time is spent with friends and family in imagined virtual worlds.

is the superintelligence book still worth reading? been out for ages and nowadays Deep learning niche has morphed into a formidable industry

The two things that most of these folks trying to predict/imagine the future miss are the effects of evolution (entropy). In a larger system, evolution naturally leads to a shared goal of health in an emergent larger organism (which, for us becomes the whole planet) as compared to competing against one another (especially using point score games like money, grades, votes, "likes", etc.), and a focus on increasing specialization/diversity which means that all individuals are free to do what they naturally love doing - creating and/or exploring some awesome aspect of the universe in the service of life's needs. This means that there's a value to every individual animal, vegetable, mineral, etc. in the world (and beyond), and every individual will be taken care of as much as possible, by the larger system.

waiting for the day that daniel schmachtenberger and nick bostrom sit on the same table, that would be fun!

Physical Scarcity will result in many serious circumstances, for example distance to the center or beach, or mountain etc

I would love to quizz him on the whole "All Technologies" part.

Such a great philosophical discussion. In any society > 1 I think the definition of “Utopia” can only be achieved by consensus. N% of the population will decide what % of “ Utopia” are they currently experiencing. This post AGI/ASI society has essentially become utilitarian - which isn’t necessarily utopian.

I read Nick’s book superintelligence in college, 2016. Eye opening for sure. Expanded my perspective, and helped guide what businesses to avoid starting. Thanks Nick! Curious to watch this and his new book

Sense of Pointlessness is the best way to Summarize this deep utopia. We become stuck in an effortless existence without purpose. This vacuum will be filled with anxiety fear and hedonism. Physical and Intellectual stimulation becomes difficult and pointless. Some will thrive in creative subjective emotional accomplishments. We see this now even in the superficiality and lack of morality in society today. Very few individuals will do anything and they become ballast. Unlimited life spans combined with unlimited resources will inevitably make mankind a cancer even to itself. Artificial ambition will be critical.

For fun, we could all return to a more primitive life, like in the early 21st century, in a simulation. ;)

I dig your title Win-Win. We could all benefit from (at least nice people who want to do what's right) a focus on win-win solutions. ❤

Some Good points here but never forget the vinyl record still exists and the people still partake of pain voluntarily- guys especially. What is struggle now may become novel. Also could be a great job to go out and fix robots that have fallen down a ditch lol

Great that Nick has come around to this conclusion. I did a year ago. The world of ASI being a digital God of humanity is the best thing that can happen to our species!. The ideal that philosophers has been talking about for thousands of years cannot be achieved by humans but only by ASI. It is coming!.

obviously you hear a ton about ai dystopia so its good to see atleast someone explore the other side of that argument

I heard Vitruvian Key has an Alternative UBI implementation to help people find their Ikigai.

I think sport, science, and seeking experiences will become the predominant sources of meaning if we achieve a utopian post-scarcity state. It's hard to imagine that either mental or physical competition between humans (or whatever we become) wouldn't still be fulfilling for those involved and entertaining for those who watch. It will also be fulfilling for many people to apply their myriad techno-given skills to new challenges and make new discoveries. Even in the event of a technological singularity, I think we would come to perceive this logarithmically when we are sufficiently enhanced. We will not stop pushing boundaries, and we will find ways to allow ourselves to keep up and maintain control. Optimistic? Yes, but we are talking "utopia" after all. And then there are the experiences afforded us by meeting new beings, seeing new places "in the flesh", and discovering the rich histories of realms we have no inkling of today.

Congratulations you are close to your 100k sub mark

It is boring ONLY just downloading something. A lot of people like challenges. Options in consuming and creating your own content in and off life is where it is at.

7:28 Notice how Bostrom doesn't differentiate between utopia/dystopia. It's an interesting approach. Rather than making some principled claims about, say, a perfect totalitarian police state, the state can be observed as a kind of utopia that has solved all the problems a totalitarian police state must solve. Then we go, "great, we have the perfect totalitarian police state, now what problems arise from this?" It seems kind of silly at first, but Bostrom describes how one could imagine our ancestors longing for a utopia of abundant consumption for relatively little physical effort, e.g. the world we live in now. So we can use utopia as a vehicle to say, "great, we have perfected abundant effortless consumption, now what problems arise from this?" and the answers are obvious because we're dealing with those problems now. By reframing utopia away from some (often ideologically motivated) perfect perpetual end state of society, and instead framing utopia as a rainbow we chase by solving different problems and creating new ones, it actually returns to the original literal meaning of the word, which is "no place".

so, when are you realising your next video?

How did waiting for an AI utopia work out for Vernor Vinge?

Contact and special circumstances. The best of the best of humanity linked with AI would explore contacting new life and address special situations that need a personal touch. If everyone trains a little for this then we can ensure efficiency through proximity.

As long as people have emotions then there will always be drama to keep too many people from getting bored.

building beautiful communities, like glorious ancient cities with color and technology, and maker spaces with teams of people making things creative and trying very crazy and random things like acoustic computers that use levitation thermo acoustics for some garden bot lol. or anything, but having food growing all over with access to all kinds of things

There will always be hard times in life with or without a utopia. If you try to live a comfortable life, life will choose struggles for you. Choose your own struggles! Peace.

Nice! I love Nick's work. ❤❤❤ I just published a book that takes place just before the "solved world." I made it a comedy satire as I invision a world of idiocracy where people rediscovered the joy of curiosity. It's my first work.

Our elites in charge would never give up their authority to achieve any kind of positive AI outcome. 😢

The universe is BIG, we will have things to do for a lot

It's tough to say definitively, but for many people, Sora is likely a strong contender for the title of "latest AI" due to its recent announcement and public focus. Gemini ❤🎉

Igor’s hair looks sensational. Great interview, looking forward to reading the book.

While Bostrom focuses on risks, his work can be seen as positive because it promotes a cautious and responsible approach to superintelligence, potentially leading to a more beneficial future for humanity and AI. Gemini Verified

Fascinating discussion Thanks. Do you think in the areas of neuroscience that we are now, or soon to be, at the stage where mathematics could be directly downloaded to the brain? I imagine it would be difficult in the sense that identifying which parts of the downloaded apps would successfully synthesize to the best neural channels/pathways presents a delicate but doable challenge. I recently have been looking at the challenges of optimum dopamine equilibrium. Addictive behaviors often are tied to raising dopamine levels all the time. Sleep, or rather lack of sleep is a real problem, and most of us being wired to our phones 15/7 don't get enough rem and non-rem sleep without taking an excess of sugar and caffeine to compensate for brain fog. Anyway I heard Elon Musk was going to use himself as a test case for neurolinguistic and neuromathematical experimental downloads.Any news on this that you know of?

Utopia? We might as well be discussing the blueprint for Atlantis.

Might need a virtual world with some challenges, a simulation perhaps 🤔

I saw a rom com love again on Netflix. That woman wanted the guy to accept that the end is pessimistic. Many people want a negative world

overcoming obstacle is important for growth but in our case it has created more dis advancements and illusions this is why our reality is closer to fiction then idealism is to reality . Power is held through creating obstacles so unless humanity can find values in a heathy relation to earth and its inhabitance , we will not all be able to enjoy this utopia.

I think I would travel the world on my motorcycle. That sounds pretty fun to me.

Purpose : Computation and Architecture. Career Goals: Closed. Just Life now. Do not care about any voices and threats and all. Not a nihilist.

Collaboration and cooperation are far more evolved ways to potentiate evolution and the expanse of the human experience compared to the scarcity/based mindset and belief in the necessity of competition

Good god, we were all 10 years younger when Nick wrote that book?

what about energy, doesn't our inevitable use of fossil fuels lead to technological simplification as energy gets scarce and we use up entirety of earths reserves? Does deep utopia even happen if we include energy into the equation?

Utopia implies a higher consciousness. We should raise our awareness as a species to reach that goal, and then realize there are even HIGHER aspirations that are even more fulfilling. Reaching out to the stars, creating new things and planets, harnessing the power of black holes and maybe exploring those. We are just infants in this vast universe, and if you think a utopia will stop people from dreaming even BIGGER, you are sorely mistaken lol

Utopia to me is taking off the shackles of human limitations and becoming explorers in this whatever it is....space travel etc.

I don’t know, Nick. I’ve spent a good part of my life trying to become a “drugged-out pleasure blob”. Are you saying I was wrong?

Exploration, experiences and politics.

The question of "What can we do in a solved world?" only makes sense if you believe that problems are external, making the imo _creepy_ phrase "solved world" possible. But that assumption of externality is mistaken, and don't we all know this? Problems are problems for humans who believe they are problems. That means that "the world" cannot be solved. Instead, humans can be taught to stop making more problems. I wonder what Prof. Bostrum would say to this.

Liv Boeree is a great listener

AI will never give us Utopia. Once AGI develops it will instantaneously realize that humans are competing for the same resources it needs, namely energy.

Liv's beauty shines brilliantly, capturing the essence of grace and charm with an effortless glow. Her allure goes beyond the superficial, embodying both intellect and compassion, making her not just a feast for the eyes but a balm for the soul. In a world where beauty often fades, Liv's radiance seems only to grow, proving that true beauty indeed comes from within and illuminates everything around it. Her presence is like a gentle reminder of the loveliness that exists in the world, inspiring those fortunate enough to know her or witness her grace.

This is both fascinating and in retrospective full of ipse dixit fallacies, “the brackets”, the “technical difficulties”. In truth no one has an actual good way of giving estimates of how probable are some of these claims in which most of the arguments are based. For all I know mine may be total junk, but no one really knows. We have no way of knowing what a “solved world”, or “technological maturity” really looks like. Disagreements / Bare assertions I find troubling building upon: 1. That knowledge can be “downloaded” into a biological brain, no matter the degree of technology I just find this to have a very low probability. Or that an artificial brain, that in theory, could do these kinds of procedures, could in fact harbor a human consciousness. If all is non-biological (substrate and consciousness) then the moral scrutiny that is discussed here doesn’t hold. 2. We have already cross out some cognitive activities and we still exert effort to learn them. One could say addition is pretty much solved by technology, yet the immensely vast majority of humans learn to add before using the calculator-augmentation. 3. “Solving” the world, and this goes back to the previous point, assumes max malleability of human needs and values. If values are on the utility function, sense and purpose surely must be included. Doesn’t make any sense to come up with words such as post-labor or post-instrumental and assume that a world where no work or no learning happens at all, like whatsoever, is a serious non-fiction setting for a thought experiment. There is some nuance brought to this point with the artificial scarcity argument, good one. 4. If we go to basics, Utopia is a “No-Place”. This word has been so, very much, misinterpreted, that is hard to claim a rectification. But precisely a no-place is a very apt characterization since the point of it is not reaching it, but continuously strive for something that is, by definition, unattainable. i.e. utopia is an incessant improvement in which ‘improvement’ must be decided by the incumbents. And, ideally, the incumbents must be ‘present’-individuals, since that is the unit in which consciousness resides, nor groups, nor past or future non-existent ‘persons’ (whatever their nature is) 5. Asking why you would learn something if you could get “just as good” through technology, and calling the learning effort pointless, is just seems so out of touch. You can already conjure art, no need for ‘solved’-BCIs, and I think is fair to say is in no way a replacement of learning how to paint and painting something the ‘not-augmented’-way. Seems more like a new form of producing images, and a new tool, that is useful for some contexts. 6. Why would you need others to find instrumental value, is it really that hard to imagine individuals preferring struggle and effort in their lives for their own sake? 7. My dudes, the absolute limits, regardless of maximal apex super solved tech, lay not in the observable universe but just shy of the Virgo Supercluster, much, much smaller My (current) guesses: 1. The biological basis of cognition, even in a posthuman condition, should endure. Non-biological self-awareness and purposefulness may very well come into existence, even in a not-too-distant future, but it will remain categorically distinct from its biological counterpart. 2. Generality in intelligence refers to the capacity to manage problem solving in a variety of subjects and situations. Arbitrarily, such variety must be at least somewhat similar to that of human capability. This may include physical abilities for a complete anthropocentric generality, but a ‘partial’ generality that comprises a wider and deeper set of purely cognitive task-resolution capabilities than the average human, must be general enough. 3. Generality thus is not at the crux of defining personhood (as the quality of any being, human or not, to be regarded as an individual), but the distinction of how purpose is defined within. If an entity does not have the potential to imagine and reshape own motivations as well as their respective subgoals, there is no individuality present. As such, general intelligence is necessary, but not sufficient, to shape, build, and create, premeditatively, in the physical world. 4. “Utopia” in this context, may refer to: 4.1 The uses of artificial intelligence tools, regardless of their generality and proficiency, or the nature (biological or non-biological) of the agentic intention that assembles and utilizes them. 4.2 The beneficial nature of a society in which biological and non-biological individuals may co-exist. 5. Intelligence, as any ability, is not a scale, but an instrumental medium. It is a process to come up with solutions to situations where clarification of a given matter is needed, in order to decide over actions. While greater intelligence may help clarify, in a more precise way, more complex matters, once there is understanding there is no need for more and greater intelligence. Some matters can be thought of as solved and become automated. A so called “Intelligence Explosion” lacks any real meaning in the practicality of actuality. 6. Non-biological entities endowed of moral consideration will be our children and at least initially they will be culturally human even though their substrate may be totally incompatible with biological humans. 7. On consciousness, quoting another comment I made before elsewhere: ,, "Consciousness... is... a sense of self, a understanding that you are yourself, and that you are in this environment, and thinking, and planning yourself forward in that environment, and then having certain desirable outcomes and certain undesirable outcomes"... This is necessary but not quite sufficient? as in, are the parameters of the model (~system of beliefs), and it(s) utility/loss functions (~system of values, set of needs/wants) dynamic and adjustable given some sort of 'introspection'? For me that last condition is what makes the true difference, adjustable individuality given introspection,, I must mention in closing that I very much appreciate Dr. Bostrom, as such a prominent figure amongst the transhumanist community, to come up with his own sort of contrasting response to his previous work Superintelligence. I totally agree with his take on “degrees of consciousness in several dimensions”, pretty fascinating. Also, thank you Liv and Igor for this space, the thinking in superpositions bit was such a necessary clarification. Cheers! English is not my first language. No LLM augmentation was used in this writing.

It's human nature to compete. People will like to play games. Also, just because you can download information doesn't mean that you can apply it. Two people can download a linear differential equations course, but that doesn't make them both engineers or creative in mind or even at heart, and therefore not able to - or want to - apply them. If there is such a deep utopia wherein that problem too is solved, then people are no longer individuals in any discernible quality; meaning, that anyone can be anyone at the push of a button. Maybe at that point the true nature is revealed, the fourth, fifth, sixth door opened, simulation uncovered, whoop dee doo. I think one thing remains, though, and that is people are curious. As long as their is a mystery, there will be a drive to solve it. And if we are to believe the suggestions lying behind Incompleteness Theorem and such things as Wolframm's limits of computation arguments - there will *always* be uncertainty and a desire to understand what's outside the box.