for - progress trap - AI - inequality

for - progress trap - AI - democracies becoming autocracies - Hitler - Trump

for - progress trap - AI - propaganda - brainwashing

for - progress trap - AI - Panopticon - Orwellian

for - progress trap - AI - arms race amongst dictators! - Trump - Putin - Kim Jong Un

for - progress trap - AI - misuse of power - Palantir?

for - progress trap - AI - technology as an amplifier - technology acts as an amplifier, allowing humans to fly, to move at speeds faster than any known animal, to lift things no living creature can, etc - The danger is ignorance and polarized views combined with extreme self-rightiousness

for - progress trap - AI - legislation required - tell it to Trump!

for - progress trap - AI - rogue AI vendors - like Grok

for - AI - progress trap - interpretability testing - deception - Is it possible that AI could even change their node behavior as a deceptive move?

for - progress trap - AI - hackers - hack the AI constitution

for - progress trap - AI - metaphor - psychopath with a nuclear bomb

for - progress trap - AI - abstraction - progress trap - AI with feelings & AI without feelings - no win? - One major and obvious aspect of current AI LLMs is that they are not only artificial in their intelligence, but also artificial in their lack of real world experiences. They are not embodied (and it would likely be a highly dubious ethical justification for their embodiment as in AI - powered robots) - Once we have the first known AI robot killing a human, it will be an indicator we have crossed the Rubicon - AI LLMs have ZERO realworld experience AND they are trained as artificial COGNITIE intelligence, not artificial EMOTIONAL intelligence - Without having the morals and social norms a human being is brought up with, it can become psychotic because they don't intrinsically value life - To attempt to program them with morals is equally dangerous because of moral relativity. A Christian nationalist's morality might be that anyone who is associated with abortions don't have a right to live and should be killed - an eye for an eye. Or a jihadist and muslim extremist with ISIS might feel all westerners do not have a right to exist because they don't follow Allah. - Do we really want moral programmability? - When we have a psychotic person armed with a lethal weapon, that is a dangerous situation. If we have a nation of super geniuses who go rogue, that is danger multiplied many orders of magnitude.

for - AI - progress trap - training - movies like Terminator - This is a case of reality imitating movies - Another example - humans mismanagement of the biosphere and elite abuse and intransigency

for - AI - progress trap - developing harmful traits

for - AI - progress trap - AI won't always listen to humans - probably due to unintentional design

for - progress trap - AI - 5 risks - autonomy - misuse for destruction - misuse for seizing power - economic disruption - indirect effects

for - AI progress trap - Trump

for - AI progress trap - ie. Trump family investing heavily in AI

for - progress trap - social media - misinformation - AI algorithms hijacking and pretending to be human

for - progress trap - AI - totalitarian government - can exploit for centralized, non-self-correcting control

for - progress trap - AI can become political lobbyist for increasing rights of AI

for - progress trap - AI as legal person (US Corporation) - richest person in the world could be an AI

for - progress trap - AI

for - progress trap - AI - safety - irony

for - progress trap - AI - brakes reality checking loop

for - progress trap - AI - suicides

for - progress trap - AI - sycophants,- example

for - progress trap - AI - sycophantic design

for - progress trap - AI designed to be affirming

for - progress trap - AI therapy

for - progress trap - AI - therapist - subtle attachment

for - progress trap - AI - assisted suicide

for - progress trap - AI

for - progress trap - AI - like NAFTA

for - progress trap - AI - narrow boundary

for - Progress trap - AI - low trust society

for - consciousness, AI, Alex Gomez- Marin, neuroscience, hard problem of consciousness, nonmaterialism, materialism - progress trap - transhumanism - AI - war on conciousness

Summary - Alex advocates - for a nonmaterialist perspective on consciousness and argues - that there is an urgency to educate the public on this perspective - due to the transhumanist agenda that could threaten the future of humanity - He argues that the problem of whether consciousness is best explained by materialism or not is central to resolving the threat posed by the direction AI takes - In this regard, he interprets that the very words that David Chalmers chose to articulate the Hard Problem of Consciousness reveals the assumption of a materialist reference frame. - He used a legal metaphor too illustrate his point: - When a lawyer poses three question "how did you kill that person" - the question is entrapping the accused . It already contains the assumption of guilt. - I would characterize his role as a scientist who practices authentic seeker of wisdom - will learn from a young child if they have something valuable to teach and - will help educate a senior if they have something to learn - The efficacy of timebinding depends on authenticity and is harmed by dogma

for - progress trap - transhumanism - AI - war on consciousness

for - AI - progress trap - Open AI device

for - youtube - BBC - AI2027 - Futures - AI - progress trap - AI - to AI2027 website - https://hyp.is/0VHJqH3cEfCm9JM_EB3ypQ/ai-2027.com/

summary - This dystopian futures scenario is the brainchild of former OpenAI researcher Daniel Kokotajlo, - It is premised on human behavior in modernity including - confirmation bias of AI researchers - entrenched competing political ideologies that motivate an AI arms race - entrenched capitalist market behavior that motivates an AI arms race - AI becoming embodied, resulting in Artificially Embodied Artificial Intelligence (AEAI), posing the danger to humanity because it's no longer just talk, but action - Can it happen? The probability is not zero.We don't really understand the behavior of the AI LLM's we design, they are nonpredictable, and as we give them even greater power, that is a slippery slope - AI can become humanity's ultimate progress trap, which is ironic, because the technology that promises to be the most efficient of all, can become so efficient, it no longer need human beings - Remember Jerry Kaplan's book "Humans need not apply"? - https://hyp.is/o0lBFH3fEfC1QLfnLSs5Bg/www.youtube.com/watch?v=JiiP5ROnzw8 - This dystopian futures scenario goes further and explores the idea that "humans need not exist"!

question - What about emulating climate change gamification of "Bend the Curve" of emissions? - Use the AI 2027 trajectory as a template and see how much real-life follows this trajectory - Just as we have the countdown to the https://climateclock.world/ ( 3 years and change remaining as of today) - perhaps we can have an AI 2027 clock? - What can we do to "bend the dystopian AI 2027 curve" AWAY from the dystopian future?

for - progress trap - AI - sharing intimate details with

for - question - progress trap - open source AI models - for blackmail and ransom - Could a bad actor take an open source codebase and twist it to do harm like find out about an rogue AI creator's adversary, enemy or victim and blackmail them? - progress trap - open source AI - criminals - exploit to identify and blackmail victiims

for - progress trap - AI - Anthropic Claude 4 - blackmail - from - youtube - Kyle Kilinski Show - AI is completely out of control - https://hyp.is/GhDOzj0nEfCvHZdiUaw4gQ/www.youtube.com/watch?v=4j1gjSoRt8Q

for - progress trap - AI - Anthropic Claude 4 - blackmail - rare behavior - but still possible! It only has to happen once!

for - progress trap - AI - blackmail - AI - autonomy - progress trap - AI - Anthropic - Claude Opus 4 - to - article - Anthropic Claude 4 blackmail and news leak - progress trap - AI - article - Anthropic Claude 4 - blackmail - rare behavior - Anthropic’s new AI model didn’t just “blackmail” researchers in tests — it tried to leak information to news outlets

for - progress trap - AI - AI - blackmailing human creators - AI - autonomy

for - progress trap - AI - autonomy - example - Grok

for - progress trap - AI - Grok - Elon Musk programs Grok to lie about South African refuges

for - definition - AI containment - progress trap - AI containment

for - claim - AI for a new type of democracy? - progress trap - AI - future democracy

for - AI - progress trap - Youtube - bioweapons is not the only threat. Nano technology and many others can be turned into weapons of mass destruction - RDeepSeek R1 just caught up with OpenAIs o1 - There is no moat@ What does this mean? - David Shapiro - 2025, Jan 29

for - progress trap - AI superintelligence - interview - AI safety researcher and director of the Cyber Security Laboratory at the University of Louisville - Roman Yampolskiy - progress trap - over 99% chance AI superintelligence arriving as early as 2027 will destroy humanity - article UofL - Q&A: UofL AI safety expert says artificial superintelligence could harm humanity - 2024, July 15

for - AI - progress trap - interview Eric Schmidt - meme - AI progress trap - high intelligence + low compassion = existential threat

Summary - After watching the interview, I would sum it up this way. Humanity faces an existential threat from AI due to: - AI is extreme concentration of power and intelligence (NOT wisdom!) - Humanity still have many traumatized people who want to harm others - low compassion - The deadly combination is: - proliferation of tools that give anyone extreme concentration of power and intelligence combined with - a sufficiently high percentage of traumatized people with - low levels of compassion and - high levels of unlimited aggression - All it takes is ONE bad actor with the right combination of circumstances and conditions to wreak harm on a global scale, and that will not be prevented by millions of good applications of the same technology

for - quote - progress trap - AI - developed by unwise humans - John Churchill

quote - progress trap - AI - developed by unwise humans - John Churchill - (See below) - We haven't even got to a planetary place yet really - and we're about to unleash Galactic level technology - So we have a we have a lot of catchup that needs to happen in a very short period of time

for - AI - progress trap - example - Facebook AI algorithm - target - increase user engagement - by spreading hateful conspiracy theories - AI did this autonomously - no morality - Yuval Noah Harari story

for - AI - progress trap - example - no morality - Open AI - GPT4 - could not solve captcha - so hired human at Task Rabbit to solve - Yuval Noah Harari story

26:30 Brings up progress traps of this new technology

26:48

question How do we shift our (human being's) relationship with the rest of nature

27:00

metaphor - interspecies communications - AI can be compared to a new scientific instrument that extends our ability to see - We may discover that humanity is not the center of the universe

32:54

Question - Dr Doolittle question - Will we be able to talk to the animals? - Wittgenstein said no - Human Umwelt is different from others - but it may very well happen

34:54

species have culture - Marine mammals enact behavior similar to humans

• Unknown unknowns will likely move to known unknowns and to some known knowns

36:29

citizen science bioacoustic projects - audio moth - sound invisible to humans - ultrasonic sound - intrasonic sound - example - Amazonian river turtles have been found to have hundreds of unique vocalizations to call their baby turtles to safety out in the ocean

41:56

ocean habitat for whales - they can communicate across the entire ocean of the earth - They tell of a story of a whale in Bermuda can communicate with a whale in Ireland

43:00

progress trap - AI for interspecies communications - examples - examples - poachers or eco tourism can misuse

44:08

progress trap - AI for interspecies communications - policy

45:16

whale protection technology - Kim Davies - University of New Brunswick - aquatic drones - drones triangulate whales - ships must not get near 1,000 km of whales to avoid collision - Canadian government fines are up to 250,000 dollars for violating

50:35

environmental regulation - overhaul for the next century - instead of - treatment, we now have the data tools for - prevention

56:40 - ecological relationship - pollinators and plants have co-evolved

1:00:26

AI for interspecies communication - example - human cultural evolution controlling evolution of life on earth

for - progress trap - AI -

article details - title - Hollow, world! (Part 1 of 5) - author - James Allen - date - 10 July, 2024 - publication - substack - self link - https://allenj.substack.com/p/hollow-world-part-1-of-5

summary James Allen provides an insightful description of ultra-anthropomorphic AI, AI that attempts to simulate an entire, whole human being.

In short, he points out the fundamental distinction between the real experience of another human being, and a simulation of one. In so doing, he gets to the heart of what it is to be human.

An AI is a simulation of a human being. No matter how realistic it's responses and actions, it is not evolved out of biology. I have no doubts that scientists are hard at work trying to make a biological AI. The distinction becomes fuzzier then.

Current AI cannot possibly simulate the experience of being in a fragile and mortal body and all that this entails. If an AI robot says it understands joy or pain, that statement isn't built on the combined exteroception and interoception of being in a biological body, rather, it is based on many linguistic statements it has assimilated.

for - progress trap - AI - threat of superintendence - interview - Leopold Aschenbrenner - former Open AI employee - from -. YouTube - review of Leopold Aschenbrenner's essay on Situational Awareness - https://hyp.is/ofu1EDC3Ee-YHqOyRrKvKg/docdrop.org/video/om5KAKSSpNg/

for - key insight - AI - progress trap - nightmare scenario - dictator controlling superintelligence

meet insight - AI - progress trap - nightmare scenario - locked in dictatorship controlling superintelligence - millions of AI controlled robotic law and enforcement agents could police their populace - Mass surveillance would be hypercharged - Dictator loyal AI agents could individually assess every single citizen for descent with near perfect lie detection sensor - rooting out any disloyalty e - Essentially - the robotic military and police force could be wholly controlled by a single political leader and - programmed to be perfectly obedient and there's going to be no risks of coups or rebellions and - his strategy is going to be perfect because he has super intelligence behind them - what does a look like when we have super intelligence in control by a dictator ? - there's simply no version of that where you escape literally - past dictatorships were not permanent but - superintelligence could eliminate any historical threat to a dictator's Rule and - lock in their power - If you believe in freedom and democracy this is an issue because - someone in power, - even if they're good - they could still stay in power - but you still need the freedom and democracy to be able to choose - This is why the Free World must Prevail so - there is so much at stake here that - This is why everyone is not taking this into account

for - progress trap - double bind - AI - ubiquity

progress trap - double bind - AI - ubiquity - Rationale: we will have to equip many systems with AI - including military systems - Already connected to the internet - AI will be embedded in every critical piece of infrastructure in the future - What happens if something goes wrong? - Now there is an alignment failure everywhere - We will potentially have superintelligence within 3 years - Alignment failures will become catastrophic with them

for - progress trap - AI - example - give prompt for AI to earn money

progress trap - AI - example - instruct AI to earn money - Getting a base model to make money. By default it may well learn - to lie - to commit fraud - to deceive - to hack - to seek power - because in the real world - people actually use this to make money - even maybe they'll learn to - behave nicely when humans are looking and then - pursue more nefarious strategies when we aren't watching

for - progress trap - AI - one nightmare scenario

progress trap - AI - one nightmare scenario - Whoever is the first to control superintelligence will possibly have enough power to - seize control from pre superintelligence forces - even without the robots small civilization of superintelligence would be able to - hack any undefended military election television system and cunningly persuade generals electoral and economically out compete nation states - design new synthetic bioweapons and then - pay a human in Bitcoin to synthetically synthesize it

for - progress trap - AI and even more powerful weapons of destruction

progress trap - AI and even more powerful weapons of destruction - The podcaster's excitement seems to overshadow any concern of the tragic unintended consequences of weapons even more powerful than nuclear warheads. - With human base emotions still stuck in the past and our species continued reliance on violence to solve problems, more powerful weapons is not the solution, - indeed, they only make the problem worse - Here is where Ronald Wright's quote is so apt: - We humans are running modern software on 50,000 year old hardware systems - Our cultural evolution, of which AI is a part of, is happening so quickly, that - it is racing ahead of our biological evolution - We aren't able to adapt fast enough for the rapid cultural changes that AI is going to create, and it may very well destroy us

for - progress trap - AI triggering massive economic growth - planetary boundaries

progress trap - AI triggering massive economic growth - planetary boundaries - The podcaster does not consider the ramifications of the potential disastrous impact of such economic growth if not managed properly

for - progress trap - AI and human enslavement?

progress trap - human enslavement? - Isn't what the speaker is talking about here is that - AI will be the masters and - humans will become slaves?

for - progress trap - debate - nobody is discussing the dangers of such a project!

progress trap - debate - nobody is discussing the dangers of such a project! - Civlization's journey has to create more and more powerful tools for human beings to use - but this tool is different because it can act autonomously - It can solve problems that will dwarf our individual or even group ability to solve - Philosophically, the problem / solution paradigm becomes a central question because, - As presented in Deep Humanity praxis, - humans have never stopped producing progress traps as shadow sides of technology because - the reductionist problem solving approach always reaches conclusions based on finite amount of knowledge of the relationships of any one particular area of focus - in contrast to the infinite, fractal relationships found at every scale of nature - Supercomputing can never bridge the gap between finite and infinite - A superintelligent artifact with that autonomy of pattern recognition may recognize a pattern in which humans are not efficient and in fact, greater efficiency gains can be had by eliminating us

for - progress trap - AI - milestone - automated AI researcher

progress trap - AI - milestone - automated AI researcher - This is a serious concern that must be debated - An AI researcher that does research on itself has no moral compass and can encode undecipherable code into future generations of AI that provides no back door to AI if something goes wrong. - For instance, if AI reached the conclusion that humans need to be eliminated in order to save the biosphere, - it can disseminate its strategies covertly under secret communications with unbreakable code

for - AI - progress trap - nice simple explanation of how progress traps propagate

for - progress trap - Generative AI - copyright infringement vs Unfair business practice argument

for - progress trap - AI music - oversaturated market

for - progress trap - AI music - dilution of general royalty pool - due to large volume

For - progress trap - AI - Generative AI - IP - Yale Law Journal

for - progress trap - AI music - critique - Folia Sound Studio - to - P2P Foundation - Michel Bauwens - Commons Transition Plan - Netarchical Capitalism - Predatory Capitalism

to - P2P Foundation - Michel Bauwens - Commons Transition Plan - Netarchical Capitalism - Predatory Capitalism https://hyp.is/o-Hp-DCAEe-8IYef613YKg/wiki.p2pfoundation.net/Commons_Transition_Plan

for - progress trap - AI

for - AI progress trap - bad actor - synthesize bioweapon - AI progress trap - Coscientist and cloud lab - bad actors

for - progress trap - sexually explicit AI deepfake - Taylor Swift - sexually explicit AI deepfake

• for: canonical unit, collaborative commons - missing part - open learning commons, question - process trap - natural capital

• comment

  • in this context, indyweb and Indranet are not the canonical unit, but then, it seems the model is fundamentally missing the functionality provided but the Indyweb and Indranet, which is and open learning system.
  • without such an open learning system that captures the essence of his humans learn, the activity of problem-solving cannot be properly contextualised, along with all of limitations leading to progress traps.
  • The entire approach of posing a problem, then solving it is inherently limited due to the fractal intertwingularity of reality.

• question: progress trap - natural capital

  • It is important to be aware that there is a real potential for a progress trap to emerge here, as any metric is liable to be abused

• annotate
• for: AI - progress trap, technology - colonialism

• for: example - progress trap - AI - mistranslation

• for: progress trap -AI, carbon footprint - AI, progress trap - AI - bias, progress trap - AI - situatedness

• for: quote - Yuval Noah Harari - AI progress trap, progress trap - AI, quote - progress trap

• quote it is extremely dangerous to create such an autonomous agent when we do not know how to control it, when we can't ensure that it will not escape our control ad start making decisions and creating new things which will harm us instead of benefit us

• author: Yuval Noah Harari
• date 2023

• for: example - stupid AI - nuclear launch, AI - progress trap - example - nuclear launch

• for: quote - Thomas Homer-Dixon, quote - danger of AI, AI progress trap

• quote: danger of AI

  • I think the most dangerous thing about AI is not super smart AI, it's stupid AI that is good enough to be put in charge of certain processes but not good enough to not make really bad mistakes
• author: Thomas Homer-Dixon
• date: 2021

• for: progress trap, progress trap - AI, progress trap - AI - writing research papers

• comment

  • potential fakes
    • climate science fakes by big oil think tanks
    • Covid and virus research
    • race issues
    • gender issues

• for: instant evolution, AI - instant evolution, progress trap, progress trap - AI - instant evolution, progress trap - robotics

• for: progress trap, progress trap - AI

• for: progress trap, AI, AI - care drive
• comment
  • the precautionary principle needs to be observed with AI because it has such vast artificial cognitive, pattern-recognition processes at its disposal
  • AI will also make mistakes, but the degree of power behind the mistaken decision, recommendation or action is the degree of unintended consequences or progress trap
  • An example nightmare scenario could be:
    • AI could decide that humans are contradicting their own goal of a stable climate system and if it's in control, may think it knows better and perform whole system change that dramatically reduces human induced climate change but actually harms a lot of humans in the process, for reaching the goal of saving the climate system plus a sufficient subset of humans to start all over.

• Title
  • One Billion Happy

• Author

  • Mo Gawdat

• Description

  • Mo Gawdat was former chief business officer at Google X, Google's innovation center.
  • Mo left Google after seeing the rapid pace of AI development was going to lead to a progress trap in which
    • the risk of AI destroying human civilization is becoming real because AI will be learning from too many unhappy people whose trauma AI will learn and incorporate into its algorithms
  • Hence, human happiness becomes paramount to prevent this catastrophe from happening
• See Ronald Wright's prescient quote
  • https://hyp.is/go?url=https%3A%2F%2Fwww.goodreads.com%2Fwork%2Fquotes%2F321797-a-short-history-of-progress&group=Q2X5j1pR

• Singularity
  • will happen within the next few decades

• It is the immoral human being that is the real problem
• they will teach AI to be immoral and with its power, can end up destroying humanity

• Title
  • Mo Gawdat Warns the Dangers of AI Are "Happening As We Speak"
• Author
  • Piers Morgan Uncensored

• "if we reverse this

  • if we have the best of us take charge
  • the best of us will tell AI
    • don't try to kill the the enemy,
      • try to reconcile with the enemy
    • don't try to create a competitive product
      • that allows me to lead with electric cars,
        • create something that helps all of us overcome global climate change
    • that's the interesting bit
      • the actual threat ahead of us is
        • not the machines at all
          • the machines are pure potential pure potential
        • the threat is how we're going to use them"

• comment

  • again, see Ronald Wright's quote above
  • it's very salient to this context

• comment
  • the machines are only coded to do what we tell them to do
  • Ronald' Wright's quote is very salient here
    • https://hyp.is/6Lb6Uv5NEe2ZerOrftOHfA/www.goodreads.com/work/quotes/321797-a-short-history-of-progress

• limited (machine) intelligence

  • cannot help but exist
  • if the original (human) authors of the AI code are themselves limited in their intelligence

• comment

  • this limitation is essentially what will result in AI progress traps
  • Indeed,
    • progress and their shadow artefacts,
    • progress traps,
    • is the proper framework to analyze the existential dilemma posed by AI

• Comment
  • Mary echos Ronald Wright's progress traps

Key finding Paraphrase - So what does a conscious universe have to do with AI and existential risk? - It all comes back to whether our primary orientation is around - quantity, or around - quality. - An understanding of reality - that recognises consciousness as fundamental - views the quality of your experience as - equal to, - or greater than, - what can be quantified.

• Orienting toward quality,
  • toward the experience of being alive,
• can radically change
  • how we build technology,
  • how we approach complex problems,
  • and how we treat one another.

Quote - metaphysics of quality - would open the door for ways of knowing made secondary by physicalism

Author - Robert Persig - Zen and the Art of Motorcycle Maintenance // - When we elevate the quality of each our experience - we elevate the life of each individual - and recognize each individual life as sacred - we each matter - The measurable is also the limited - whilst the immeasurable and directly felt is the infinite - Our finite world that all technology is built upon - is itself built on the raw material of the infinite

//

Title Reality Eats Culture For Breakfast: AI, Existential Risk and Ethical Tech Why calls for ethical technology are missing something crucial Author Alexander Beiner

Summary - Beiner unpacks the existential risk posed by AI - reflecting on recent calls by tech and AI thought leaders - to stop AI research and hold a moratorium.

• Beiner unpacks the risk from a philosophical perspective

  • that gets right to the deepest cultural assumptions that subsume modernity,
  • ideas that are deeply acculturated into the citizens of modernity.

• He argues convincingly that

  • the quandry we are in requires this level of re-assessment
    • of what it means to be human,
    • and that a change in our fundamental cultural story is needed to derisk AI.

= progress trap example

= Comment - this is worrying. - if the Chatbots perceive an enemy it to harm it, it could take haarmful actions against the perceived threat

= progress trap example - Bing ChatGPT - example of AI progress trap

• example of = AI progress trap
• Bing can be seen
  • insulting users,
  • lying to them,
  • sulking,
  • gaslighting
  • emotionally manipulating people,
  • questioning its own existence,
  • describing someone who found a way to force the bot to disclose its hidden rules as its “enemy,” and
  • claiming it spied on Microsoft’s own developers through the webcams on their laptops.