for - progress trap - AI - threat of superintendence - interview - Leopold Aschenbrenner - former Open AI employee - from -. YouTube - review of Leopold Aschenbrenner's essay on Situational Awareness - https://hyp.is/ofu1EDC3Ee-YHqOyRrKvKg/docdrop.org/video/om5KAKSSpNg/

for - key insight - AI - progress trap - nightmare scenario - dictator controlling superintelligence

meet insight - AI - progress trap - nightmare scenario - locked in dictatorship controlling superintelligence - millions of AI controlled robotic law and enforcement agents could police their populace - Mass surveillance would be hypercharged - Dictator loyal AI agents could individually assess every single citizen for descent with near perfect lie detection sensor - rooting out any disloyalty e - Essentially - the robotic military and police force could be wholly controlled by a single political leader and - programmed to be perfectly obedient and there's going to be no risks of coups or rebellions and - his strategy is going to be perfect because he has super intelligence behind them - what does a look like when we have super intelligence in control by a dictator ? - there's simply no version of that where you escape literally - past dictatorships were not permanent but - superintelligence could eliminate any historical threat to a dictator's Rule and - lock in their power - If you believe in freedom and democracy this is an issue because - someone in power, - even if they're good - they could still stay in power - but you still need the freedom and democracy to be able to choose - This is why the Free World must Prevail so - there is so much at stake here that - This is why everyone is not taking this into account

for - progress trap - double bind - AI - ubiquity

progress trap - double bind - AI - ubiquity - Rationale: we will have to equip many systems with AI - including military systems - Already connected to the internet - AI will be embedded in every critical piece of infrastructure in the future - What happens if something goes wrong? - Now there is an alignment failure everywhere - We will potentially have superintelligence within 3 years - Alignment failures will become catastrophic with them

for - progress trap - AI - example - give prompt for AI to earn money

progress trap - AI - example - instruct AI to earn money - Getting a base model to make money. By default it may well learn - to lie - to commit fraud - to deceive - to hack - to seek power - because in the real world - people actually use this to make money - even maybe they'll learn to - behave nicely when humans are looking and then - pursue more nefarious strategies when we aren't watching

for - progress trap - AI - one nightmare scenario

progress trap - AI - one nightmare scenario - Whoever is the first to control superintelligence will possibly have enough power to - seize control from pre superintelligence forces - even without the robots small civilization of superintelligence would be able to - hack any undefended military election television system and cunningly persuade generals electoral and economically out compete nation states - design new synthetic bioweapons and then - pay a human in Bitcoin to synthetically synthesize it

for - progress trap - AI and even more powerful weapons of destruction

progress trap - AI and even more powerful weapons of destruction - The podcaster's excitement seems to overshadow any concern of the tragic unintended consequences of weapons even more powerful than nuclear warheads. - With human base emotions still stuck in the past and our species continued reliance on violence to solve problems, more powerful weapons is not the solution, - indeed, they only make the problem worse - Here is where Ronald Wright's quote is so apt: - We humans are running modern software on 50,000 year old hardware systems - Our cultural evolution, of which AI is a part of, is happening so quickly, that - it is racing ahead of our biological evolution - We aren't able to adapt fast enough for the rapid cultural changes that AI is going to create, and it may very well destroy us

for - progress trap - AI triggering massive economic growth - planetary boundaries

progress trap - AI triggering massive economic growth - planetary boundaries - The podcaster does not consider the ramifications of the potential disastrous impact of such economic growth if not managed properly

for - progress trap - AI and human enslavement?

progress trap - human enslavement? - Isn't what the speaker is talking about here is that - AI will be the masters and - humans will become slaves?

for - progress trap - debate - nobody is discussing the dangers of such a project!

progress trap - debate - nobody is discussing the dangers of such a project! - Civlization's journey has to create more and more powerful tools for human beings to use - but this tool is different because it can act autonomously - It can solve problems that will dwarf our individual or even group ability to solve - Philosophically, the problem / solution paradigm becomes a central question because, - As presented in Deep Humanity praxis, - humans have never stopped producing progress traps as shadow sides of technology because - the reductionist problem solving approach always reaches conclusions based on finite amount of knowledge of the relationships of any one particular area of focus - in contrast to the infinite, fractal relationships found at every scale of nature - Supercomputing can never bridge the gap between finite and infinite - A superintelligent artifact with that autonomy of pattern recognition may recognize a pattern in which humans are not efficient and in fact, greater efficiency gains can be had by eliminating us

for - progress trap - AI - milestone - automated AI researcher

progress trap - AI - milestone - automated AI researcher - This is a serious concern that must be debated - An AI researcher that does research on itself has no moral compass and can encode undecipherable code into future generations of AI that provides no back door to AI if something goes wrong. - For instance, if AI reached the conclusion that humans need to be eliminated in order to save the biosphere, - it can disseminate its strategies covertly under secret communications with unbreakable code

for - AI - progress trap - nice simple explanation of how progress traps propagate

for - progress trap - Generative AI - copyright infringement vs Unfair business practice argument

for - progress trap - AI music - oversaturated market

for - progress trap - AI music - dilution of general royalty pool - due to large volume

For - progress trap - AI - Generative AI - IP - Yale Law Journal

for - progress trap - AI music - critique - Folia Sound Studio - to - P2P Foundation - Michel Bauwens - Commons Transition Plan - Netarchical Capitalism - Predatory Capitalism

to - P2P Foundation - Michel Bauwens - Commons Transition Plan - Netarchical Capitalism - Predatory Capitalism https://hyp.is/o-Hp-DCAEe-8IYef613YKg/wiki.p2pfoundation.net/Commons_Transition_Plan

for - progress trap - AI

for - AI progress trap - bad actor - synthesize bioweapon - AI progress trap - Coscientist and cloud lab - bad actors

for - progress trap - sexually explicit AI deepfake - Taylor Swift - sexually explicit AI deepfake

• for: canonical unit, collaborative commons - missing part - open learning commons, question - process trap - natural capital

• comment

  • in this context, indyweb and Indranet are not the canonical unit, but then, it seems the model is fundamentally missing the functionality provided but the Indyweb and Indranet, which is and open learning system.
  • without such an open learning system that captures the essence of his humans learn, the activity of problem-solving cannot be properly contextualised, along with all of limitations leading to progress traps.
  • The entire approach of posing a problem, then solving it is inherently limited due to the fractal intertwingularity of reality.

• question: progress trap - natural capital

  • It is important to be aware that there is a real potential for a progress trap to emerge here, as any metric is liable to be abused

• annotate
• for: AI - progress trap, technology - colonialism

• for: example - progress trap - AI - mistranslation

• for: progress trap -AI, carbon footprint - AI, progress trap - AI - bias, progress trap - AI - situatedness

• for: quote - Yuval Noah Harari - AI progress trap, progress trap - AI, quote - progress trap

• quote it is extremely dangerous to create such an autonomous agent when we do not know how to control it, when we can't ensure that it will not escape our control ad start making decisions and creating new things which will harm us instead of benefit us

• author: Yuval Noah Harari
• date 2023

• for: example - stupid AI - nuclear launch, AI - progress trap - example - nuclear launch

• for: quote - Thomas Homer-Dixon, quote - danger of AI, AI progress trap

• quote: danger of AI

  • I think the most dangerous thing about AI is not super smart AI, it's stupid AI that is good enough to be put in charge of certain processes but not good enough to not make really bad mistakes
• author: Thomas Homer-Dixon
• date: 2021

• for: progress trap, progress trap - AI, progress trap - AI - writing research papers

• comment

  • potential fakes
    • climate science fakes by big oil think tanks
    • Covid and virus research
    • race issues
    • gender issues

• for: instant evolution, AI - instant evolution, progress trap, progress trap - AI - instant evolution, progress trap - robotics

• for: progress trap, progress trap - AI

• for: progress trap, AI, AI - care drive
• comment
  • the precautionary principle needs to be observed with AI because it has such vast artificial cognitive, pattern-recognition processes at its disposal
  • AI will also make mistakes, but the degree of power behind the mistaken decision, recommendation or action is the degree of unintended consequences or progress trap
  • An example nightmare scenario could be:
    • AI could decide that humans are contradicting their own goal of a stable climate system and if it's in control, may think it knows better and perform whole system change that dramatically reduces human induced climate change but actually harms a lot of humans in the process, for reaching the goal of saving the climate system plus a sufficient subset of humans to start all over.

• Title
  • One Billion Happy

• Author

  • Mo Gawdat

• Description

  • Mo Gawdat was former chief business officer at Google X, Google's innovation center.
  • Mo left Google after seeing the rapid pace of AI development was going to lead to a progress trap in which
    • the risk of AI destroying human civilization is becoming real because AI will be learning from too many unhappy people whose trauma AI will learn and incorporate into its algorithms
  • Hence, human happiness becomes paramount to prevent this catastrophe from happening
• See Ronald Wright's prescient quote
  • https://hyp.is/go?url=https%3A%2F%2Fwww.goodreads.com%2Fwork%2Fquotes%2F321797-a-short-history-of-progress&group=Q2X5j1pR

• Singularity
  • will happen within the next few decades

• It is the immoral human being that is the real problem
• they will teach AI to be immoral and with its power, can end up destroying humanity

• Title
  • Mo Gawdat Warns the Dangers of AI Are "Happening As We Speak"
• Author
  • Piers Morgan Uncensored

• "if we reverse this

  • if we have the best of us take charge
  • the best of us will tell AI
    • don't try to kill the the enemy,
      • try to reconcile with the enemy
    • don't try to create a competitive product
      • that allows me to lead with electric cars,
        • create something that helps all of us overcome global climate change
    • that's the interesting bit
      • the actual threat ahead of us is
        • not the machines at all
          • the machines are pure potential pure potential
        • the threat is how we're going to use them"

• comment

  • again, see Ronald Wright's quote above
  • it's very salient to this context

• comment
  • the machines are only coded to do what we tell them to do
  • Ronald' Wright's quote is very salient here
    • https://hyp.is/6Lb6Uv5NEe2ZerOrftOHfA/www.goodreads.com/work/quotes/321797-a-short-history-of-progress

• limited (machine) intelligence

  • cannot help but exist
  • if the original (human) authors of the AI code are themselves limited in their intelligence

• comment

  • this limitation is essentially what will result in AI progress traps
  • Indeed,
    • progress and their shadow artefacts,
    • progress traps,
    • is the proper framework to analyze the existential dilemma posed by AI

• Comment
  • Mary echos Ronald Wright's progress traps

Key finding Paraphrase - So what does a conscious universe have to do with AI and existential risk? - It all comes back to whether our primary orientation is around - quantity, or around - quality. - An understanding of reality - that recognises consciousness as fundamental - views the quality of your experience as - equal to, - or greater than, - what can be quantified.

• Orienting toward quality,
  • toward the experience of being alive,
• can radically change
  • how we build technology,
  • how we approach complex problems,
  • and how we treat one another.

Quote - metaphysics of quality - would open the door for ways of knowing made secondary by physicalism

Author - Robert Persig - Zen and the Art of Motorcycle Maintenance // - When we elevate the quality of each our experience - we elevate the life of each individual - and recognize each individual life as sacred - we each matter - The measurable is also the limited - whilst the immeasurable and directly felt is the infinite - Our finite world that all technology is built upon - is itself built on the raw material of the infinite

//

Title Reality Eats Culture For Breakfast: AI, Existential Risk and Ethical Tech Why calls for ethical technology are missing something crucial Author Alexander Beiner

Summary - Beiner unpacks the existential risk posed by AI - reflecting on recent calls by tech and AI thought leaders - to stop AI research and hold a moratorium.

• Beiner unpacks the risk from a philosophical perspective

  • that gets right to the deepest cultural assumptions that subsume modernity,
  • ideas that are deeply acculturated into the citizens of modernity.

• He argues convincingly that

  • the quandry we are in requires this level of re-assessment
    • of what it means to be human,
    • and that a change in our fundamental cultural story is needed to derisk AI.

= progress trap example

= Comment - this is worrying. - if the Chatbots perceive an enemy it to harm it, it could take haarmful actions against the perceived threat

= progress trap example - Bing ChatGPT - example of AI progress trap

• example of = AI progress trap
• Bing can be seen
  • insulting users,
  • lying to them,
  • sulking,
  • gaslighting
  • emotionally manipulating people,
  • questioning its own existence,
  • describing someone who found a way to force the bot to disclose its hidden rules as its “enemy,” and
  • claiming it spied on Microsoft’s own developers through the webcams on their laptops.