147 Matching Annotations
  1. Mar 2021
  2. Dec 2020
    1. From the principles outlined in the work of Sinclair and his associates, two basic tenets are central to the system of the analysis presented in this book. First, fol-lowing Sinclair (1981/2004c; 1985/2004d) discourse can best be described as a dynamic process in which the reader/listener is continually processing incoming language in a linear fashion. The second tenet is that fundamentally the same syn-tagmatic mechanisms are present in both grammar and discourse and therefore a unified system of analysis can be used simultaneously in the description of both. In grammar, the text of the moment (Sinclair 1993/2004e: 82) is the element, in discourse it is the linear unit. The text of the moment, following Sinclair, is con-ceived as being the focus of attention for the reader/listener at that moment and changes from one unit of analysis to the next in a linear fashion. Fundamentally, the researcher analyzes the present unit for analysis as if s/he has read everything in the discourse up to that point but has not read beyond it. The researcher then tries to make a connection, if any, with the previous unit and decides if the pres-ent text of the moment prospects anything in the upcoming unit. Thus, this in situperspective of the researcher attempts to recreate the dynamism of the interaction of the reader as s/he proceeds through the text. This inevitably makes for a linear analysis as the researcher/reader is limited to relating the text of the moment to its immediate surroundings. What becomes paramount in such analysis are consid-erations such as what will appear in linear sequence in the text immediately after the text of the moment, whether this is signalled or predicted in any way and how the text of the moment relates to both the text immediately preceding it

      Principles of Linear Unit Grammar

    1. First,we borrow heavily from Shannon: every process is a communicationchannel. In particular, we posit that any system is a channel thatcommunicates its past to its future through its present. Second, wetake into account the context of interpretation. We view buildingmodels as akin to decrypting nature’s secrets. How do we cometo understand a system’s randomness and organization, given onlythe available, indirect measurements that an instrument provides?To answer this, we borrow again from Shannon, viewing modelbuilding also in terms of a channel: one experimentalist attemptsto explain her results to another.

      Information from systems

    1. There is a considerable computational and representational advan-tage to reason under the CWA since negative information should beinferred by default [35]. The Careful CWA (CCWA) is an extensionof the CWA [15]. It allows us to restrict the effects of closing theworld by specifying the predicates which may be affected by theCWA rule in indefinite databases.

      CWA = Closed world assumption

  3. Nov 2020
    1. The Argument From Slavic Pessimism We can't build anything right. We can't even build a secure webcam. So how are we supposed to solve ethics and code a moral fixed point for a recursively self-improving intelligence without fucking it up, in a situation where the proponents argue we only get one chance? Consider the recent experience with Ethereum, an attempt to codify contract law into software code, where a design flaw was immediately exploited to drain tens of millions of dollars. Time has shown that even code that has been heavily audited and used for years can harbor crippling errors. The idea that we can securely design the most complex system ever built, and have it remain secure through thousands of rounds of recursive self-modification, does not match our experience.

      Not a strong argument if you believe recursive self-improvement is possible

    2. The Argument From Stephen Hawking's Cat Stephen Hawking is one of the most brilliant people alive, but say he wants to get his cat into the cat carrier. How's he going to do it? He can model the cat's behavior in his mind and figure out ways to persuade it. He knows a lot about feline behavior. But ultimately, if the cat doesn't want to get in the carrier, there's nothing Hawking can do about it despite his overpowering advantage in intelligence. Even if he devoted his career to feline motivation and behavior, rather than theoretical physics, he still couldn't talk the cat into it. You might think I'm being offensive or cheating because Stephen Hawking is disabled. But an artificial intelligence would also initially not be embodied, it would be sitting on a server somewhere, lacking agency in the world. It would have to talk to people to get what it wants. With a big enough gap in intelligence, there's no guarantee that an entity would be able to "think like a human" any more than we can "think like a cat".

      Not persuasive, because people are stupid, and APIs are a thing

    1. Digital storage of biological information further improves the possibilities for fidelity of replication,since digital replication is essentially error free. Technical advances in DNA sequencing and synthesis mean that information originally encoded as DNA sequences can now be stored digitally for extended periods, with the potential for artificial re-synthesis of organisms at a later date [35]. This has already been achieved for bacteria [36], and syntheses of eukaryotic genomes are under way [37]. Re-synthesis of multicellular organisms will require greater understanding of developmental programming and epigenetics, but in principle, these technological hurdles are not insurmountable. The replication of many thousands of digital genome sequences representing diverse species now occurs with virtually perfect fidelity at multiple nodes across the Internet. Continued accumulation of polished genome sequences will eventually result in alibrary of all the information required to reconstruct a significant proportion of current biodiversity [38]. Storing this information indefinitely on solid media could be done withoutsignificant energy cost.

      Resynthesis of organisms

    1. microtheories. Many microtheories are defined in considerableinternal detail. In the spatial area, there are microtheories for ‘naive geometry’ and ‘naivephysics’, as well as a microtheory called ‘naive spatial’, intended to ‘to represent the natu-ral way we reason about spatial relations’ (Cycorp 2004, NaiveSpatialMt). The basicentity within Cyc defined for space is SpatialThing, possessing, for better or worse, anextremely diverse collection of subtypes. The overall treatment of space in Cyc is wellbeyond the scope of the current article; however, more information on the connectionbetween Cyc and qualitative spatial reasoning is given by Grenon (2003), while a generaloverview of Cyc’s treatment of space can be found in Bateman & Farrar (2004a).

      Space in Cyc

    Annotators

    1. Ultimately, a full specification of the ontology should precisely define the collection of object-regions to be used; but some aspects of this definition are rather arbitrary and we will not try toresolve them here. For instance: do we want to assume that all regions are polyhedra, or that allregions are smooth, or do we want an ontology that includes both smooth regions and polyhedra?The advantage of including polyhedra is that rectangular boxes and so on are useful entities withnice physical properties; and artifacts that are actually polyhedral to a very high precision are quitecommon. The drawbacks of including polyhedral regions are that some aspects of physics are definedin terms of the normal to the surface, and for non-smooth objects, since there is no unique normal,one has to work around this (which can be done, but it takes work); and that most though notall ostensibly polyhedral objects actually have visibly curved edges if you look closely enough. Theadvantages of restricting the class of regions to polyhedral objects are, first, that polyhedra supportcomparatively efficient algorithms when there are exact specifications; and, second, that the class ofpolyhedra has nice closure properties e.g. it is closed under union. The advantage of allowing non-polyhedral objects is that sphere, ovoids, helical screws and so on are also useful physical entities;they can be manufactured or arise in nature to very high precision; and it is a substantial nuisanceto have a theory that posits that only high-precision polyhedral approximations for these exist. Wewill not here attempt to decide this

      Object-regions

    2. Perhaps a robot should be able to think about space in a number of different ways correspondingto different tasks or circumstances. For instance, it could model space as three-dimensional whengrating cheese; as the two-dimensional surface of a sphere when thinking about geography; andas a network of points and edges when planning a route through the subway. There is, in factconsiderable psychological evidence that suggest that people use multiple spatial models in this way.In particular, people are often extremely poor at geometric reasoning in three-dimensions — forinstance, at visualizing the three dimensional rotation of an object of any complexity — and preferto project down to two dimensions, if that is in any way an option.

      Thinking space in multiple models at once

    3. From thestandpoint of software engineering, there are good reasons for having a separate spatial reasoningcomponent with abstract spatial objects; these are, in fact, much the same reasons that geometry,abstracted from its specific applications, became a separate discipline 2500 years ago.•There is generally a clear distinction between purely geometric reasoning and reasoning thatinvolves other issues. (As a point of contrast, it can be very difficult to draw a line betweenfolk psychology and folk sociology.)•The validity of a geometric inference is essentially independent of the application, though theimportance of one or another type of geometric inference depends strongly on the application.•There is a largely self-contained body of techniques used for geometric calculation and geo-metric inference which are not applicable to other subject matters.Therefore placing geometric reasoning in a separate component with abstract geometric enti-ties serves the software engineering goal of modularization; this significantly reduces the risk ofunnecessary repetition of code, of inconsistencies, and of errors.

      Why it's worth having a dedicated spacial ontology

    4. Many enthusiasts for this kind of approachbelieve that it will suffice to achieve all of AI. If so, then the answer to the question in the chaptertitle would be “There is no need for a robot to think about space,” or at least, “There is no needfor us to think about how a robot should think about space,” or indeed any other abstract concept.This approach has a number of points in its favor. First, it passes the buck; rather than worryabout what to do with space, we can simply rely on a machine learning facility which we willneed anyway for other purposes. Second, it will presumably by design tend to home in on spatialproperties that are important to the robot and ignore those that are not, which can be a difficultproblem for hand-designed ontologies.Third, there is a strong argument to be made that this must in principle be possible. After all,any concept that a person knows was acquired in one of three ways: he learned it himself fromexperience, or he was, explicitly or implicitly, taught it from a store of cultural capital, or it wasinnate. In each of these cases, the concept was learned from experience at some point at some level;either the individual learned it, or the culture learned it, or the species or one of its ancestors learnedit through evolution. In any case, the concept must be learnable.

      Steel-manning the idea that all AI can be achieved with end-to-end machine learning without ontologies

    5. This definition of ontology gives a partial answer to what is known as the “symbol grounding”problem: What determines the meaning of a symbol in a computer program? On this view ofontology, a symbol means what the programmer intends that it should mean. That does not solvethe general symbol grounding problem because it does not give any account of symbols that theprogram generates on the fly but, as far as it goes, it seems reasonable.

      Symbol grounding with ontologies

    6. If a programmer is writing a “physics engine” to simulate interactionsamong rigid solid objects, she will probably begin by choose some specific idealized model of solidobject physics: e.g. perfect rigidity, uniform density, Newtonian mechanics, Coulomb friction, in-elastic collisions. This, then, is the ontology

      Engines are ontologies

    1. An important feature of nineteenth century manufacturing technologies isthat they were largely “deskilling” –i.e.they substituted for skills through thesimplification of tasks (Braverman, 1974; Hounshell, 1985;James and Skinner,1985; Goldin and Katz, 1998). The deskilling process occurred as the factorysystem began to displace the artisan shop, and it picked up pace as produc-tion increasingly mechanized with the adoption of steam power (Goldin andSokoloff, 1982; Atack,et al., 2008a). Work that had previously been performedby artisans was now decomposed into smaller, highly specialised, sequences,requiring less skill, but more workers, to perform.9Some innovations wereeven designed to be deskilling. For example, Eli Whitney, a pioneer of inter-changeable parts, described the objective of this technology as “to substitutecorrect and effective operations of machinery for the skillof the artist which isacquired only by long practice and experience; a species of skill which is notpossessed in this country to any considerable extent” (Habakkuk, 1962, p. 22).

      deskilling

    2. Semi-nal work by Autor,et al.(2003), for example, distinguishes between cognitiveand manual tasks on the one hand, and routine and non-routinetasks on theother.

      Autor's 2x2

    3. The title “Lousy andLovely Jobs”, of recent work by Goos and Manning (2007), thuscaptures theessence of the current trend towards labour market polarization, with growingemployment in high-income cognitive jobs and low-income manual occupa-tions, accompanied by a hollowing-out of middle-income routine jobs.

      Labor market polarization

    4. More recently, the poor performance of labour markets across advancedeconomies has intensified the debate about technological unemployment amongeconomists. While there is ongoing disagreement about the driving forcesbehind the persistently high unemployment rates, a number of scholars havepointed at computer-controlled equipment as a possible explanation for recentjobless growth (see, for example, Brynjolfsson and McAfee,2011).

      Debate about role of technology in poor performance of labor markets

    1. A neuron that is not part of the currently regarded cortex fi eld is called a register (Dörner, 2002, p. 71). Neural programs are chains of registers that call associators, dissociators, activators, and inhibitors. (These “calls” are just activations of the respective elements.) In the course of neural execution, elements in the cortex fi eld are summarily linked to specifi c registers that are part of the executed chain of neurons. Then operations are performed on them, before they are unlinked again (Dörner, 2002, pp. 42, 70).

      "Register" neurons in Psi

    2. There are four types of neurons: activating, inhibitive, associative, and dissociative. While the latter two types only play a role in creating, changing and removing temporary links, the former are used to calculate the activation of their successor.

      Four types of Psi neurons

    3. Perceptions derived from the environment and the actions that have been performed on it become part of the agent’s situation image, a description of the present situation. This situation image is the head of a protocol chain that holds the agent’s past. The strength of the links in the chain depends on the motivational relevance of the events in the current situation: whenever an appetitive goal is fulfi lled (i.e., a demand is satis-fi ed), or an aversive event ensues and leads to a sudden rise of a demand, the links of the current situation to its immediate past are strengthened, so that relevant situations become associated both to the demand and to the sequence of events that lead to the fulfi llment of the demand.

      PSI "situation image"

    4. The next step may consist of de-coupling the feedback-loops from the sensors and introducing switches, which depend on internal state sen-sors. For instance, if the internal state sensors signal a lack of fuel, then the switch for the fuel-seeking behavior is turned on. If there is a lack of water, then the system might override the fuel-seeking behavior and turn on the water-seeking feedback loop. And if the system has gotten too wet, it might inhibit the water-seeking behavior and switch on a water-avoiding behavior. At all times, it is crucial to maintain a homeostasis, a dynamic balance of some control values in the face of the disturbances created by changes in the environment and by the agent’s actions.

      Braitenberg vehicles enhanced with switches

    5. Dörner introduces his theory in Bauplan für eine Seele incrementally. He starts out with a very simple system: a Braitenberg vehicle. (Braitenberg, 1984) In its basic form, it consists of a robot with locomotive abilities (two independently driven wheels) and a pair of light receptive sensors (see Figure 2.2). Each sensor controls a wheel: if the sensor gets a stron-ger signal, it speeds up the respective engine. If the sensors are crosswise-connected, then the sensor closer to the light source will give its stronger signal to the more distant wheel, and consequently, the vehicle will turn towards the light source.Conversely, if the sensors are wired in parallel, then the sensor closer to the light source will give its stronger signal to the closer wheel, thus turning the vehicle away. Of course, the sensors do not need to be light-receptive—they could react to humidity, to sound, or to the smell of fuel. With these simple mechanisms, and using multiple sets of sensors, it is possible to build a system that shows simple behaviors such as seek-ing out certain targets and avoiding others.

      Baisc Braitenberg vehicles

    6. PSI agents are usually little virtual steam vehicles that depend on fuel and water for their survival. When they are instantiated into their envi-ronment, they have no knowledge of how to attain these needs—they do not even know about the needs. All they have is the simulacrum of a body that is endowed with external sensors for environmental fea-tures and internal sensors for its physiological and cognitive demands.

      PSI agents starting point

    7. The PSI theory, which is the brain-child of the German psychologist Dietrich Dörner, is an attempt at representing the mind as a specifi c kind of machine, much in the same way that physics represents the uni-verse as a kind of machine. Here, a machine amounts to a (possibly very large, but not infi nite) set of if-then statements. Such a description is Dörner’s requirement to psychology, as long as it wants to be treated as a (natural) science, and, of course, it does not ask anyone to abstain from recognizing the mind as adaptive and creative, or to neglect the reality of phenomenal experience.

      PSI Theory

    8. Slipnets (Hofstadter & Mitchell, 1994) allow analogical reasoning and metaphor fi nding by organizing knowledge into hierarchies accord-ing to the available operators that allow transition between knowledge states, and then allowing “slippages” on the individual levels.

      Slipnets

    9. The Cognition and Affect Project (CogAff: Sloman, Chrisley, & Scheutz, 2005) is not an implementation, but a conceptual analysis for cognitive architectures in general. It provides a framework and a termi-nology to discuss existing architectures and defi ne the demands of broad models of cognition. CogAff is not restricted to descriptions of human cognition—this is regarded as a special case (H-CogAff ).CogAff is inspired by the idea that a model of the mind should not be restricted to the execution of unit tasks, deliberative acts, and complex operations (see Newell’s suggestion of the cognitive band in Table 1.1), but may also include the understanding of the exchange between refl exes, deliberation, and day-to-day refl ection and planning. Also, it should not restrict its explanations to a single dimension, such as the level of com-plexity of a cognitive task, but offer a perspective that includes stages and types of cognitive phenomena.

      CogAff

    10. The classical example for agents without symbolic representations is Rodney Brooks’ subsumption architecture (Brooks, 1986). A sub-sumption agent is a layered collection of modules (usually, fi nite state machines) that may be connected to sensors, actuators or other mod-ules. Simple behaviors may be implemented by the interaction of the modules on a low level, and more complex behaviors are the result of the mediation of low-level behavior by modules on a higher level. Subsumption agents do not have a world model and no capabilities for deliberative processing. There is no central control (although Brooks, 1991, later introduced a “hormonal activation” that provides a mode of distributed control of all modules that have a “receptor” for the respec-tive “hormone”).

      Brooks' "subsumption architecture"

    11. Furthermore, the agent will need to have information about prefer-ences among the states it can achieve to defi ne objectives. These objec-tives are very much like goals in the rule-based systems, but they might contradict each other, and be subject to constant changes. The objectives are called desires and defi ne a motivational component. Eventually, the agent will have to pick an objective and commit itself to following it with some persistence; otherwise, there would be a need for continuous re-planning and reconsideration of all possible courses of action, which is usually not possible in a dynamic and complex domain. These commit-ments are called intentions.

      "Intentions" in BDI

    12. Belief-Desire-Intention (BDI) systems are not so much a model of human cognition but an engineering stance and a terminological framework. When designing an autonomous agent, the need arises to equip it with some data structure that represents the state of its environment. These pieces of knowledge are usually acquired by some perceptual mecha-nism or inferred from previous states, and so they might misrepresent the environment; they are called beliefs. (Rao & Georgeff, 1995, charac-terize beliefs as something that provides information about the state of the system.)

      Belief-Desire-Intention system (BDI)

    13. Also, the actions of the agent may leave permanent traces, or different instances of interactions may be independent from each other (episodic).

      Notion of "episodic" interactions

    14. Usually, the model is depicted as a system that receives input, processes it according to an organizational principle or a predefi ned goal and generates an output, as opposed to an autonomous entity embedded in an environment that it may infl uence and change according to its needs. Such a paradigm, the autonomous agent, was introduced in artifi cial intelligence in the 1980s. “An autonomous agent is a system situated within and part of an environment that senses that envi-ronment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” (Franklin & Graesser, 1996)

      The agent paradigm

    15. The “neat and scruffy“ distinction has been described by Robert Abelson (1981), according to whom it goes back to Roger Schank. It alludes to two different families of AI models: those favoring clean, orderly structures with nicely provable properties, and those that let the ghosts of fuzziness, distributedness, and recurrency out of their respec-tive bottles. While the term “New AI” is sometimes used to refer to fuzziness19, AI does not really consist of an “old,” neat phase, and a “new,” scruffy era. Even in the 1960s and 1970s, people were designing logic-based systems and theorem provers (for instance, McCarthy & Hayes, 1969; Nilsson, 1971), and at the same time, others argued for their inad-equacy (e.g., Minsky & Papert, 1967), suggesting less general-purpose approaches and the use of distributed systems with specifi c functional-ity instead.

      "Neat/scruffy" distinction

    16. Despite the benefi ts of symbolic architectures and their semi-symbolic extensions, there are some criticisms. Symbolic cognitive architectures might be just too neat to depict what they are meant to model, their simple and straightforward formalisms might not be suited to capture the scruffi ness of a real-world environment and real-world problem solving. For example, while the discrete representations of Soar and ACT are well suited to describe objects and cognitive algorithms for mental arithmetic, they might run into diffi culties when object hier-archies are ambiguous and circular, perceptual data is noisy, goals are not well-defi ned, categories are vague, and so on. In domains where the extraction of suitable rules is practically infeasible, neural learning methods and distributed representations may be the method of choice, and while this is often refl ected in the perceptual and motor modules of symbolic architectures, it is not always clear if their application should end there.

      The "neat"-ness of symbolic architectures are both main pro and main con

    17. Hybrid archictures, which combine symbolic with sub-symbolic rea-soning using different modules or layers, are exemplifi ed in Ron Sun’s architecture Clarion (Sun, 2005, 2003). Clarion stands for Connectionist Learning with Adoptive Rule Indication On-Line. Its representations are based on Ron Sun’s work on CONSYDERR (1993) that models categorical inheritance using a two-layered connectionist system, whereby one layer is distributed, the other localist. Memory in Clarion likewise consists of a localist, rule-based layer that encodes explicit, symbolic knowledge, and an underlying distributed layer with implicit, sub-symbolic repre-sentations. Knowledge can be translated between the layers, by translat-ing symbolic rules into a sub-symbolic representation, or by extracting rules from the sub-symbolic layer. (Clarion is also notable for providing a motivational system, based on a set of drives.)

      Clarion (hybrid architecture)

    18. Using chunks and productions, ACT-R can encode temporal strings (which are somewhat like scripts, see Schank & Abelsson, 1977), spa-tial images (similar to schemas; Minsky, 1975) and abstract propositions. The activity of the system is determined by a probabilistic, goal-oriented matching process of productions, which leads to the acquisition of new procedures (productions) and the manipulation of declarative knowledge.

      ACT-R 'temporal strings' ~ scripts, 'spatial images' ~schemas, 'abstract propositions'

    19. In addition to the declarative memory, ACT-R proposes a procedural memory. Such a distinction has, for instance, been suggested by Squire (1994), but is far from being undisputed in the literature of psychology (Müller, 1993). Procedural memory consists of production rules, which coordinate the cognitive behavior using a goal stack that is laid out in working memory (see Figure 1.1).

      ACT-R procedural memory

    20. Many of the criticisms that apply to Soar have later been addressed by John Anderson‘s ACT theory17 (Anderson, 1983, 1990; Anderson & Lebiere, 1998). ACT is—next to Soar—currently the most extensively covered and applied model in the fi eld of symbolic cognitive architec-tures, and probably the one best grounded in experimental psychological research literature (Morrison, 2003, p. 30). Just as Soar, ACT-R is based on production rules, but unlike Soar, it allows for real-valued activa-tions (instead of a binary on-off ), which are biologically more plausible

      ACT theory / ACT-R as an alternative to Soar

    21. The actual problem solving work in Soar is delivered by the operators. Operators are algorithms that describe how to reach the next state; they are executed upon the fi lling of the context slots of a problem space. Soar can develop new operators on its own, but its models typically work with a set of predefi ned operators (often augmented with a library of about 50 default rules for planning and search, including means-end analysis, hill-climbing, alpha-beta search, branch and bound); the system may learn which one to apply in a given context. This represents a consider-able extension over Newell’s and Simon’s earlier attempt at a universal problem-solving mechanism, the General Problem Solver (1961), which did, among many other restrictions, only have a single problem space and two operators: means-end analysis and sub-goaling to fi nd a new operator. Also, it lacked the impasse mechanism to recognize missing knowledge (see also Newell, 1992).

      Operators in Soar -- build on Newell & Simon's earlier General Problem Solver

    22. Central to Soar is the notion of Problem Spaces. According to Newell, human rational action can be described bya set of knowledge states —operators for state transitions —constraints for the application of operators —control knowledge about the next applicable operator. —Consequently, a problem space consists of a set of states (with a dedi-cated start state and fi nal state) and operators over these states. Any task is represented as a collection of problem spaces. Initially, a problem space is selected, and then a start state within this problem space. The goal is the fi nal state of that problem space. During execution, state transitions are followed through until the goal state is reached or it is unclear how to proceed. In that case, Soar reaches an impasse. An impasse creates and selects a new problem space, which has the resolution of the impasse as its goal. The initial problem spaces are predefi ned by the modeler.

      "Problem Spaces" in Soar

    23. Alan Newell has set out to fi nd an architecture that—while being as simple as possible—is still able to fulfi ll the tasks of the cognitive level, a minimally complex architecture for general intelligence (i.e., with the smallest possible set of orthogonal mechanisms). To reproduce results from experimental psychology, so-called regularities (covering all con-ceivable domains, be it chess-playing, language, memory tasks, and even skiing), algorithms would be implemented within these organiza-tional principles. Newell’s architecture (Newell, 1990; Laird, Newell, & Rosenbloom, 1987) is called Soar (originally an acronym that stood for State, Operator and Result) and originated in his conceptions of human problem solving (Newell 1968; Newell & Simon, 1972). Soar embodies three principles: heuristic search for the solution of problems with little knowledge, a procedural method for routine tasks, and a symbolic the-ory for bottom-up learning, implementing the Power Law of Learning(Laird, Newell, & Rosenbloom, 1986).

      Motivations of Soar

    24. According to Newell (1990), cognitive acts span action coordination, deliberation, basic reasoning and immediate decision-making—those mental operations of an individual that take place in the order of hun-dreds of milliseconds to several seconds

      Level of abstraction of symbolic cognitive acts

    25. The advantages of a symbolic architecture are obvious: because a large part of human knowledge is symbolic, it may easily be encoded (Lenat, 1990); reasoning in symbolic languages allows for some straightforward conceptualizations of human reasoning, and a symbolic architecture can easily be made computation complete (i.e., Turing computational: Turing 1936).

      Obvious advantages of symbolic architecture

    26. To function, a symbol system has to observe some basic require-ments: it needs suffi cient memory, and it has to realize composability and interpretability. The fi rst condition, composability, specifi es that the operators have to allow the composition of any symbol structure, and interpretability asks that symbol structures can encode any valid arrangement of operators.A fi xed structure that implements such a symbol system is called a symbolic architecture. The behavior of this structure (that is, the pro-gram) only depends on the properties of the symbols, operators and interpretations, not on the actual implementation; it is independent of the physical substrate of the computational mechanism, of the program-ming language and so on.

      Def symbolic architecture

    27. If the gearbox is removed and replaced by a different unit that provides the same conver-sion, the function of the overall system—the car—might be preserved. Such a separation is often successful in biological systems too. A kidney, for instance, may be described as a system to fi lter certain chemicals from the bloodstream. If the kidneys are replaced by an artifi cial contraption that fi lters the same chemicals (during dialysis, for instance), the organ-ism may continue to function as before. There are counterexamples, too: a misconstrued ontology may specify the fuel of the car simply as an energy source. If the fuel tank would be replaced by an arbitrarily chosen energy source, such as an electrical battery, the car would cease to func-tion, because fuel is not just an energy source—to be compatible with a combustion engine, it needs to be a very specifi c agent that when mixed with air and ignited shows specifi c expansive properties. The car’s fuel may perhaps be replaced with a different agent that exhibits similar func-tional properties, such as alcohol or natural gas, provided that the com-patibility with the engine is maintained. Even then, there might be slight differences in function that lead to failure of the system in the long run, for instance, if the original fuel has been providing a lubricating function that has been overlooked in the replacement.

      The functional nature of working ontologies.

      Misconstrued ontologies

    28. Contrary to the intuition that machines are always artifacts, here, a machine is simply seen as a system of interrelated parts that are defi ned by their functionality with respect to the whole:Machines need not be artifi cial: organisms are machines, in the sense of “machine” that refers to complex functioning wholes whose parts work together to produce effects. Even a thundercloud is a machine in that sense. In contrast, each organism can be viewed simultaneously as several machines of different sorts. Clearly organisms are machines that can reorganize matter in their environment and within themselves, e.g. when growing. Like thunderclouds, windmills and dynamos, animals are also machines that acquire, store, transform and use energy. (Sloman & Chrisley, 2005)

      Organism as machine

    29. All theories that are expressed in such a way that they may be completely translated into a strict formal language are computational in nature. The ontological or methodological assumption that is made by the computational theory of mind is not unique to cogni-tive science, but ubiquitously shared by all nomothetic (Rickert, 1926) sciences, that is, all areas that aim at theories that describe a domain exhaustively using strict laws, rules, and relations. This is especially the case for physics, chemistry, and molecular biology.

      Computational = translatable into a formal language

    30. The theory may be seen as an ontological commitment (in the form that either the universe itself is a computational process (e.g., Wolfram, 2002), and thus everything within it—such as minds—is computational too, or that at least mental processes amount to infor-mation processing). But even if one does not subscribe to such a strong view, the theory of mind may be treated as a methodological commit-ment. This second view, which I would like to call the “weak computa-tional theory,” has been nicely formulated by Johnson-Laird, when he said:Is the mind a computational phenomenon? No one knows. It may be; or it may depend on operations that cannot be captured by any sort of computer. ( . . . ) Theories of the mind, however, should not be confused with the mind itself, any more than theories about the weather should be confused with rain or sunshine. And what is clear is that computability provides an appropriate conceptual apparatus for theories of the mind. This apparatus takes nothing for granted that is not obvious. ( . . . ) any clear and explicit account of, say, how people recognize faces, reason deductively, create new ideas or control skilled actions can always be modelled by a computer program. (Johnson-Laird, 1988)Indeed, cognitive models can be seen as the attempt to elucidate the workings of the mind by treating them as computations, not necessarily of the sort carried out by the amiliar digital computer, but of a sort that lies within the broader framework of computation (ibid, p. 9).

      Strong and weak computational theories of mind. Johnson-Laird expresses the latter: "Theories of mind... should not be confused with the mind itself, any more than theories about the weather should be confused with rain or sunshine... cognitive models can be seen as the attempt to elucidate the workings of the mind by treating them as computations, not necessarily of the sort carried out by the familiar digital computer, but of a sort that lies within the broader framework of computation."

    31. Cognitive architectures defi ne computational machines as models of parts of the mind, as part of the interaction between cognitive functions and an environment, or as an ongoing attempt to explain the full range of cognitive phenomena as computational activity. This does not, of course, equate the human mind with a certain computer architecture, just as a computational theory of cosmology—a unifi ed mathematical theory of physics—maintains that the universe is possessed by a certain computer architecture. It is merely a way of expressing the belief that scientifi c theories of the mind, or crucial parts of research committed to a better understanding of the mind, may be expressed as laws, as rules, as sys-tematized regularities, that these regularities can be joined to a system-atic, formal theory, and that this theory can be tested and expanded by implementing and executing it as a computer program.

      Formation of computation models of mind are just about expression of regularities, not the search for some exact "real" computer in the mind

    32. I see two objections to radical behavior-based approaches, which in my view limit their applicability to the study of cognitive phenomena: First, while a majority of organisms (Drosophila, the fruitfl y, for instance) manages to capitalize on its tight integration with physical properties of its environment, only a small minority of these organisms exhibits what we might call cognitive capabilities. And second, this majority of tightly integrated organisms apparently fails to include famous physicist Stephen Hawking, who is struck with the dystrophic muscular disease ALS and interacts with the world through a well-defi ned mechatronic interface—his friendship with physics takes place on an almost entirely knowledge-based level. In other words, tight sensor-coupling with a rich physical environment seems neither a suffi cient nor a necessary condi-tion for cognitive capabilities.

      Bach's critique of radical behavior-based approaches

    33. Not all theorists of cognitive modeling, even though they tend to accept functionalist materialism and the representational theory of mind, agree with Fodor’s proposal. Many connectionists argue that symbolic systems lack the descriptive power to capture cognitive pro-cesses (for a review, see Aydede, 1995). Yet they will have to answer to the requirements posed by productivity, systematicity, and inferential coherence by providing an architecture that produces these aspects of mental processes as an emergent property of nonsymbolic processing. Fodor (Fodor & Pylyshyn, 1988) maintains that a connectionist architec-ture capable of productivity, systematicity and inferential coherence will be a functional realization of a symbolic system (i.e., the connectionist implementation will serve as a substrate for a symbolic architecture).

      Fodor's LOTH is not uncontroversial

    34. Fodor did not exactly state something new in 1975, and thus did not open up a new research paradigm in cognitive science. Rather, he spelled out the assumptions behind artifi cial intelligence models and cybernetic models in psychology: Perception is the fi xation of beliefs, the learning of concepts amounts to forming and confi rming hypotheses, and deci-sion making depends on representing and evaluating the consequences of actions depending on a set of preferences.

      Fodor's Language of Thought Hypothesis was not a new paradigm so much as the spelling-out of certain assumptions underlying AI models at the time

    35. Daniel Dennett, in an introduction to Gilbert Ryle’s classic “Ghost in the machine” (Dennett, 2002), introduces the idea of a “zombank” to illustrate this. A zombankwould be something that looks and acts like a fi nancial institution, where people could have an account, store and withdraw money and so on, but which is not a real bank, because it lacks some invisible essence beneath its interface and functionality that makes a bank a bank. Just as the notion of a zombank strikes us absurd (after all, a bank is commonly and without loss of generality defi ned by its interface and functionality), Dennett suggests that the idea of a philosophical “zombie,” a cognitive system that just acts as if it had a mind, including the ability for dis-course, creative problem solving, emotional expression and so on, but lacks some secret essence, is absurd.

      Zombank argument against philosophical "zombies"

    36. A system capable of fulfi lling the breadth of cognitive tasks required for general intelligence is a model of a unifi ed theory of cognition (Newell, 1987), an implementation of a so-called cognitive architecture.

      A system capable of fulfilling the breadth of cognitive tasks required for general intelligence is a model of a unified theory of cognition (Newell, 1987), an implementation of a so-called cognitive architecture.

      Notion of general intelligence in the context of the "unified theory of cognition" / "cognitive architecture" approach

    37. The idea of a full-featured model of the crucial components of human cognition was advanced by Alan Newell and Herbert Simon as a con-sequence of the physical symbol system hypothesis (Newell & Simon, 1976). According to this hypothesis, a physical symbol system, that is, an implemented Turing machine, “has the necessary and suffi cient means for general intelligent action. By “necessary” we mean that any system that exhibits general intelligence will prove upon analysis to be a phys-ical symbol system. By “suffi cient” we mean that any physical symbol system of suffi cient size can be organized further to exhibit general intelligence”8 ( Newell, 1987, p. 41).

      Definition of physical system hypothesis

    38. The idea of describing the mind itself as a functional system has had an enormous impact on a cer-tain area on psychology and philosophy that has consequently been asso-ciated with the term functionalism (Fodor, 1987; Putnam, 1975, 1988). If a functionalist subscribes to representationalism (the view that the functional prevalence of a mental state entails its representation within a representing system) a functionalist model of cognitive processes might be implemented as a computer program (computationalism) and per-haps even verifi ed this way, so functionalism often goes hand in hand with computer science’s proposal of Artifi cial Intelligence.7 Even if mental processes could not be modeled as a computational model—any detailed, formal theory on how the mind works certainly can (Johnson-Laird, 1988, p. 9).

      Position that modeling of the brain in terms of its functions is possible

    39. What the universe makes visible to science (and any observer) is what we might call functionality. Functionality, with respect to an object, is-loosely put—the set of causally relevant properties of its feature vec-tor.6 Features reduce to information, to discernible differences, and the notions we process in our perception and imagination are systematically structured information, making up a dynamic system. The description of such systems is the domain of cybernetics or systems science

      Functionality is what is visible to us in the universe.

      Information = discernible differences

      Dynamic systems

    40. The observer does not have an exclusive, intimate access to the objects of its cognition and representation that would enable it to wit-ness “real” mental states. What we know about ourselves, including our fi rst-person-perspective, we do not know because we have it available on “our side of the interface.” Everything we know about ourselves is a sim-ilar ordering we found over features available at the interface; we know of mental phenoma only insofar as they are explicitly accessible patterns or constructed over these patterns. Even though our cognitive processes are responsible for the functionality of ordering/conceptualization and recognition, they are—insofar as they are objects of our examination—“out there” and only available as regularities over patterns (over those patterns that we take to be aspects of the cognitive processes).

      Our self-access is no different from how we access information from the rest of the external environment

    41. A similar example is supplied by Hilary Putnam (1975): Individuals in a hypothet-ical twin-world to earth on which all water has been replaced by a chemical compound XYZ with identical properties would arrive at the same observations and conceptual-izations. Thus, the content of a concept that is encoded in a mental state refers to the functional role of the codifi ed object.

      Twin earth hypothesis in support of functionalist contructivism

    42. An opponent of this view (arguing, for instance, from an essentialist or realist perspective) might suggest that we intuitively do have access to physical objects in the world; but this argument may be tackled using a simple thought experiment: if someone would remove one of the objects of our world and just continue to send the related patterns to our sys-temic interface (for instance, to our retina) that correspond to the contin-ued existence of the object and its interaction to what we conceptualize as other physical objects, we would still infer the same properties, and no difference could be evident. If, for instance, all electrons in the world would be replaced by entities that behave in just the same way, batteries would continue to supply electrical energy, atoms would not collapse and so on: no difference could ever become evident.4 Now imagine the removal of the complete environment. Instead, we (the observers) are directly connected (for instance, by our sensory nerves) to an intricate pattern generator that is capable of producing the same inputs (i.e., the same patterns and regularities) as the environment before—we would still conceptualize und recognize the same objects, the same world as we did in the hypothetical world of “real” objects. There can be no difference, because everything that is given is the set of regularities (re-occurrence and seeming dependencies between the patterns).

      Critique of realism

    43. Functionalist constructivism is based on the epistemological posi-tion of philosophical constructivism (see, for instance, von Foerster & von Glasersfeld, 1999) that all our knowledge about the world is based on what is given at our systemic interface. At this interface, we do not receive a description of an environment, but features, certain patterns over which we construct possible orderings. These orderings are func-tional relationships, systems of categories, feature spaces, objects, states, state transitions, and so on. We do not really recognize the given objects of our environment; we construct them over the regularities in the infor-mation that presents itself at the systemic interface of our cognitive system.For example: if we take a glance out of the window on a cloudless day, we do not simply perceive the sun as given by nature, rather, we identify something we take as a certain luminance and gestalt in what we take to be a certain direction, relatively to what we take to be a point in time. A certain direction is understood as something we take as a character-istic body alignment to something we take as a certain place and which makes a certain set of information accessible that we take to be a certain fi eld of view. In such a way, we may decompose all our notions into the functional features that are the foundation of their construction. Thus, all our notions are just attempts at ordering patterns:

      Definition of functionalist contructivism

    44. Psychology, which originally had its roots as a natural science in the psychophysics of Fechner and Helmholtz, became an independent disci-pline when Helmholtz’ pupil Wilhelm Wundt founded his experimental laboratory at the University of Leipzig in 1874 (Boring, 1929). The under-standing of psychology as an experimental science was later challenged, especially by the psychoanalytic movement, starting in the 1890s, and because of the speculative nature of the psychoanalytic assumptions, psychology came under heavy fi re from positivists and empiricists in the fi rst half of the twentieth century (see Gellner 1985, Grünbaum 1984). The pendulum swung backwards so violently that the psycholog-ical mainstream turned away from structuralism and confi ned itself to the study of directly observable behavior. Behaviorism, as proposed by John B. Watson (1913) became very infl uential, and in the form of radi-cal behaviorism (Skinner, 1938) not only neglected the nature of mental entities as on object of inquiry, but denied their existence altogether. At the same time, this tendency to deny the notion of mental states any scientifi c merit was supported by the advent of ordinary language phi-losophy (Wittgenstein, 1953, see also Ryle, 1949). Obviously, the negli-gence of internal states of the mind makes it diffi cult to form conclusive theories of cognition, especially with respect to imagination, language (Chomsky, 1959) and consciousness, so radical behaviorism eventually lost its foothold. Yet, methodological behaviorism is still prevalent, and most contemporary psychology deals with experiments of quantitative nature (Kuhl, 2001). Unlike physics, where previously unknown enti-ties and mechanisms involving these entities are routinely postulated whenever warranted by the need to explain empirical facts, and then evidence is sought in favor of or against these entities and mechanisms, psychology shuns the introduction of experimentally ungrounded, but technically justifi ed concepts. Thus, even cognitive psychology shows reluctance when it comes to building unifi ed theories of mental pro-cesses. While Piaget’s work (especially Piaget, 1954) might be one of the notable exceptions that prove the rule, psychology as a fi eld has a prefer-ence for small, easily testable microtheories (Anderson, 1993, p. 69).

      History of metanarratives in psychology -- resistance to unified theories in the modern field

    45. In philosophy, cognition usually relates to intentional phenomena, which in functionalist terms are interpreted as mental content and the processes that are involved with its manipulation. The position that intentional phenomena can be understood as mental representations and operations performed upon them is by no means shared by all of contemporary and most traditional philosophy; often it is upheld that intentionality may not possibly be nat-uralized (which usually means reduced to brain functions). However, the concept that intentional states can be explained using a representational theory of the mind is relatively widespread and in some sense the foun-dation of most of cognitive science.

      Intentionality in philosophy vs cognitive science

    46. They must give their ideas expression as programs; a computer program will fail on the slightest mistake. This makes computer scientists continuously aware of the limits of human intuition, and of the need to revoke and improve their original theories. Thus, computer scientists are often sober and careful when compared to many a philosopher; humble, pedantic, and pessimistic when compared to some psychologists. They also tend to know that there are many examples for models of mental processes that are technically simplistic, but not easy to invent because of their unin-tuitive nature, such as self-organizing maps and backpropagation learn-ing. On the other hand, they know of problems that are apparently much easier to understand than to fully conceptualize and implement, such as frames (Minsky, 1975).

      Computer scientist's mindset

    47. Yet, there is reason to believe that, despite inevitable diffi culties and methodological problems, the design of unifi ed architectures modeling the breadth of mental capabilities in a single system is a crucial stage in understanding the human mind, one that has to be faced by researchers working at the interface of the different sciences concerned with human abilities and information processing.

      Bach urges that even if our initial attempts will appear over-simplified, it is still worthwhile to design a working cognitive architecture for the sake of a variety of sciences

    48. Such broad architectures are necessarily shallow at fi rst, replacing crucial component s with scaffolding and complex behaviors with simple ones. They have to rely on ideas, paradigms, results, and opinions stem-ming from many disciplines, each sporting their own—often incom-patible—methods and terminology. Consequently they will be full of mistakes and incorrect assumptions

      The first versions of cognitive architectures are necessarily shallow in design

    49. It might be diffi cult to draw a dividing line between the physical stance and the design stance: the levers and wheels making up a watch are physical entities as well as parts from an engineer’s toolbox. On the other hand, the lowest levels of description used in physics are in a state of fl ux. They are competing theories, pitched against each other with respect to how well they construct observable behaviour. In fact, both the physical stance and the design stance are functionalist stances, each with regard to a dif-ferent level of functionality.

      Physical vs design stances

    50. I n r e c e n t y e a r s , t h e r e h a v e b e e n c a l l s f r o m d i f f e r e n t d i s c i p l i n e s — including AI, psychology, cognitive neuroscience, and philosophy—to gather under the new roof of cognitive science and to concentrate on integrative archi-tectures that are laid out specifi cally for the purpose of modeling and understanding the human mind. These cognitive architectures, including EPAM (Gobet, Richman, Staszewski, & Simon 1997); Newell, Laird, and Rosenbloom’s Soar (1987); and Anderson and Lebière’s ACT (Anderson, 1983, 1990), have become a fl ourishing new paradigm.

      Cognitive architectures

    51. When looking at a system, we might take different stances, as the philosopher Daniel Dennett suggested (1971): the physical stance, which attempts a description at the level of the relevant physical entities (the physical make-up and the governing laws); the design stance (how the system is constructed); and the intentional stance (a description of the system in terms of beliefs, desires, intentions, attitudes, and so on).1Computer science allows taking an active design stance: the one of a con-structing engineer (Sloman, 2000). Understanding a system in computer science means one is able to express it fully in a formal language, and expressing it in a formal language amounts (within certain constraints) to obtaining a functional model of the thing in question. If the system to be modeled is a physical system, such as a thunderstorm, then the result will be a simulation of its functioning. But if we are looking at an infor-mation processing system that in itself is supervening over its substrate, then we are replacing the substrate with the implementation layer of our model and may obtain a functional equivalent.
      • Dennett's three stances to take when looking at systems: physical stance, design stance, and intentional stance.

      • The "active" design stance as an addition to Dennett's three stances. Defines the approach allowed by computer science: "Understanding a system in computer science means one is able to express it fully in a formal language, and expressing it in a formal language amounts (within certain constraints) to obtaining a functional model of the thing in question."

      "If the system to be modeled is a physical system, such as a thunderstorm, then the result will be a simulation of its functioning. But if we are looking at an information processing system that in itself is supervening over its substrate, then we are replacing the substrate with the implementation layer of our model and may obtain a functional equivalent.

    52. This book is completely dedicated to understanding the functional workings of intelligence and the mechanisms that underlie human behavior by creating a new cognitive architecture. That might seem like a piece of really old fashioned artifi cial intelligence (AI) research, because AI, a movement that set out 50 years ago with the goal of explaining the mind as a computational system, has met with diffi culties and cultural resistance.However, AI as an engineering discipline is still going strong. The exploration of individual aspects of intelligence such as perception, representation, memory and learning, planning, reasoning, and behav-ior control led to tremendous insights and fruitful applications. Results from AI have brought forth independent fi elds of research as diverse as computer vision, knowledge management, data mining, data compres-sion, and the design of autonomous robots.

      Cognitive-style AI research has met resistance, but has nevertheless led to highly fruitful findings in knowledge management, data mining, data compression, etc.

    53. The PSI architecture, while including perceptual, motor, learn-ing, and cognitive processing components, also includes several novel knowledge representations: temporal structures, spatial memories, and several new information processing mechanisms and behaviors, includ-ing progress through types of knowledge sources when problem solv-ing (the Rasmussen ladder)

      Novel knowledge representations of MicroPSI

    54. MicroPSI has several lessons for other architectures and models. Most notably, the PSI architecture includes drives and thus directly addresses questions of motivation, autonomous behavior, and emotions. MicroPSIsuggests how emotions arise, and how drives and emotions are different. Including drives also changes the way that the architecture works on a fundamental level, providing an architecture suited for behaving auton-omously, which it does in a simulated world. PSI includes three types of drives, physiological (e.g., hunger), social (i.e., affi liation needs), and cognitive (i.e., reduction of uncertainty and expression of competency). These drives routinely infl uence goal formation and knowledge selec-tion and application. The resulting architecture generates new kinds of behaviors, including context-dependent memories, socially motivated behavior, and internally motivated task switching. This architecture illustrates how physiological drives and emotions can be included in an embodied cognitive architecture.

      "Drives" are central to MicroPSI

    1. In Heider’s view, the psychological explanations told by the subjects in Heider andSimmel’s experiments were not based on scientific theories, but rather on a com-monsense theory of psychology. Here the behaviors of the triangles and circle areexplained using mentalistic terms for the characteristics attributed to these objects,including their beliefs, goals, plans, and emotions. This was the same theory thatpeople used to explain the behavior of others in everyday interpersonal relations,and that served as the basis for everyday predictions, decisions, and planning thatinvolved other people. Commonsense psychology is not a theory in the scientificsense, meant to describe how our brains actually operate. Instead, it is a common-sense theory of how we think we think.

      Commonsense psychology = "how we think we think."

    2. In 1944 when this work was published, psychology in the United States was stillunder the strong influence of Behaviorism and its efforts to eschew mentalistic con-structs in explanations of human behavior. Set in this context, Heider and Simmel’sexperiment might be viewed as a poignant challenge to their contemporaries, high-lighting a particular human behavior that seems impossible to understand withoutappealing to mentalistic causes. Instead, it may be better to view this work as a con-tinuation of the Gestalt psychological traditions that Fritz Heider brought with himfrom Europe. After receiving his Ph in 1920 from the University of Graz in Austria,Heider spent a decade in Germany working under the intellectual leaders of theGestalt school of psychology, including Wolfgang Koehler, Max Wertheimer, KurtKoffka, and Kurt Lewin. The study of visual perception was a major focus of thisgroup of academics, and their use of simple geometric shapes in experimental stim-uli was standard practice before and after Heider left Germany for the United Statesin 1930.

      Intellectually contextualizing the Heider-Simmel study. Not so much a repudiation of behaviorism as a continuation of the Gestalt tradition

    3. Nearly every subject in the first group described the events in anthropomorphicterms, typically involving a fight between two men (the triangles) over a woman(the circle). The natural tendency was to describe what happened in terms of humanbehavior, even without instructions to do so. The narratives of these subjects employmentalistic phrases: The girl hesitates; she doesn’t want to be with the first man; thegirl gets worried; man number two is still weak from his efforts to open the door; thegirl and man number two finally elude man number one and get away; man numberone is blinded by rage and frustration.The second group of subjects further explainedthese anthropomorphic interpretations: the larger triangle is an aggressive bully, thesmaller triangle is heroic, the circle is helpless, and the actions of the smaller triangleand circle lead to their successful escape from the larger triangle, sending him into arage for being thus thwarted. The third group of subjects, seeing the film in reverse,also interpreted the movements as human actions. Here the variations in narrativewere much greater: a man resolved to his fate is disrupted by a woman accompa-nied by evil incarnate, a prisoner thwarts a murder attempt and then escapes, and amother chastises a father over the behavior of their child.

      Anthropomorphization in the Heider-Simmel study. "The natural tendency was to describe what happened in terms of human behavior, even without instructions to do so."

    4. In 1944 Fritz Heider and Marianne Simmel published the results of a novel exper-iment conducted at Smith College in Northampton, Massachusetts. In “An experi-mental study of apparent behavior” Heider and Simmel (1944) prepared a brief filmdepicting the movements of a two triangles and a circle in and around a shape madeto look like a room with a door. The film was shown to 114 undergraduate womenat Smith College divided into three experimental groups. In the first, subjects weregiven the general instruction to “write down what happened in the picture.” In thesecond, subjects were instructed to interpret the movements of the figures as actionsof persons and to answer ten questions in a written questionnaire. The third groupwas shown the film in reverse, by running the filmstrip backwards through the pro-jector, and asked a subset of the same questions.

      Heider-Simmel study (1944)

    1. At the time, Brooks was winding down collaborative robotics startup Rethink Robotics, which shuttered in October 2018. He soon signed on as a cofounder and chief technology officer. The company’s three other cofounders have similar stellar pedigrees: Mohamed Amer previously led AI and machine learning projects at SRI International; Anthony Jules had been chief technology officer of Formant.io, an intelligence platform for robot fleets and a senior product manager at X; and Henrik Christensen is a professor at UC San Diego and director of its Contextual Robotics Institute.

      Background of Rethink founders

    1. In chapter I I said that years of study have convinced me that computation is not a distinct or autonomous suycct matter, but is instead a complexp&c, involving the design, construction, maintenance, and use of intentional systems.

      computation is not a distinct or autonomous subject matter, but is instead a complex practice, involving the design, construction, maintenance, and use of intentional systems.

    2. This is the world vim captured in Kronecker's famous dictum: that "God made the integers; all else is the work of man."s If my metaphysical picture is right, Kronecker's order ofexplanation is chse to backwardr. Metaphysical indefiniteness is the base case, continuity needs to be extruded from the flux, and then dis- creteness won, at a very high price, from that

      This is the world vim captured in Kronecker's famous dictum: that "God made the integers; all else is the work of man."s If my metaphysical picture is right, Kronecker's order of explanation is close to backwards. Metaphysical indefiniteness is the base case, continuity needs to be extruded from the flux, and then discreteness won, at a very high price, from that.

      Cantwell-Smith's philosophy as opposite of the common assumption in mathematics that the discrete is somehow basic

    3. At its most basic level, that world is depicted as one of cosmic and ultimately ineffable particularity: a criti- cally rich and all-enveloping deictic flux. Neither formally rigid nor nihilistically sloppy, the flux sustains complex processes of refitration: a form of interaction, subsuming both representa- tion and ontology, in which "s-regions" or subjects stabilize patches of the flux, in part through processes of intervention, adjusting and building them and beating them into shape, and also through patterns of disconnection and long-distance coor- dination, necessary in order to take the patch to be an object, or more generally, to be something in andof the worLL

      Summary of author's philosophy ("philosophy of presence")

      At its most basic level, that world is depicted as one of cosmic and ultimately ineffable particularity: a critically rich and all-enveloping deictic flux. Neither formally rigid nor nihilistically sloppy, the flux sustains complex processes of registration: a form of interaction, subsuming both representation and ontology, in which "s-regions" or subjects stabilize patches of the flux, in part through processes of intervention, adjusting and building them and beating them into shape, and also through patterns of disconnection and long-distance coordination, necessary in order to take the patch to be an object, or more generally, to be somethingin and of the world.

    4. In recent work in artificial life and theoretical biology, it has been hypothesized that much of life, if viewed as a dynamical system, occupies a "critical region" between stability and chaos. "Stable" does not mean static or passive; rather, the contrast is between behavioral regions that are linear or at least smoothly regular, in ways we have studied for centuries in classical dynam- ics, and only much more recently theorized regions of genuine chaos or turbulence. It is the boundary between the two that is called m'tical, poised unstably between these two extremes.

      In recent work in artificial life and theoretical biology, it has been hypothesized that much of life, if viewed as a dynamical system, occupies a "critical region" between stability and chaos. "Stable" does not mean static or passive; rather, the contrast is between behavioral regions that are linear or at least smoothly regular, in ways we have studied for centuries in classical dynamics, and only much more recently theorized regions of genuine chaos or turbulence. It is the boundary between the two that is called critical, poised unstably between these two extremes.

    5. Empty space, it is said, is not really empty if you look very very hard. Instead it everywhere and always (to say nothing of al- ready) boils and bubbles, toils and troubles, with countless mil- lions of subatomic particles and their antimatter opposites seething in a somewhat random but thoroughly intermixed pat- tern of activity. At even quite a low level of statistical averaging, all this activity cancels itself out, adding up to nothingness - adding up, that is, to what we idealize with our formal notions of "Empty Space" and "Vacuum." But au fond space is more unruly than that.

      Visualization of quantum mechanical image of "empty space"

    6. Abstraction is a process of going from the ineffibly rich mi- crodetails of a particular circumstance to an abstract or categori- cal characterization of it, a characterization that washes out 95 percent of those details and places it alongside other circum- stances in a general conceptual typology.

      Somewhat novel: an image of abstraction as "washing out details"

    7. Even the words I have used for what we do to objects ('sediment' and 'extrude,' when activity was in focus; 'still' or 'quieten' or 'stabilize', when the passivity of the re- sult was relevant; as well of course as 'register') place responsibil- ity too heavily on the subject. Traditional intentional terms like 'see', 'refer to,' 'recognize,' 'perceive,' 'discover,' etc., place it too heavily on the object. As suggested by the garden example, above and in the preface, agriculture is a useful source of meta- phors: words such as 'grow,' 'till,' 'shepherd,' and the like, con- vey a salutary sense of collaboration between fanner and farmed-and thus of the resultant phenomenon's being located in an appropriately textured middle region.

      Finding the language of collaboration: 'grow', 'shepard', or 'steward' vs the overly subject-oriented "stabilize" or "register" as well as the overly object-oriented "recognize", "perceive" etc.

    8. I have said many times, that an account of full-scale intentionality, as opposed merely to registration, must be an account ofpanin$aiion or en- gagement-not just of cognition, and not even just of experi- ence. Planting a garden, driving across the tundra, and spilling dioxin in the water supply are aggressive and violent acts, not capturable by being described as a way of experiencing the world. And as suggested in chapter I, a participatory moral is also a clear result of studying computation in the wild. In spite of the fact that the theoretical frameworks passed down to us from intellectual history, especially those inherited from the formal tradition, by and large treat computation as detached symbol manipulation, in actual fact computers are inextricably involve

      I have said many times, that an account of full-scale intentionality, as opposed merely to registration, must be an account of participation or engagement--not just of cognition, and not even just of experience. Planting a garden, driving across the tundra, and spilling dioxin in the water supply are aggressive and violent acts, not capturable by being described as a way of experiencing the world. And as suggested in chapter I, a participatory moral is also a clear result of studying computation in the wild. In spite of the fact that the theoretical frameworks passed down to us from intellectual history, especially those inherited from the formal tradition, by and large treat computation as detached symbol manipulation, in actual fact computers are inextricably involved in their subject matters, whether that is a world of documents, electronic mail, network servers, microwave ovens, keyboards, or computerized brakes. No model of them as purely representational, or even as purely experiencial, can begin to do them justice.

      note: I wonder if this begins to articulate, in a sense, a critique of the notion of the "virtual"..

    9. Imagine, instead, a scenario in which one starts with the world as a whole, and slowly takes it apart into objects-pulls them apart, separates them, makes distance between them, as it were, but only gradually, the way a potter would gradually pull pots out of a prior mass of clay. The pieces are (Partial&) sedi- mented or ddfiom the whole; the whole ir not put together fiom thepieces-a fact of great significance, implying for exam- ple that there are infinitely many unregistered (and hence un- separated) connections and ties and kinds of connective tissue tying everything together, more than can possibly be imagined, or than could possibly be the result of any given articulation.

      Imagine, instead, a scenario in which one starts with the world as a whole, and slowly takes it apart into objects-pulls them apart, separates them, makes distance between them, as it were, but only gradually, the way a potter would gradually pull pots out of a prior mass of clay. The pieces are (partially) sedimented or extruded from the whole; the whole is not put together from the pieces--a fact of great significance, implying for example that there are infinitely many unregistered (and hence unseparated) connections and ties and kinds of connective tissue tying everything together, more than can possibly be imagined, or than could possibly be the result of any given articulation.

    10. In order to stabilk the 0-region as an extended unity, the rregion must employ all sorts of compensatory strategies in or- der to invert the contribution of every other element of the world that has been involved in obtaining access to the o-region in the first place. It must, as I said, deconvolve the deixis-pre- cursor to the later process of shifting the registration of the ob- ject from egocentric to allocentric coordinates. This helps it to begin the long and tough process of triangulating on the object, and washing out the mibution of everything else

      Stabilizing objects as "deconvolving the deixis"

    11. Something very like this is the function of the vestibular- ocular reflex. In synchrony with rotations of our head, and with displacements in our body, we adjust the angle of the orientation of our eyes in their sockets so that a constant (distal) point of vi- sual focus is maintained. We can even do it when we are not our- selves responsible for the body's motion. Thus imagine looking out the window of a train. It is possible to hold one's head and eyes fixed, which turns the visual field into an incomprehensible blur. The natural thing to do, however, guided by visual feed- back, is to rotate our eyes and head in such a way as to compen- sate for the motion of the train-with, again, a net result of maintaining a stable distal point of focus. It is this sedimenta- tion of distal stability that begins to answer the question, raised in the introduction, ofwhy we see trees, not electro-magnetic ra- diation.

      Vestibular-ocular reflex

    12. The described non-e&ctive tracking mechanisms still suftice only to maintain perceptual invariants: quietened, somewhat abstract (though still particular), deictic "relations becween sub- ject and object." They compensate for the effective separation, and thus succeed in maintaining relational stability. But they do not yet mploit the separation-and thus do nothing to stabilize the object as an object (or as anything else, for that matter).

      The described non-e&ctive tracking mechanisms still suffice only to maintain perceptual invariants: quietened, somewhat abstract (though still particular), deictic "relations between subject and object." They compensate for the effective separation, and thus succeed in maintaining relational stability. But they do not yet exploit the separation--and thus do nothing to stabilize the object as an object (or as anything else, for that matter).

      Objectivization as exploitation of separation

    13. It would be bizarre to say that your jacket "detects" your mo- tion --even though, sure enough, it changes state in a way that is lawfully correlated with your motion. It changes state subject to the constraints of overarching physical law because it is con- nected to you- by coming along with you. So too of retinal ac- tivity and flight of fly.

      wtf

    14. it would be more accurate and more modest, at least in the first in- stance, to call them edgepa&cipatots, rather than edge detectors.

      it would be more accurate and more modest, at least in the first instance, to call them edge participators, rather than edge detectors.

    15. Even more specifically, consider the otherwise ordinary- sounding claim that an edge detector in the fiog's retina is firPd or trig;gered by the edge of the image of the fly. For a theorist to register a frog's neuronal activity in that way is to commit to a coincidence of three boundaries: (a) that between quiescence and firing, being triggered or not, in the contended mechanism in the frog (on or off); (b) the corresponding edge of the fly (be- ingfly, and not bkngjy, in the mass density region several feet in front of the frog); and (c) the edge of the intermediating pattern of directed electromagnetic radiation. But then, once one real- izes that no warrant has yet been provided for separating these three boundaries, one sees that what is really going on is best understood in purely field-theoretic terms: there is a single com- mon edge to a columnar-shaped multimedia disturbance that reaches continuously between (what we register as) frog and (what we register as) fly.

      Even more specifically, consider the otherwise ordinary- sounding claim that an edge detector in the frog's retina is fired or triggered by the edge of the image of the fly. For a theorist to register a frog's neuronal activity in that way is to commit to a coincidence of three boundaries: (a) that between quiescence and firing, being triggered or not, in the contended mechanism in the frog (on or off); (b) the corresponding edge of the fly (being fly, and not being fly, in the mass density region several feet in front of the frog); and (c) the edge of the intermediating pattern of directed electromagnetic radiation. But then, once one realizes that no warrant has yet been provided for separating these three boundaries, one sees that what is really going on is best understood in purely field-theoretic terms: there is a single common edge to a columnar-shaped multimedia disturbance that reaches continuously between (what we register as) frog and (what we register as) fly.

    16. edge detectors are among the first neuronal cir- cuits to get into the registrational act. Except that to call them edge &tenon is both misleading and expensive. It is expensive because it describes the situation in terms dangerously dose to the structure of the solution ('detect' being a fully intentional word). It is misleading because all we have, so far, is a causal loop involving subject and environment. Without further patterns of disconnected coordination, there is no justification for invoking the heavy language of detection-or, for that matter, for saying that the fro% sees anything at all. To register the circuitry in the frog's eye as an edge detector, without paying its cost, is to over- interpret the situation. It hides the fict that the mechanism sim- ply involves mow ~zccretzd kutuiaty.

      Edge detectors are among the first neuronal circuits to get into the registrational act. Except that to call them edge detectors is both misleading and expensive. It is expensive because it describes the situation in terms dangerously dose to the structure of the solution ('detect' being a fully intentional word). It is misleading because all we have, so far, is a causal loop involving subject and environment. Without further patterns of disconnected coordination, there is no justification for invoking the heavy language of detection--or, for that matter, for saying that the frog sees anything at all. To register the circuitry in the frog's eye as an edge detector, without paying its cost, is to over- interpret the situation. It hides the fact that the mechanism simply involves more accreted boundary

    17. It is claimed in the text that intentionality is not a calrsal phenomenon, and will not have a causal explanation, and, even more strongly, that ob- jects are also non-causal, and will similarly not have causal explanations. Strictly speaking, this is only true to the extent that 'causal' means 'effec- tive.' The real claims are that intentionality is not an gective phenome- non, and will not have an gective explanation, and similarly that objects are non-gective, and will not have effective explanations.

      causal = effective

    18. Intentionality L a way of exploiting loca(fieednm or slop in orh to esublirh coordination with what L beyond gective reach

      Intentionality is a way of exploiting local freedom or slop in order to establish coordination with what is beyond effective reach.

    19. Throughout, too, it is easy to see how the flex and slop are im- plicated. To start with, they underwrite the very notion of sepa- ration-and hence of connection, disconnection, and the limits of effective reach. If, as in the gear world, there were no flex, there would be no warrant for saying that two parts of the world had come apart. In hct there would probably be no warrant for saying that the world had parts at all, and certainly no warrant for saying that any two parts were fir away from each other. Fur- thermore, exactly because of this slop, it is not automatic that the proximal system will stay synchronized with what is distal. This is why I said at the outset that there is a sense in which slop "establishes the problem" of intentionality.

      "Intentionality" emergent from the fact that connections between part of the world are not automatic.

      An elegant definition! But useful?

    20. There is a great deal more to intentionality than that, and a great deal to say about what constitutes coupling, coordination, and so on, but in various forms these notions of connection, gradual dis- connection, maintenance of coordination while disconnected or separated, and ultimate reconnection or reconciliation per- meate all kinds of more sophisticated example. There is nothing more basic to intentionality than this pattern of coming to- gether and coming apart, at one moment being fully engaged, at another point being separated, but separated-this is the point -in such a way as to stay coordinated with what at that mo- ment is distal and beyond effective reach.

      Intentionality is about staying connected to an intended object, even if physically or perceptually it ebbs in and out of "effective" reach.

    21. blocked from eEective view, or removed from effective grasp, it is not thereby eliminated from one's thoughts. Inten- tional directedness is not held in place with physical glue. Long after the physical tie has been broken, or has stretched too thin to hold anything together, or has aged so much that it has be- come brittle and cracked and yellow, the semantic relation tying a subject to their subject matter will persevere, &u less battered by the buffking of circumstances, far less deteriorated by the ravages of time.

      even if no longer grasped perceptively, the object remains in memory.

    22. The gear world would lack slop. Effects would not dissipate. If one gear were to move by even a tiny amount, every other gear in the universe, no matter how fir flung, would instantly and proportionally be &ected.ll

      In gear world, "effects would not dissipate."

    23. 1 have phrased this as if there would be time and distance in the gear world, but that is almost surely fdse. We daim that there is no action at a distance, but that gets it backwards. Distance is what thm is no artion a

      I have phrased this as if there would be time and distance int he gear world, but that is almost certainly false. We claim that there is not action at a distance, but that gets it backwards. Distance is what there is no action at.

    24. The world's flex and slop is so obvious that it is a little hard to tak about. As a contrast, therefore, imagine a world quite unlike ours, consisting, as suggested in figure 6.1, of nothing but an endless series of interlocked gears. Suppose, to make this precise, that every gear is constructed so as to mesh with one or more im- mediate neighbors, and that the entire gear universe is intercon- nected, but in such a way that it is still possible for them all to be turned-i.e., so that it does not lock up. Suppose, too, that the gears are perfect: no friction, no play between the teeth, and shaped so that rotating one at an even speed causes the others to rotate evenly as well, though at a potentially &&rent speed.

      The opposite of the flex/slop picture is a world of totally interlocked gears

    25. it is essential to the picture developed so fir, and also an anchor of common sense, that the multi-various parts of the world do not march in lock- step together. The world is fundamentally characterized by an underlying flex or slop-a kind of slack or "play" that allows some bits to move about or adjust without much influencing, and without being much influenced by, other bits. Thus we can play jazz in Helsinki, as loud as we please, without troubling the Trappists in Montana. Moths can fly into the night with only a minima expenditure of energy, because they have to rearrange only a tiny fraction of the world's mass.

      "flex/slop" : the underlying assumption certain bits of the world can adjust without much affecting other parts.

    26. Note too that although there is something right about speaking of individual subjects as the entities or agents that register, this is not to deny that in all likelihood it will be whole cultures, language communities, communities of practice;' or collectivities of people-and-instruments-and- organizations-and-documents-and- tools-and-other-essential- but-expensive-entities that are the I11 sustaining locus of this intentional achievement.

      Registration happening at the level of systems!

    27. Although I will differentiate the two notions in a moment (registration is much broader), there are important ways in which registration is similar to perception. In ordinary parlance, and also as I will understand it, perception is an activity on the part of an intentional subject, and furthermore an activity that relates it to, or engages it with, something in the world. Typically this will be something in the extml world, something outside the subject, something that may even be quite far away. Thus we talk of perceiving a sparrow, a wry look, a distant thunder- cloud-and not, except on higher-order reflection, of perceiv- ing our perceptions or sensations. Furthermore, perception not only relates the subject to that distal object, but also implies a kind of intentional success. Perception, that is to say, is an uchievement

      Perception as similar but differentiable from registration. Perception is an achievement

    28. For now I will leave it at this: that there is nothing in fact, or in logical possibility, or in meta- physical possibility, or accessible to the imagination, or in any other form, for the world to be dependent on, independent of, separated from, or external to. The world has no "other."

      Subject is not independent from the "world"

    29. Why do these theories all fd? The most celebrated difficulties have to do with semantics. It is widely recognized that computation is in one way or an- other a symbolic or representational or information-based or semantical-i.e., as philosophers would say, an intentional- phenomenon.9 Somehow or other, though in ways we do not yet understand, the states of a computer can model or simulate or represent or stand for or carry information about or signify other states in the world (or at least can be taken by people to do so-see the sidebar on original vs. derivative semantics, page 10). This semantical or intentional character of computation is betrayed by such phrases as symbolmanipulation, in@rmation processing, programming hnguages, knowledge representation, htd bases, etc.

      The whole concept of "computation" is underspecified

    30. separation and engagement-is shown to underlie a single notion taken to unify representation and ontology: that of a sub- ject's mgitration of the world.

      a subject's "registration" as the unifier of the physical and abstract worlds

    31. Fundamental to the view is a claim that objects, properties, practice, and politics-indeed everything ontological-live in what is called the "middle distance": an intermediate realm be- tween a proximal though ultimately ineffable connection, remi- niscent of the fmiliar physical bumping and shoving of the world, and a more remote dirronnection, a form of unbridgeable separation that lies at the root of abstraction and of the partial (and painful) subject-object divide. No sense attends to the idea of complete connection or complete disconnection; limit ideal- izations are outmoded.

      The "middle distance" between the physical world (the world of "connection") and the world of abstraction ("disconnection")

    1. although I take seriously many of the critiques of first-wave AI articulated in the 1970s, this book is by no means intended to be an updated treatise along the lines of Dreyfus’s What Computers Can’t Do.7 On the contrary, one of my goals is to develop conceptual resources in terms of which to understand what computers can do. In fact, the entire discussion is intended to be positive. I do not plead for halting the development of AI, or argue that we should bar AI systems from functioning in situations of moral gravity. (When landing in San Francisco, I am glad the airplane is guided by sophisticated computer systems, rather than by pilots peering out through the fog looking for the airport.) I am also not worried, at least here, about whether AI systems will grow more powerful than we hu-mans, or that they will develop their own consciousness. And I take seriously the fact that we will soon need to learn how to live in productive communion with synthetic intel-ligent creatures of our own (and ultimately their) design.

      in terms of other discourses on the ai question. Neither an update to Dreyfus, nor luddite in character.

    2. one of my aims is to unsettle reigning understandings of rationality, in part to break it up into different kinds, but also to sug-gest that reason in its fullest sense—reason of any sort to which we should aspire—necessarily incorporates some of the commitments and compulsions to action that are often associated with affectual states. These moves arise out of a larger commitment: if we are to give the prospect of AIthe importance it deserves, we must not assume that time-honored conceptions of rationality will survive unscathed.

      Reason v. emotion/affect is false dichotomy

    3. judgment in the sense I am defending it is an overarching, systemic capac-ity or commitment, involving the whole commitment of a whole system to a whole world. I do not see it as an isolable property of individual capacities; nor do I believe it is likely to arise from any particular architectural feature—includ-ing any architectural feature missing in current designs. Readers should not expect to find specific architectural suggestions here, or recommendations for technical repair. The issues are deeper, and the stakes higher, than can be reached by any such approach.

      Skirting around any formal definition of judgement?

    4. Yet attempting to reach this conclusion by drawing a distinction between DNA- and silicon-based creatures would be a grave mistake, in my view—chauvinist, sen-timental, and fatally shallow. Rigor demands that we ar-ticulate a space of possible kinds of intelligence in terms of which AIs, humans, and nonhuman animals can all be evenly and nonprejudicially assessed.

      Agrees with me that it is essential that the space of potential intelligences not discriminate on the arbitrary basis of substrate

    5. I see no reason to doubt that it may someday be possible to construct synthetic computational systems capable of genuine judgment.

      Important reminder. He's only arguing about the past and present--not arguing that synthetic "judgement" is impossible outright.

    6. I use the term reckoning for the types of calculative prowess at which computer and AI systems already excel—skills of extraordinary utility and importance, on which there is every reason to suppose computers will continue to advance (ultimately far surpassing us in many cases, where they do not do so already), but skills embodied in devices that lack the ethical commitment, deep contextual aware-ness, and ontological sensitivity of judgment. The differ-ence between reckoning and judgment, which I will argue to be profound, highlights the need for a textured map of intelligence’s kinds6—a map in terms of which to explain why reckoning systems are so astonishingly powerful in some respects, yet fall so spectacularly short in others.

      Reckoning vs. judgement. Argument that AI systems are only capable of the former.

    7. It need not be articulate, “ra-tionalistic,” or independent of creativity, compassion, and generosity—failings of which (especially formal) logic is often accused. Rather, by judgment I mean something like what we get at (or should be getting at) when we say that someone “has good judgment”: a form of thinking that is reliable, just,5 and committed—to truth, to the world as it is.

      Normative judgement need not be "rationalistic"

    8. I use judgment for the normative ideal to which I argue we should hold full-blooded human intelligence—a form of dispassionate2 deliberative thought, grounded in ethi-cal commitment and responsible action, appropriate to the situation in which it is deployed. Not every human cogni-tive act meets this ideal—not even every conscious act of every person we call intelligent. I claim only that judgment is the standard to which human thinking must ultimately aspire.3Judgment of this sort, I believe, is a capacity we strive to instill in our children, and a principle to which we hold adults accountable.

      judgement

    9. With an eye toward such questions, this book develops some intellectual tools with which to assess current devel-opments in AI—especially recent advances in deep learn-ing and second-wave AI that have led to such excitement, anxiety, and debate. The book will not assess specific proj-ects, or recommend incremental advances. Instead, I will adopt a general strategy, in pursuit of two aims:1. To unpack the notion of intelligence itself, in order to understand what sort(s) we humans have, what sort(s) AI aims at, what AI has accomplished so far, what can be expected in the foreseeable future, and what sorts of tasks can responsibly be assigned to systems currently constructed or imagined.2. To develop a better understanding of the underly-ing nature of the world, to which I believe all forms of intelligence are ultimately accountable.

      Finally, someone who wants to offer a definition of intelligence itself before using it to refute the possibility of artificial intelligence!

    1. The infrastructure of programmatic advertising is architected so that, theoretically, publishers are able to sell their inventory of attention to the highest bidder among a pool of buyers. This is generally done through an arrangement known as real-time bidding (RTB). RTB is initiated by the tiniest of actions—clicking on a link or loading a piece of content—which sets off a rapid, orderly cascade of events, Rube Goldberg–style. As soon as an opportunity for delivering an advertisement appears, an ad server leaps into action, announcing to the marketplace the opportunity to bid for inventory.One of the most incredible aspects of the RTB system is that the entire process takes place in real time. The advertisements you see online are not predetermined. At the moment you click the link and load up the page, a signal from the ad server triggers an instantaneous auction to determine which ad will be delivered. The highest bidder gets to load its ad on the website and into your eyeballs.

      RTB = real-time bidding

    2. In the midst of all this criticism, it’s worth taking a moment to think about what precisely advertising is. Media buyers—whether they are a global company, a mom-and-pop shop, or a nefarious band of trolls seeking to influence an election—are all looking to get their message in front of people. Online platforms—whether they are a billion-user social media platform or a small neighborhood newsletter—offer ways to get that message to those people. The platforms sell access to their users, and the buyers pay to acquire that access to distribute their message.Buyers and sellers. At its core, advertising is a marketplace for attention. When your eyes breeze over an advertisement as you scroll through your news feeds or read an article, a transaction has occurred. Your attention has been sold by the platform and bought by the advertiser.

      advertising is a marketplace for attention.

    1. Because philosophy works in cycles and does not move in a straight line, it is preferable to see the history of philosophy as a matter of evolution of the semantic arte-facts we develop to deal with open questions, rather than a matter of progress. In the long run, regardless of how many ups and downs, steps forward and steps back, revolutions and counter-revolutions, progress in science is measured in terms of the accumulation of answers to closed questions, answers that are no longer genuinely open to informed, rational, and honest disagreement.

      Unlike science, philosophy cannot be measured in terms of progress; only in term of "evolution of the semantic artefacts we develop to deal with open questions."

    2. In other contexts, I have spoken about it in terms of semantic artefacts: the totality of the outcome of our creative semanticization (giving meaning to and making sense of ) and conceptual design of reality. Yet this will not do in this chapter, where we need an inclu-sive adjective, comparable to ‘empirical’ and ‘logico-mathematical’ in terms of scope and level of abstraction (more on the latter in the next section). Therfore, let me suggest we opt for noetic.16 It is because of the noetic nature of the resources required to answer philosophical problems that the latter have always tended to be seen more akin to log-ico-mathematical problems than to empirical ones. Let me quote Dummett one more time:Philosophy shares with mathematics the peculiarity that it does not appeal to any new sources of [empirical (my addition)] information but relies solely upon reasoning on the basis of what we already know. (Dummett 2010, p. 10)Dummett’s ‘what we already know’ is part of what I have called noetic resources.

      Defining "semantic artefacts" -> "noetic resources"

    3. In philosophy, there are no experiments and no calculations, for reasons that will soon be obvious. What I wish to borrow, from computational complexity theory, is only the simple, yet very powerful, insight that the nature of problems may be fruitfully studied by focus-ing on what it may take in principle to solve them, rather than on their form, meaning, reference, scope, and relevance.

      Floridi urges us to take the model of the turing machine in shifting our focus from the morphology of the problems we face, to their complexity class

    4. Turing gave us a clear analysis of what an algorithm is. This is crucial in order to shift our perspective on the nature of computational problems, because having a standard way of formulating algorithms means having a universal parameter to calculate the complexity of the problems that such algorithms are supposed to solve. Thus, a Turing Machine works a bit like the standard metre in Paris. The result, a few decades after Turing’s ground-breaking research, is that, in computational theory, one investigates the nature of problems by looking at their complexity, understood as the degree of difficulty of solving them, by studying the resources required to solve them.7 This means that one does not focus on the specific morphology of the problems—because this is where having a universal model, such as a Turing machine, helps us not to be side-tracked—on their semantic features—because we are interested in whole classes of problems independently of their specific content and topic—or on their scope and relevance—because we are interested in their complexity, independently of their timeless applicability. Rather, one investigates the complexity of computational prob-lems by studying the quantity and quality of resources it would take to solve them. This usually means considering the amount of time (number of steps) or space (quantity of memory) required. In this way, it becomes possible to study classes of problems that share the same degree of complexity, and to structure such classes in hierarchies of different degrees of complexity.

      The turing machine is a kind of universal measurement which allows us to think of problems in terms of their complexity, rather than their morphology

    5. The three volumes may be understood as seeking to invert four misconceptions, easily explainable by using the classic communication model introduced by Shannon: sender, message, receiver, channel. Epistemology focuses too much on the passive receiver/consumer of knowledge, when it should be concerned with the active sender/producer, because knowing is constructing. Ethics focuses too much on the sender/agent, when it should be concerned with the receiver/patient, because the keys to being good are care, respect, and tolerance. Metaphysics focuses too much on the relata, the sender/producer/agent/receiver/consumer/patient, which it conceives as entities, when it should be concerned with the message/relations, because dynamic structures constitute the structured. And logic focuses too much on channels of communication as justifying or grounding our conclusions, when it should be concerned with channels that enable us to extract (and transfer) information from a variety of sources reliably, because the logic of information design is a logic of relata and clusters of relations, rather than a logic of things as bearers of predicates.

      Critique of Shannon's classic information model (sender, message, receiver, channel) is critique of mainstream objects of philosophy in general. Epistemology is flawed if it sees the receiver/consumer as knowledge as passive: "because knowledge is constructing." Ethics is focused on the sender/agent, but to shift to the receiver/patient would be more empathetic. Logic "focused too much on channels of communication as justifying or grounding our conclusions, when it should be concerned with channels that enable us to extract (and transfer) information from a variety of sources reliably, because the logic of information design is a logic of relata and clusters of relations, rather than a logic of relata and clusters of relations

    6. Semantic information is well-formed, meaningful, and truthful data; knowledge is relevant semantic infor-mation properly accounted for; humans are the only known semantic engines and conscious informational organisms who can design and understand semantic arte-facts, and thus develop a growing knowledge of reality

      ...and reality is totality of information.

    7. Modelling is not just dealing with what there is; it is often designing what there could, or should be. Having a different word helps here: a blue-print is a model of something that does not yet exist but that we want to design, not of something that is already there and that we want to explain, for example. So, this book is a constructionist study in the conceptual logic of semantic information both as a model(mimesis) and as a blueprint (poiesis). We have reached the full description. And this can now be used to contextualize this book within the wider project for the foundation of the philosophy of information.

      The language of modeling

    8. We always access any reality through specific interfaces, what I shall define in Chapter 2 as levels of abstraction, borrowing a conceptual framework from computer science. The world provides the data, to be understood as constraining affordances, and we transform them into information, like semantic engines. But such transformation or repurposing (see Chapter 4) is not equivalent to portraying, or picturing, or photo-graphing, or photocopying anything. It is more like cooking: the dish does not repre-sent the ingredients, it uses them to make something else out of them, yet the reality of the dish and its properties hugely depend on the reality and the properties of the ingre-dients. Models are not representations understood as pictures, but interpretations understood as data elaborations, of systems. Thus, the whole book may also be read as an articulation and defence of the thesis that knowledge is design, and that philosophy is the ultimate form of conceptual design.

      Floridi is unapologetically Kantian. Knowledge is design.

    1. Faced with all these challenges, humanity will need to be even more intelligent and critical. Complementarity among human and artificial tasks, and successful human–computer interactions will have to be developed. Business models should be revised (advertisement is mostly a waste of resources). It may be necessary to draw clear boundaries between what is what, e.g., in the same way as a restored, ancient vase shows clearly and explicitly where the intervention occurs. New mechanisms for the allocation of responsibility for the production of semantic artefacts will probably be needed. Indeed, copyright legislation was developed in response to the reproducibility of goods. A better digital culture will be required, to make current and future citizens, users and consumers aware of the new infosphere in which they live and work (Floridi 2014a), of the new onlife condition (Floridi 2014b) in it, and hence able to understand and leverage the huge advantages offered by advanced digital solutions such as GPT-3, while avoiding or minimising their shortcomings. None of this will be easy, so we had better start now, at home, at school, at work, and in our societies.

      How humanity will have to step up to the challenge posed by GPT

    2. At the same time, it is reasonable to expect that, thanks to GPT-3-like applications, intelligence and analytics systems will become more sophisticated, and able to identify patterns not immediately perceivable in huge amounts of data. Conversational marketing systems (chatbots) and knowledge management will be able to improve relationships between consumers and producers, customers and companies.

      Potential positive effect of GPT on relationship between customers and companies

    3. On the other hand, fake news and disinformation may also get a boost. For it will be even easier to lie or mislead very credibly (think of style, and choice of words) with automatically-fabricated texts of all kinds (McGuffie and Newhouse 2020). The joining of automatic text production, advertisement-based business models, and the spread of fake news means that the polarization of opinions and the proliferation of “filter bubbles” is likely to increase, because automation can create texts that are increasingly tailored to the tastes and intellectual abilities (or lack thereof) of a reader. In the end, the gullible will delegate to some automatic text producer the last word, like today they ask existential questions to Google.

      GPT will only strengthen the "filter bubble" issue

    4. The amount of texts available will skyrocket because the cost of their production will become negligible, like plastic objects. This huge growth of content will put pressure on the available space for recording (at any given time there is only a finite amount of physical memory available in the world, and data production far exceeds its size). It will also translate into an immense spread of semantic garbage, from cheap novels to countless articles published by predatory journalsFootnote 9: if you can simply push a key and get some “written stuff”, “written stuff” will be published.

      Plasticization of text ; "semantic garbage"

    5. One day classics will be divided between those written only by humans and those written collaboratively, by humans and some software, or maybe just by software. It may be necessary to update the rules for the Pulitzer Prize and the Nobel Prize in literature. If this seems a far-fetched idea consider that regulations about copyright are already adapting. AIVA (Artificial Intelligence Virtual Artist) is an electronic music composer that is recognized by SACEM (Société des auteurs, compositeurs et éditeurs de musique) in France and Luxembourg. Its products are protected by copyright (Rudra 2019).

      Surprising suggestion that in the future even literary texts may be produced wholly and collaboratively with machines

    6. Readers and consumers of texts will have to get used to not knowing whether the source is artificial or human. Probably they will not notice, or even mind—just as today we could not care less about knowing who mowed the lawn or cleaned the dishes. Future readers may even notice an improvement, with fewer typos and better grammar. Think of the instruction manuals and user guides supplied with almost every consumer product, which may be legally mandatory but are often very poorly written or translated. However, in other contexts GPT-3 will probably learn from its human creators all their bad linguistic habits, from ignoring the distinction between “if” and “whether”, to using expressions like “beg the question” or “the exception that proves the rule” incorrectly.

      Getting habituated to text production as just another task that machines can do

    7. People whose jobs still consist in writing will be supported, increasingly, by tools such as GPT-3. Forget the mere cut & paste, they will need to be good at prompt & collate.Footnote 8 Because they will have to learn the new editorial skills required to shape, intelligently, the prompts that deliver the best results, and to collect and combine (collate) intelligently the results obtained, e.g. when a system like GPT-3 produces several valuable texts, which must be amalgamated together, as in the case of the article in The Guardian. We write “intelligently” to remind us that, unfortunately, for those who see human intelligence on the verge of replacement, these new jobs will still require a lot of human brain power, just a different application of it. For example, GPT-3-like tools will make it possible to reconstruct missing parts of texts or complete them, not unlike what happens with missing parts of archaeological artefacts. One could use a GPT-3 tool to write and complete Jane Austen’s Sanditon, not unlike what happened with an AI system that finished the last two movements of Schubert’s Symphony No. 8 (Davis 2019), which Schubert started in 1822 but never completed (only the first two movements are available and fragments of the last two).

      For more sophisticated writing work: from "cut & paste" to "prompt & collate." New skills involved in working with these statistical systems

    8. GPT-3 writes better than many people (Elkins and Chun 2020). Its availability represents the arrival of a new age in which we can now mass produce good and cheap semantic artefacts. Translations, summaries, minutes, comments, webpages, catalogues, newspaper articles, guides, manuals, forms to fill, reports, recipes … soon an AI service may write, or at least draft, the necessary texts, which today still require human effort. It is the biggest transformation of the writing process since the word processor. Some of its most significant consequences are already imaginable.Writers will have less work, at least in the sense in which writing has functioned since it was invented. Newspapers already use software to publish texts that need to be available and updated in real time, such as comments on financial transactions, or on trends of a stock exchange while it is open. They also use software to write texts that can be rather formulaic, such as sports news. Last May, Microsoft announced the sacking of dozens of journalists, replaced by automatic systems for the production of news on MSN (Baker 2020).

      GPT may replace very simple writing jobs

    9. We now live in an age when AI produces excellent prose. It is a phenomenon we have already encountered with photos (Vincent 2020), videos (Balaganur 2019), music (Puiu 2018), painting (Reynolds 2016), poetry (Burgess 2016), and deepfakes as well (Floridi 2018). Of course, as should be clear from the example of Ambrogio and the mowed lawn, all this means nothing in terms of the true “intelligence” of the artificial sources of such remarkable outputs. That said, not being able to distinguish between a human and an artificial source can generate some confusionFootnote 6 and has significant consequences.

      But they haven't given a very strong definition of intelligence to go with this assertion ..

    10. The first iteration of GPT in 2018 used 110 million learning parameters (i.e., the values that a neural network tries to optimize during training). A year later, GPT-2 used 1.5 billion of them. Today, GPT-3 uses 175 billion parameters. It is trained on Microsoft’s Azure’s AI supercomputer (Scott 2020). It is a very expensive training, estimated to have costed $ 12 million (Wiggers 2020). This computational approach works for a wide range of use cases, including summarization, translation, grammar correction, question answering, chatbots, composing emails, and much more.

      GPT's computational scale

    11. Ambrogio mowing the lawn—and producing an outcome that is indistinguishable from anything Alice could achieve—does not make Ambrogio like Alice in any sense, either bodily, cognitively, or behaviourally. This is why “what computers cannot do” is not a convincing title for any publication in the field. It never was. The real point about AI is that we are increasingly decoupling the ability to solve a problem effectively—as regards the final goal—from any need to be intelligent to do so (Floridi 2017).

      We should stop using the word "intelligence" to describe forms of problem solving

    12. Hobbes spent an inordinate amount of time trying to prove how to square the circle. Newton studied alchemy, possibly trying to discover the philosopher’s stone. Turing believed in true Artificial Intelligence, the kind you see in Star Wars. Even geniuses make mistakes.

      "Even geniuses make mistakes."

    13. Semantic questions, precisely because they may produce “reversible” answers, can be used as a test, to identify the nature of their source. Therefore, it goes without saying that it is perfectly reasonable to argue that human and artificial sources may produce indistinguishable answers, because some kinds of questions are indeed irreversible—while at the same time pointing out that there are still (again, more on this qualification presently) some kinds of questions, like semantic ones, that can be used to spot the difference between a human and artificial source. Enter the Turing Test.

      Semantic questions as a natural test of intelligence

    1. Scott Sumner points out that the Covid recession is better described by economic theory than by recent history. It’s a supply shock, whereas most US recessions are demand shocks. In a demand shock recession, the purpose of economic stimulus is to get people spending at their normal level again. In a supply shock, the goal is not to do that—it would be very bad news if the government stimulated the economy so effectively that restaurants, bars, and cruise ships were packed—but to ensure that workers in the affected sectors of the economy can continue to pay their bills until they get back to work.Demand shocks are more of a postmodern phenomenon; they’re a recession about nothing, or a recession whose largest single cause is a drop in spending caused by fears of a recession. Supply shocks are much more real: if some part of the economy physically can’t function, then of course overall output drops, and no policy can perfectly offset those real-world problems. Policy can, however, ensure that a supply shock in one part of the economy doesn’t turn into a demand problem for the rest of it.

      Supply shocks vs. demand shocks: in response to the latter, economic stimulus is meant to encourage spending, whereas for the former, to support workers whose sectors have heave been affected until they can go back to work again.

      Most US recessions have been demand shocks, whereas the Covid recession is a supply shock.

      Demand shocks "are more of a postmodern phenomenon; they're a recession about nothing, or a recession where the largest single cause is a drop in spending caused by fears of a recession. Supply shocks are much more real."

    1. This prediction has one obvious catch: the ability of the US education and job training system (both public and private) to produce the kinds of workers who will thrive in these middle-skill jobs of the future can be called into question. In this and other ways, the issue is not that middle-class workers are doomed by automation and technology, but instead that human capital investment must be at the heart of any long-term strategy for producing skills that are complemented by rather than substituted for by technological change. In 1900, the typical young, native-born American had only a common school education, about the equivalent of sixth to eighth grades. By the late 19th century, many Americans recognized that this level of schooling was inadequate: farm employment was declining, industry was rising, and their children would need additional skills to earn a living. The United States responded to this challenge over the first four decades of the 20th century by becoming the first nation in the world to deliver universal high school education to its citizens (Goldin and Katz 2008). Tellingly, the high school movement was led by the farm states. Societal adjustments to earlier waves of technological advancement were neither rapid, automatic, nor cheap. But they did pay off handsomely.

      Education is key, and may be the failure point for the US economy :(

    2. the rapid employment growth in both high- and low-education jobs has substantially reduced the share of employment accounted for by “middle-skill” jobs. In 1979, the four middle-skill occupations (sales; office and administrative workers; production workers; and operatives) accounted for 60 percent of employment. In 2007, this number was 49 percent, and in 2012, it was 46 percent. The employment share of service occupations was essentially flat between 1959 and 1979, and so their rapid growth since 1980 marks a sharp trend reversal (Autor and Dorn 2013)

      "middle-skill" jobs are declining, as they are the most easily substitutable

    3. Over the very long run, gains in productivity have not led to a shortfall of demand for goods and services: instead, household consumption has largely kept pace with household incomes. We know this because the share of the population engaged in paid employment has generally risen over (at least) the past century despite vast improvements in material standards of living. An average US worker in 2015 wishing to live at the income level of an average worker in 1915 could roughly achieve this goal by working about 17 weeks per year.2 Most citizens would not consider this tradeoff between hours and income desirable, however, suggesting that consumption demands have risen along with productivity. Of course, citizens in high-income countries work fewer annual hours, take more vacations, and retire earlier (relative to death) than a century ago—implying that they choose to spend part of their rising incomes on increased leisure. This is clearly good news on many fronts, but does it also imply that consumption demands are approaching satia-tion? I think not. In high-income countries, consumption and leisure appear to be complements; citizens spend much of their leisure time consuming—shopping, traveling, dining, and, less pleasantly, obtaining medical care

      "gains in productivity have not led to a shortfall of demand for goods and services: instead, household consumption has largely kept pace with household incomes... but [are] consumption demands... approaching satiation? I think not. In high-income countries, consumption and leisure appear to be complements; citizens spend much of their leisure time consuming--shopping, traveling, dining, and, less pleasantly, obtaining medical care." (8)

    4. The best-known early example is the Luddite movement of the early 19th century, in which a group of English textile artisans protested the automation of textile production by seeking to destroy some of the machines. A lesser-known but more recent example is the concern over “The Automation Jobless,” as they were called in the title of a TIME magazine story of February 24, 1961:The number of jobs lost to more efficient machines is only part of the prob-lem. What worries many job experts more is that automation may prevent the economy from creating enough new jobs. . . . Throughout industry, the trend has been to bigger production with a smaller work force. . . . Many of the losses in factory jobs have been countered by an increase in the service industries or in office jobs. But automation is beginning to move in and elimi-nate office jobs too. . . . In the past, new industries hired far more people than those they put out of business. But this is not true of many of today’s new industries. . . . Today’s new industries have comparatively few jobs for the unskilled or semiskilled, just the class of workers whose jobs are being eliminated by automation.

      "The Automation Jobless" as a reference point of automation anxiety between the luddite movement and the present

    1. A study conducted at Princeton University has analyzed to what extent public support for a policy influences the likelihood of that policy being enacted [47] in the United States; for the bottom 90 per cent of the population, their preferences have no influence. Only the preferences of the wealthiest 10 per cent of the population matter. And even within the wealthiest 10 per cent, there is a huge concentration of influence among a small number of individuals. For instance, over a five-year period, the 200 most politically active companies spent nearly $6 billion on lobbying.

      A Princeton study found that the bottom 90% of americans have no influence over policy decisions.