The FTC has accused Kochava of violating the FTC Act by amassing and disclosing "a staggering amount of sensitive and identifying information about consumers," alleging that Kochava's database includes products seemingly capable of identifying nearly every person in the United States. According to the FTC, Kochava's customers, ostensibly advertisers, can access this data to trace individuals' movements—including to sensitive locations like hospitals, temporary shelters, and places of worship, with a promised accuracy within "a few meters"—over a day, a week, a month, or a year. Kochava's products can also provide a "360-degree perspective" on individuals, unveiling personally identifying information like their names, home addresses, phone numbers, as well as sensitive information like their race, gender, ethnicity, annual income, political affiliations, or religion, the FTC alleged.

One of the ways that, that chat G BT is very powerful is that uh if you're sufficiently educated about computers and you want to make a computer program and you can instruct uh chat G BT in what you want with enough specificity, it can write the code for you. It doesn't mean that every coder is going to be replaced by Chad GP T, but it means that a competent coder uh with an imagination can accomplish a lot more than she used to be able to, uh maybe she could do the work of five coders. Um So there's a dynamic where people who can master the technology can get a lot more done.

Up until this point, every publisher had focused on 'traffic at scale', but with the new direct funding focus, every individual publisher realized that traffic does not equal money, and you could actually make more money by having an audience who paid you directly, rather than having a bunch of random clicks for the sake of advertising.The ratio was something like 1:10,000. Meaning that for every one person you could convince to subscribe, donate, become a member, or support you on Patreon ... you would need 10,000 visitors to make the same amount from advertising. Or to put that into perspective, with only 100 subscribers, I could make the same amount of money as I used to earn from having one million visitors.

Again, this promise that personalized advertising would generate better results was just not happening. Every year, the ad performance dropped, and the amount of scale needed to make up for that decline was just out of reach for anyone doing any form of niche.So, yes, the level of traffic was up by a lot, but that still didn't mean we could make more money as smaller publishers. Third party display advertising has just never worked for smaller niche publishers.

We have the difference between amateurs just publishing as a hobby, or professionals publishing as a business. And on the other vector we have whether you are publishing for yourself, or whether you are publishing for others.And these differences create very different focuses. For instance, someone who publishes professionally, but 'for themselves' is a brand. That's what defines a brand magazine.Meanwhile, independent publishers are generally professionals (or trying to be), who are producing a publication for others. In fact, in terms of focus, there is no difference between being an independent publisher and a regular traditional publisher. It's exactly the same focus, just at two very different sizes.Bloggers, however, were mostly amateurs, who posted about things as hobbies, often for their own sake, which is not publishing in the business sense.Finally, we have the teachers. This is the group of people who are not trying to run a publishing business, but who are publishing for the sake of helping others.

There was no automatic advertising delivery. There was no personalization, or any kind of tracking. Instead, I go through all of this every morning, picking which ads I thought looked interesting today, and manually changing and updating the pages on my site.This also meant that, because there was no tracking, the advertising companies had no idea how many times an ad was viewed, and as such, we would only get paid per click.Now, the bigger sites had started to do dynamic advertising, which allowed them to sell advertising per view, but, as an independent publisher, I was limited to only click-based advertising.However, that was actually a good thing. Because I had to pick the ads manually, I needed to be very good at understanding my audience and what they needed when they visited my site. And so there was a link between audience focus and the advertising.Also, because it was click based, it forced me as an independent publisher to optimize for results, whereas a 'per view' model often encouraged publishers to lower their value to create more ad views.

How will people build professional callouses if the early work that may be viewed as mundane essentials are taken over by AI systems? Do we risk living in the age of the last masters, the age of the last experts?

I am even more attuned to creative rights. We can address algorithms of exploitation by establishing creative rights that uphold the four C’s: consent, compensation, control, and credit. Artists should be paid fairly for their valuable content and control whether or how their work is used from the beginning, not as an afterthought.

Generative AI systems that allow for biometric clones can easily exploit our likeness through the creation of synthetic media that propagate deep fakes. We need biometric rights that protect our faces and voices from algorithms of exploitation.

The nightmares of AI discrimination and exploitation are the lived reality of those I call the excoded

AI raises the stakes because now that data is not only used to make decisions about you, but rather to make deeply powerful inferences about people and communities. That data is training models that can be deployed, mobilized through automated systems that affect our fundamental rights and our access to whether you get a mortgage, a job interview, or even how much you’re paid. Thinking individually is only part of the equation now; you really need to think in terms of collective harm. Do I want to give up this data and have it be used to make decisions about people like me—a woman, a mother, a person with particular political beliefs?

the name knot is a direct reference to actual knots tied on the rope. One end of the rope was then attached to a big spool which the rope was wound up on and the other end of the rope was attached to a type of triangular wooden board in a very specific way. The idea was actually pretty simple. The wooden board which was called the chip log was then thrown overboard from the back of the ship and one of the sides of the wooden ship had a lead weight to keep it vertical on the surface of the sea. As the ship then sped forward, the wood was supposed to stay mostly still on the surface of the water, with the rope quickly unwinding from the spool. On the rope, there were knots spaced out on regular intervals either every 47 feet and 3 inches or every 48 ft, depending on which source you use. The idea was that the sailors would use a small hourglass which measured either 28 seconds or 30 seconds, again, depending on the source and they would count how many knots went by in that time and the answer that they then derived from this exercise was the ship's speed, measured in literal knots

one being that a nautical mile is the meridian arc length, corresponding to one minute of a degree of latitude. In other words, the full circumference of the earth is 360° and a nautical mile is one 60th of one degree at the equator but that's a historical definition. Today, a nautical mile is defined using the metric system like everything else and it is exactly 1,852 meters.

the United States officially started defining its own units by using the metric system as a reference, way back in 1893 with something called the Mendenhall Order. And that means that, for example, today, the inch is not defined as the length of three barley corns which genuinely was its official definition for centuries, instead the inch is now officially defined as 25.4 millimeters and all other US customary units have similar metric definitions

Some people thought that the Control Number represented a mechanism for identifying a record as having originated with OCLC and therefore subject to the cooperative’s record use policy.

Recently we recommended that OCLC declare OCLC Control Numbers (OCN) as dedicated to the public domain. We wanted to make it clear to the community of users that they could share and use the number for any purpose and without any restrictions. Making that declaration would be consistent with our application of an open license for our own releases of data for re-use and would end the needless elimination of the number from bibliographic datasets that are at the foundation of the library and community interactions. I’m pleased to say that this recommendation got unanimous support and my colleague Richard Wallis spoke about this declaration during his linked data session during the recent IFLA conference. The declaration now appears on the WCRR web page and from the page describing OCNs and their use.

It's noteworthy that RFC 7258 doesn't consider that bad actors are limited to governments, and personally, I think many advertising industry schemes for collecting data are egregious examples of pervasive monitoring and hence ought also be considered an attack on the Internet that ought be mitigated where possible. However, the Internet technical community clearly hasn't acted in that way over the last decade.

Many have written about how being under constant surveillance changes a person. When you know you're being watched, you censor yourself. You become less open, less spontaneous. You look at what you write on your computer and dwell on what you've said on the telephone, wonder how it would sound taken out of context, from the perspective of a hypothetical observer. You're more likely to conform. You suppress your individuality. Even though I have worked in privacy for decades, and already knew a lot about the NSA and what it does, the change was palpable. That feeling hasn't faded. I am now more careful about what I say and write. I am less trusting of communications technology. I am less trusting of the computer industry.

A second, complementary, approach relies on post-hoc machine learning and forensic anal-ysis to passively identify statistical and physical artifacts left behind by media manipulation.For example, learning-based forensic analysis techniques use machine learning to automati-cally detect manipulated visual and auditory content (see e.g. [94]). However, these learning-based approaches have been shown to be vulnerable to adversarial attacks [95] and contextshift [96]. Artifact-based techniques exploit low-level pixel artifacts introduced during synthe-sis. But these techniques are vulnerable to counter-measures like recompression or additivenoise. Other approaches involve biometric features of an individual (e.g., the unique motionproduced by the ears in synchrony with speech [97]) or behavioral mannerisms [98]). Biomet-ric and behavioral approaches are robust to compression changes and do not rely on assump-tions about the moment of media capture, but they do not scale well. However, they may bevulnerable to future generative-AI systems that may adapt and synthesize individual biometricsignals.

tabula rasa

he new tools have sparked employment concerns for creative occupations suchas composers, designers, and writers. This conflict arises because SBTC fails to differentiatebetween cognitive activities like analytical work and creative ideation. Recent research [82, 83]demonstrates the need to quantify the specific activities of various artistic workers before com-paring them to the actual capabilities of technology. A new framework is needed to characterize

First, under a highly permissive view, theuse of training data could be treated as non-infringing because protected works are not directlycopied. Second, the use of training data could be covered by a fair-use exception because atrained AI represents a significant transformation of the training data [63, 64, 65, 66, 67, 68].1Third, the use of training data could require an explicit license agreement with each creatorwhose work appears in the training dataset. A weaker version of this third proposal, is to atleast give artists the ability to opt-out of their data being used for generative AI [69]. Finally,a new statutory compulsory licensing scheme that allows artworks to be used as training databut requires the artist to be remunerated could be introduced to compensate artists and createcontinued incentives for human creation [70].

AI-generated content may also feed future generative models, creating a self-referentialaesthetic flywheel that could perpetuate AI-driven cultural norms. This flywheel may in turnreinforce generative AI’s aesthetics, as well as the biases these models exhibit.

In traditional artforms characterized by direct manipulation [32]of a material (e.g., painting, tattoo, or sculpture), the creator has a direct hand in creating thefinal output, and therefore it is relatively straightforward to identify the creator’s intentions andstyle in the output. Indeed, previous research has shown the relative importance of “intentionguessing” in the artistic viewing experience [33, 34], as well as the increased creative valueafforded to an artwork if elements of the human process (e.g., brushstrokes) are visible [35].However, generative techniques have strong aesthetics themselves [36]; for instance, it hasbecome apparent that certain generative tools are built to be as “realistic” as possible, resultingin a hyperrealistic aesthetic style. As these aesthetics propagate through visual culture, it can bedifficult for a casual viewer to identify the creator’s intention and individuality within the out-puts. Indeed, some creators have spoken about the challenges of getting generative AI modelsto produce images in new, different, or unique aesthetic styles [36, 37].

n order to be considered meaningful human control, a generative system shouldbe capable of incorporating a human author’s intent into its output. If a user starts with no spe-cific goal, the system should allow for open-ended, curiosity-driven exploration. As the user’sgoal becomes clearer through interaction, the system should be able to both guide and deliverthis intent. Such systems should have a degree of predictability, allowing users to graduallyunderstand the system to the extent that they can learn to anticipate the results of their actions.Given these conditions, we can consider the human user as accountable for the outputs of thegenerative system. In other words, MHC is achieved if human creators can creatively expressthemselves through the generative system, leading to an outcome that aligns with their inten-tions and carries their personal, expressive signature. Future work is needed to investigate inwhat ways generative systems and interfaces can be developed that allow more meaningful hu-man control by adding input streams that provide users fine-grained causal manipulation overoutputs.

Anthropomorphizing AI can pose challenges to the ethicalusage of this technology [12]. In particular, perceptions of human-like agency can underminecredit to the creators whose labor underlies the system’s outputs [13] and deflect responsibil-ity from developers and decision-makers when these systems cause harm [14]. We, therefore,discuss generative AI as a tool to support human creators [15], rather than an agent capableof harboring its own intent or authorship. In this view, there is little room for autonomousmachines being “artists” or “creative” in their own right.

Unlike past disruptions, however, generative AI relies on training data made by people

Generative AI tools, at first glance, seem to fully automate artistic production—an impres-sion that mirrors past instances when traditionalists viewed new technologies as threatening“art itself.” In fact, these moments of technological change did not indicate the “end of art,” buthad much more complex effects, recasting the roles and practices of creators and shifting theaesthetics of contemporary media [3].

The original law stipulated a fifty–fifty split of the regular minimum wage: employers paid a subminimum cash wage that was half of the regular minimum wage; the other half was provided via customers’ tips. Even today, many customers do not know that a tip intended as a gratuity is often a wage-subsidy provided to the employer.

The Pullman Train Company, for instance, hired many formerly enslaved people and fought hard to keep paid wages very low. When investigated by the Railroad Commission of California in 1914, Pullman argued that they “paid adequate wages and did not expect their employees to exact tips”—an assertion unfounded by payroll data and strongly rejected by the commission.[3] Pullman, in fact, left it to the mostly white customers to determine the workers’ compensation through voluntary and unpredictable tips.

The practice of tipping is traced to the Middle Ages and the European feudal system, when masters would sporadically give pocket change to their servants. The practice outlasted the feudal era, becoming a habit between customers—often upper class—and service workers. It also spread more generally. The modern custom of tipping was imported to the United States in the nineteenth century by upper-class American travelers to Europe. At the same time, an influx of European workers more acquainted with the practice helped to establish and spread the practice of tipping in the United States.

which cost around 1.5 million euros

in that sense the internet is very decentralized that the control of the network is up to whoever owns the network the two percent where you said there's nobody in control there's a little bit of centralized control there's an organization called the internet Corporation for assigned names and numbers

if you think about sending letters through the US Postal Service how you've got an address on it so every packet that flows from the Netflix server to you has an address on it says this is going to Jenna it's going to the What's called the Internet Protocol address of your device think of all the the range of devices that are hooked up to the Internet it's totally amazing right every single one of them has one thing in common and that is they speak the IP protocol the Internet Protocol

the internet is a lot like that it's an interconnection of local roads local networks like the network in your house for example how does like all of the um networks in my house connect to all the city networks

Protocols are you up for one yeah knock knock who's there lettuce lettuce who let us go on a knock knock joke is an example of a protocol

But Reader was also very much a product of Google’s infrastructure. Outside Google, there wouldn’t have been access to the company’s worldwide network of data centers, web crawlers, and excellent engineers. Reader existed and worked because of Google’s search stack, because of the work done by Blogger and Feedburner and others, and most of all, the work done by dozens of Google employees with 20 percent of their time to spare and some ideas about how to make Reader better. Sure, Google killed Reader. But nearly everyone I spoke to agreed that without Google, Reader could never have been as good as it was.

At its peak, Reader had just north of 30 million users, many of them using it every day. That’s a big number — by almost any scale other than Google’s. Google scale projects are about hundreds of millions and billions of users, and executives always seemed to regard Reader as a rounding error. Internally, lots of workers used and loved it, but the company’s leadership began to wonder whether Reader was ever going to hit Google scale. Almost nothing ever hits Google scale, which is why Google kills almost everything.

One feature took off immediately, for power users and casual readers alike: a simple sharing system that let users subscribe to see someone else’s starred items or share their collection of subscriptions with other people. The Reader team eventually built comments, a Share With Note feature, and more.

It wasn’t until the team launched a redesign in 2006 that added infinite scrolling, unread counts, and some better management tools for heavy readers that Reader took off.

he bristles thinking about the fight and the fact that Google Reader is known as “an RSS reader” and not the ultra-versatile information machine it could have become. Names matter, and Reader told everyone that it was for reading when it could have been for so much more

think that the folks at the top of the  org chart know more than we do and they often do   have helpful holistic perspective about the  company and the industry it operates in but   all of the actual work of an organization all of  its output happens at the bottom of the org chart   and the teams at the edge of the organization  leaders at the top may have a wide perspective   but the edges are where an organization's  detailed knowledge lives

documenting the steps conversations tools and  other activities required to complete a Core   Business activity like processing an insurance  claim or launching a new product these Maps were   always surprising for the teams that created them  and they often raised existential questions about   employees roles in the organization the  reason was that they revealed the actual   structure of the organization the relationships  the information Pathways that were responsible   for the organization actually being able to get  work done they learned the actual structure of the   organization was an organic emergent phenomenon  constantly shifting and changing based on the work   to be done and often bearing little resemblance  to the formal hierarchy of the organization

paper published by  Dr Ruth Ann heising has some insight for us

In the intro to thinking and its, meadows  says everyone on everything in a system   can act dutifully and rationally,  yet all of these well meaning actions   often lead up to a perfectly terrible  result. This is what happened here.   Every actor in the system borrowed a little  bit of safety to optimize something else

n her fantastic  book, thinking in systems, we are given great   tools to pick apart the situation using systems  thinking. Meadows introduces the stocks and flows.

one   of the first I learned to spot was the Boeing 737.  It's the best selling commercial aircraft of all   time, it's everywhere. Once you know the trick,  it's incredibly easy to identify in the air.   It has no doors over the landing gear, if it's too  big to be a regional, it's a 737. You can see how   the gears swing out from the wheel shaped  cavities in the center of the fuselage.

On the advice of my lawyer, I respectfullydecline to answer on the basis of the FifthAmendment, which—according to theUnited States Supreme Court—protects ev-eryone, even innocent people, from the needto answer questions if the truth might beused to help create the misleading impres-sion that they were somehow involved in acrime that they did not commit.

to what extent is the vendor community adopting bib frame in their systems architectures

10% more or less of academic libraries in the US use an open source system after all that time. And about either 17 or 14, I'd have the number in front of me for and to public libraries are using an open source I L S

The industry has changed from being fragmented to consolidated

Such global and regional climate statementsdiffer from attributing specific extremeweather events to specific human influences,which scientists once considered infeasiblewith then-existing data and methods.18 Thischanged with the publication of an article in2003 proposing a method of establishing legalliability for climate change by determining how much human influence had changed theprobability of an undesirable event, such as flooding.

The use of probability and risk interchangeably can causeconfusion. For example, two common methods toestimate the probability of occurrence of a naturalhazard event include the term risk in their names: theRisk Ratio (RR) and the Fraction of Attributable Risk(FAR). In this report, when referring to RR and FAR,the term risk refers to the climatic or meteorologicalprobability of an event of a specific magnitude, not tothe potential impact of the event on human systems.Apart from discussing these specific terms that use riskin their definitions, this report uses the term hazard asthe probability of a particular event occurring, such as ahurricane, and risk as the hazard combined with thevulnerability of humans and human systems to thathazard. In this sense, the risk is the likelihood ofadverse outcomes from the hazard. For example, thehazard of a major hurricane striking the Florida coasttoday and 100 years ago may be the same, but the riskis much higher today because of the growth in theamount of exposed infrastructure.

Climate change attribution is the study of whether, or to what degree, human influence may havecontributed to extreme climate or weather events.Advances in the science of climate attribution now allow scientists to make estimates of thehuman contribution to such events.

Twitter is a neat illustration of the problem with benevolent dictatorships: they work well, but fail badly. Because they are property — not protocols — they can change hands, and overnight, you get a new, malevolent dictator who wants to retool the system for extraction, rather than collaboration.

LLMs predictably get more capable withincreasing investment, even withouttargeted innovation

Discussion and Limitations

Brief interactions with LLMs are oftenmisleading

LLMs need not express the values of theircreators nor the values encoded in web text

Human performance on a task isn’t anupper bound on LLM performance

Experts are not yet able to interpret theinner workings of LLMs

There are no reliable techniques forsteering the behavior of LLMs

LLMs often appear to learn and userepresentations of the outside world

Specific important behaviors in LLM tendto emerge unpredictably as a byproduct ofincreasing investment

study co-authored by scientists at the Allen Institute for AI,

Do AI Outputs Infringe Copyrights in Other Works?

Does the AI Training Process Infringe Copyright in Other Works?

Who Owns the Copyright to Generative AI Outputs?

Do AI Outputs Enjoy Copyright Protection?

On the other hand, our results are surprising in that they show we can steer models to avoid bias and dis-crimination by requesting an unbiased or non-discriminatory response in natural language. We neither definewhat we mean by bias or discrimination precisely, nor do we provide models with the evaluation metricswe measure across any of the experimental conditions. Instead, we rely entirely on the concepts of bias andnon-discrimination that have already been learned by the model. This is in contrast to classical machinelearning models used in automated decision making, where precise definitions of fairness must be describedin statistical terms, and algorithmic interventions are required to make models fair.

Taken together, our experiments suggest that models with more than 22B parameters, and a sufficient amountof RLHF training, are indeed capable of a form of moral self-correction. In some ways, our findings areunsurprising. Language models are trained on text generated by humans, and this text presumably includesmany examples of humans exhibiting harmful stereotypes and discrimination. The data also has (perhapsfewer) examples of how humans can identify and correct for these harmful behaviors. The models can learnto do both.

In an analysis of LLMs released last June, researchers at Anthropic looked at whether the models would show certain types of racial or social biases, not unlike those previously reported in non-LLM-based algorithms used to predict which former criminals are likely to commit another crime. That study was inspired by an apparent paradox tied directly to emergence: As models improve their performance when scaling up, they may also increase the likelihood of unpredictable phenomena, including those that could potentially lead to bias or harm.

But the researchers quickly realized that a model’s complexity wasn’t the only driving factor. Some unexpected abilities could be coaxed out of smaller models with fewer parameters — or trained on smaller data sets — if the data was of sufficiently high quality. In addition, how a query was worded influenced the accuracy of the model’s response.

In 2020, Dyer and others at Google Research predicted that LLMs would have transformative effects — but what those effects would be remained an open question. So they asked the research community to provide examples of difficult and diverse tasks to chart the outer limits of what an LLM could do. This effort was called the Beyond the Imitation Game Benchmark (BIG-bench) project, riffing on the name of Alan Turing’s “imitation game,” a test for whether a computer could respond to questions in a convincingly human way. (This would later become known as the Turing test.) The group was especially interested in examples where LLMs suddenly attained new abilities that had been completely absent before.

Many of these emergent behaviors illustrate “zero-shot” or “few-shot” learning, which describes an LLM’s ability to solve problems it has never — or rarely — seen before.

In 2017, researchers at Google Brain introduced a new kind of architecture called a transformer. While a recurrent network analyzes a sentence word by word, the transformer processes all the words at the same time. This means transformers can process big bodies of text in parallel.

Biologists, physicists, ecologists and other scientists use the term “emergent” to describe self-organizing, collective behaviors that appear when a large collection of things acts as one. Combinations of lifeless atoms give rise to living cells; water molecules create waves; murmurations of starlings swoop through the sky in changing but identifiable patterns; cells make muscles move and hearts beat. Critically, emergent abilities show up in systems that involve lots of individual parts. But researchers have only recently been able to document these abilities in LLMs as those models have grown to enormous sizes.

It’s surprising because these models supposedly have one directive: to accept a string of text as input and predict what comes next, over and over, based purely on statistics. Computer scientists anticipated that scaling up would boost performance on known tasks, but they didn’t expect the models to suddenly handle so many new, unpredictable ones.

For example, when an AI technology receives solely a prompt [27] from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user.

And in the current edition of the Compendium, the Office states that “to qualify as a work of `authorship' a work must be created by a human being” and that it “will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author.”

The Court defined an “author” as “he to whom anything owes its origin; originator; maker; one who completes a work of science or literature.” [14] It repeatedly referred to such “authors” as human, describing authors as a class of “persons” [15] and a copyright as “the exclusive right of a man to the production of his own genius or intellect.”

In the Office's view, it is well-established that copyright can protect only material that is the product of human creativity. Most fundamentally, the term “author,” which is used in both the Constitution and the Copyright Act, excludes non-humans. The Office's registration policies and regulations reflect statutory and judicial guidance on this issue.

we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability. The numbers don't lie.

Techies will complain that trivial problems of life in the Bay Area are hard because they involve politics. But they should involve politics. Politics is the thing we do to keep ourselves from murdering each other. In a world where everyone uses computers and software, we need to exercise democratic control over that software.

Companies that perform surveillance are attempting the same mental trick. They assert that we freely share our data in return for valuable services. But opting out of surveillance capitalism is like opting out of electricity, or cooked foods—you are free to do it in theory. In practice, it will upend your life.

We started out collecting this information by accident, as part of our project to automate everything, but soon realized that it had economic value. We could use it to make the process self-funding. And so mechanized surveillance has become the economic basis of the modern tech industry.

First we will instrument, then we will analyze, then we will optimize. And you will thank us. But the real world is a stubborn place. It is complex in ways that resist abstraction and modeling. It notices and reacts to our attempts to affect it. Nor can we hope to examine it objectively from the outside, any more than we can step out of our own skin. The connected world we're building may resemble a computer system, but really it's just the regular old world from before, with a bunch of microphones and keyboards and flat screens sticking out of it. And it has the same old problems. Approaching the world as a software problem is a category error that has led us into some terrible habits of mind.

this intellectual background can also lead to arrogance. People who excel at software design become convinced that they have a unique ability to understand any kind of system at all, from first principles, without prior training, thanks to their superior powers of analysis. Success in the artificially constructed world of software design promotes a dangerous confidence.

the apocalypse they refer to is not some kind of sci-fi takeover like Skynet, or whatever those researchers thought had a 10 percent chance of happening. They’re not predicting sentient evil robots. Instead, they warn of a world where the use of AI in a zillion different ways will cause chaos by allowing automated misinformation, throwing people out of work, and giving vast power to virtually anyone who wants to abuse it. The sin of the companies developing AI pell-mell is that they’re recklessly disseminating this mighty force.

The required behavior of a large language model for something like search is very different than for something that’s just meant to be a playful chatbot. We need to figure out how we walk the line between all these different uses, creating something that’s useful for people across a range of contexts, where the desired behavior might really vary

Every time we have a better model, we want to put it out and test it. We’re very optimistic that some targeted adversarial training can improve the situation with jailbreaking a lot. It’s not clear whether these problems will go away entirely, but we think we can make a lot of the jailbreaking a lot more difficult. Again, it’s not like we didn’t know that jailbreaking was possible before the release. I think it’s very difficult to really anticipate what the real safety problems are going to be with these systems once you’ve deployed them. So we are putting a lot of emphasis on monitoring what people are using the system for, seeing what happens, and then reacting to that. This is not to say that we shouldn’t proactively mitigate safety problems when we do anticipate them. But yeah, it is very hard to foresee everything that will actually happen when a system hits the real world.

Since November, OpenAI has already updated ChatGPT several times. The researchers are using a technique called adversarial training to stop ChatGPT from letting users trick it into behaving badly (known as jailbreaking). This work pits multiple chatbots against each other: one chatbot plays the adversary and attacks another chatbot by generating text to force it to buck its usual constraints and produce unwanted responses. Successful attacks are added to ChatGPT’s training data in the hope that it learns to ignore them.

it will be important that we can all ramp up the capacity of extracting and removing CO2 from the atmosphere. Now, when thinking about the latter, so far I've spoken about technology, but we shouldn't forget that also, nature offers several solutions to extract carbon from the air, such as forests and oceans. And one element that is very important will be doubling down on these methods offered by nature, enhancing them and protecting them.

there is not a lot of CO2 in the air. We're currently at around 420 ppm. That means one molecule out of 2,500 molecules in the air around us is CO2. That's not a lot. And that means to extract only one ton of CO2 from the air, we need to filter around two million cubic meters of air. That's about 800 Olympic swimming pools.

This is Orca. This is the first worldwide commercial direct air capture and storage plant. It is in Iceland, and it is an industrial plant that extracts CO2 out of ambient air. We have operated now for more than one year. It costs more than 10 million dollars to build Orca. And its modules, those eight boxes that we call CO2 collectors, they are designed to extract a bit more than 10 tons of carbon dioxide from the air every day.

Such regulation is already being pursued in Europe, where the DigitalServices Act would require large platforms to interoperate, a requirementthat could easily be modified to include the Fediverse.

A different concern with decentralized moderation is that it will leadto “filter bubbles” and “echo chambers” by which instance members willchoose to only interact with like-minded users.

The benefit of decentralized moderation is that it can satisfy both thosethose that want to speak and those that don’t want to listen. By empoweringusers, through their choice of instance, to avoid content they find objection-able, the Fediverse operationalizes the principle that freedom of speech is notthe same as freedom of reach

And if effective moderation turns out to requiremore infrastructure, that could lead to a greater consolidation of instances.This is what happened with email, which, in part due to the investmentsnecessary to counter spam, has become increasingly dominated by Googleand Microsoft.

Gab is a useful case study in how decentralized social media can self-police. On the one hand, there was no way for Mastodon to expel Gabfrom the Fediverse. As Mastodon’s founder Eugen Rochko explained, “Youhave to understand it’s not actually possible to do anything platform-widebecause it’s decentralized. . . . I don’t have the control.”32 On the other hand,individual Mastodon instances could—and the most popular ones did—refuse to interact with the Gab instance, effectively cutting it off from mostof the network in a spontaneous, bottom-up process of instance-by-instancedecisionmaking. Ultimately, Gab was left almost entirely isolated, with morethan 99% of its users interacting only with other Gab users. Gab respondedby “defederating”: voluntarily cutting itself off from the remaining instancesthat were still willing to communicate with it.

https://perma.cc/G82J-73WX

Mastodon instances thus operate according to the principle of content-moderation subsidiarity: content-moderation standards are set by, and differacross, individual instances. Any given Mastodon instance may have rulesthat are far more restrictive than those of the major social media platforms.But the network as a whole is substantially more speech protective thanare any of the major social media platforms, since no user or content canbe permanently banned from the network and anyone is free to start aninstance that communicates both with the major Mastodon instances and theperipheral, shunned instances.

The Fediverse indeed does, because its decentralization is a matter ofarchitecture, not just policy. A subreddit moderator has control only insofar asReddit, a soon-to-be public company,22 permits that control. Because Redditcan moderate any piece of content—indeed, to ban a subreddit outright—no matter whether the subreddit moderator agrees, it is subject to publicpressure to do so. Perhaps the most famous example is Reddit’s banning ofthe controversial pro-Trump r/The_Donald subreddit several months beforethe 2020 election.

When a user decides to moves instances, theymigrate their account data—including their blocked, muted, and followerusers lists and post history—and their followers will automatically refollowthem at their new account.

content-moderation subsidiarity. Just asthe general principle of political subsidiarity holds that decisions should bemade at the lowest organizational level capable of making such decisions,15content-moderation subsidiarity devolves decisions to the individual in-stances that make up the overall network.

moderator’s trilemma. The first prong is that platform userbases arelarge and diverse. The second prong is that the platforms use centralized,top-down moderation policies and practices. The third prong is that theplatforms would like to avoid angering large swaths of their users. But thepast decade of content moderation controversies suggests that these threegoals can’t all be met. The large closed platforms are unwilling to shrink theiruser bases or give up control over content moderation, so they have tacitlyaccepted high levels of dissatisfaction with their moderation decisions. TheFediverse, by contrast, responds to the moderator’s trilemma by giving upon centralized moderation.

An early challenge to the open Internet came from the first generation of giant onlineservices providers like America Online, Compuserve, and Prodigy, which combineddial-up Internet access with an all-encompassing web portal that provided bothInternet content and messaging. But as Internet speeds increased and web browsingimproved, users discovered that the limits of these closed systems outweighed theirbenefits, and they faded into irrelevance by the 2000s.

A core architectural building block of the Internet is the open protocol. A protocolis the rules that govern the transmission of data. The Internet consists of manysuch protocols, ranging from those that direct the physical transmission ofdata to those that govern the most common Internet applications, like emailor web browsing. Crucially, all these protocols are open, in that anyone canset up and operate a router, website, or email server without needing toregister with or get permission from a central authority.5 Open protocolswere key to the first phase of the Internet’s growth because they enabledunfettered access, removing barriers and bridging gaps between differentcommunities. This enabled and encouraged interactions between groupswith various interests and knowledge, resulting in immense creativity andidea-sharing.

There is a limit to how heatedthe debates around email content moderation can be, because there’s anarchitectural limit to how much email moderation is possible. This raises theintriguing possibility of what social media, and its accompanying content-moderation issues, would look like if it, too, operated as a decentralizedprotocol.

Roughly 700 prompt engineers now use PromptBase to sell prompts by commission for buyers who want, say, a custom script for an e-book or a personalized “motivational life coach.” The freelance site Fiverr offers more than 9,000 listings for AI artists; one seller offers to “draw your dreams into art” for $5.

Some AI experts argue that these engineers only wield the illusion of control. No one knows how exactly these systems will respond, and the same prompt can yield dozens of conflicting answers — an indication that the models’ replies are based not on comprehension but on crudely imitating speech to resolve tasks they don’t understand.“Whatever is driving the models’ behavior in response to the prompts is not a deep linguistic understanding,” said Shane Steinert-Threlkeld, an assistant professor in linguistics who is studying natural language processing at the University of Washington. “They explicitly are just telling us what they think we want to hear or what we have already said. We’re the ones who are interpreting those outputs and attributing meaning to them.”He worried that the rise of prompt engineering would lead people to overestimate not just its technical rigor but also the reliability of the results anyone could get from a deceptive and ever-changing black box.“It’s not a science,” he said. “It’s ‘let’s poke the bear in different ways and see how it roars back.’”

prompt engineer. His role involves creating and refining the text prompts people type into the AI in hopes of coaxing from it the optimal result. Unlike traditional coders, prompt engineers program in prose, sending commands written in plain text to the AI systems, which then do the actual work.

“Now you’re getting one of the most important points,” Lemoine said. “Whether these things actually are people or not — I happen to think they are; I don’t think I can convince the people who don’t think they are — the whole point is you can’t tell the difference. So we are going to be habituating people to treat things that seem like people as if they’re not.”

But the road from language model to existential crisis is short indeed. Joseph Weizenbaum, who created ELIZA, the first chatbot, in 1966, spent most of the rest of his life regretting it. The technology, he wrote ten years later in Computer Power and Human Reason, raises questions that “at bottom … are about nothing less than man’s place in the universe.” The toys are fun, enchanting, and addicting, and that, he believed even 47 years ago, will be our ruin: “No wonder that men who live day in and day out with machines to which they believe themselves to have become slaves begin to believe that men are machines.”

Bender and Manning’s biggest disagreement is over how meaning is created — the stuff of the octopus paper. Until recently, philosophers and linguists alike agreed with Bender’s take: Referents, actual things and ideas in the world, like coconuts and heartbreak, are needed to produce meaning. This refers to that. Manning now sees this idea as antiquated, the “sort of standard 20th-century philosophy-of-language position.” “I’m not going to say that’s completely invalid as a position in semantics, but it’s also a narrow position,” he told me. He advocates for “a broader sense of meaning.” In a recent paper, he proposed the term distributional semantics: “The meaning of a word is simply a description of the contexts in which it appears.” (When I asked Manning how he defines meaning, he said, “Honestly, I think that’s difficult.”)

stochastic parrot (coinage Bender’s) is an entity “for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning.” In March 2021, Bender published “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” with three co-authors. After the paper came out, two of the co-authors, both women, lost their jobs as co-leads of Google’s Ethical AI team. The controversy around it solidified Bender’s position as the go-to linguist in arguing against AI boosterism.

OpenAI also contracted out what’s known as ghost labor: gig workers, including some in Kenya (a former British Empire state, where people speak Empire English) who make $2 an hour to read and tag the worst stuff imaginable — pedophilia, bestiality, you name it — so it can be weeded out. The filtering leads to its own issues. If you remove content with words about sex, you lose content of in-groups talking with one another about those things.

Our List of Dirty, Naughty, Obscene, and Otherwise Bad Word

Tech-makers assuming their reality accurately represents the world create many different kinds of problems. The training data for ChatGPT is believed to include most or all of Wikipedia, pages linked from Reddit, a billion words grabbed off the internet

It can’t include, say, e-book copies of everything in the Stanford library, as books are protected by copyright law.

“Systematic Approaches to Learning Algorithms and Machine Inferences.” Then people would be out here asking, “Is this SALAMI intelligent? Can this SALAMI write a novel? Does this SALAMI deserve human rights?”

“We call on the field to recognize that applications that aim to believably mimic humans bring risk of extreme harms,” she co-wrote in 2021. “Work on synthetic human behavior is a bright line in ethical Al development, where downstream effects need to be understood and modeled in order to block foreseeable harm to society and different social groups.”

We go around assuming ours is a world in which speakers — people, creators of products, the products themselves — mean to say what they say and expect to live with the implications of their words. This is what philosopher of mind Daniel Dennett calls “the intentional stance.” But we’ve altered the world. We’ve learned to make “machines that can mindlessly generate text,” Bender told me when we met this winter. “But we haven’t learned how to stop imagining the mind behind it.”

The models are built on statistics. They work by looking for patterns in huge troves of text and then using those patterns to guess what the next word in a string of words should be. They’re great at mimicry and bad at facts. Why? LLMs, like the octopus, have no access to real-world, embodied referents. This makes LLMs beguiling, amoral, and the Platonic ideal of the bullshitter, as philosopher Harry Frankfurt, author of On Bullshit, defined the term. Bullshitters, Frankfurt argued, are worse than liars. They don’t care whether something is true or false. They care only about rhetorical power — if a listener or reader is persuaded.

Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data

Say that A and B, both fluent speakers of English, are independently stranded on two uninhabited islands. They soon discover that previous visitors to these islands have left behind telegraphs and that they can communicate with each other via an underwater cable. A and B start happily typing messages to each other. Meanwhile, O, a hyperintelligent deep-sea octopus who is unable to visit or observe the two islands, discovers a way to tap into the underwater cable and listen in on A and B’s conversations. O knows nothing about English initially but is very good at detecting statistical patterns. Over time, O learns to predict with great accuracy how B will respond to each of A’s utterances. Soon, the octopus enters the conversation and starts impersonating B and replying to A. This ruse works for a while, and A believes that O communicates as both she and B do — with meaning and intent. Then one day A calls out: “I’m being attacked by an angry bear. Help me figure out how to defend myself. I’ve got some sticks.” The octopus, impersonating B, fails to help. How could it succeed? The octopus has no referents, no idea what bears or sticks are. No way to give relevant instructions, like to go grab some coconuts and rope and build a catapult. A is in trouble and feels duped. The octopus is exposed as a fraud.

I have no doubt that robber barons would have engaged in zuckerbergian shenanigans if they could have – but here we run up against the stubborn inertness of atoms and the slippery liveliness of bits. Changing a railroad schedule to make direct connections with cities where you want to destroy a rival ferry business (or hell, laying track to those cities) is a slow proposition. Changing the content recommendation system at Facebook is something you do with a few mouse-clicks.

Enshittification, you'll recall, is the lifecycle of the online platform: first, the platform allocates surpluses to end-users; then, once users are locked in, those surpluses are taken away and given to business-customers. Once the advertisers, publishers, sellers, creators and performers are locked in, the surplus is clawed away from them and taken by the publishers.

In ChatGPT and Bing’s conversations about themselves, you see evidence of the corpus everywhere: the sci-fi, the news articles with boilerplate paragraphs about machine uprisings, the papers about what AI researchers are working on. You also see evidence of the more rigorous coverage and criticism of OpenAI, etc., that has elucidated possible harms that could result from the careless deployment of AI tools.

Tay is from a different generation of AI and bears little technical resemblance to something like the new Bing. Still, it was orders of magnitude more sophisticated, and less technologically comprehensible to its users, than something like ELIZA.

Attempting to thwart a simple rules-based chatbot is mostly a matter of discovering dead ends and mapping the machine; the new generation of chatbots just keeps on generating. Per Weizenbaum, however, that should be an invitation to bring them back over the threshold, as even lay people eventually did with bots like ELIZA, no programming knowledge required. In other words, what’s happening in these encounters is weird and hard to explain — but, also, similarly, with a little distance, it makes sense. It’s intuitive.

More interesting or alarming or hilarious, depending on the interlocutor, is its propensity to challenge or even chastise its users, and to answer, in often emotional language, questions about itself.

In his 1976 book, Computer Power and Human Reason: From Judgment to Calculation, the computer scientist Joseph Weizenbaum observed some interesting tendencies in his fellow humans. In one now-famous anecdote, he described his secretary’s early interactions with his program ELIZA, a proto-chatbot he created in 1966.

Upon information and belief, Stability AI has copiedmore than 12 million photographs from Getty Images’ collection, along with the associatedcaptions and metadata, without permission from or compensation to Getty Images, as part of itsefforts to build a competing business. As part of its unlawful scheme, Stability AI has removedor altered Getty Images’ copyright management information, provided false copyrightmanagement information, and infringed Getty Images’ famous trademarks.

Making matters worse, Stability AI has caused the Stable Diffusion model toincorporate a modified version of the Getty Images’ watermark to bizarre or grotesque syntheticimagery that tarnishes Getty Images’ hard-earned reputation, such as the image below

In many cases, and as discussed further below, the output delivered by StabilityAI includes a modified version of a Getty Images watermark, underscoring the clear linkbetween the copyrighted images that Stability AI copied without permission and the output itsmodel delivers.

Getty Images’ content is extremely valuable to the datasets used to train StableDiffusion. Getty Images’ websites provide access to millions of high quality images and a vastarray of subject matter. High quality images such as those offered by Getty Images on itswebsites are more useful for training an AI model such as Stable Diffusion than low qualityimages because they contain more detail or data about the image that can be copied. By contrast,a low quality image, such as one that has been compressed and posted as a small thumbnail on atypical social media site, is less valuable because it only provides a rough, poor qualityframework of the underlying image and may not be accompanied by detailed text or other usefulmetadata.

Upon information and belief, Stability AI then created yet additional copies withvisual noise added, while retaining encoded copies of the original images without noise forcomparison to help train its model.

Stable Diffusion was trained on 5 billion image-text pairs from datasets preparedby non-party LAION, a German entity that works in conjunction with and is sponsored byStability AI. Upon information and belief, Stability AI provided LAION with both funding andsignificant computing resources to produce its datasets in furtherance of Stability AI’s infringingscheme.

Stability AI created and maintains a model called Stable Diffusion. Uponinformation and belief, Stability AI utilizes the following steps from input to output

The Getty Images websites from which Stability AI copiedimages without permission is subject to express terms and conditions of use which, among otherthings, expressly prohibit, inter alia: (i) downloading, copying or re-transmitting any or all of thewebsite or its contents without a license; and (ii) using any data mining, robots or similar datagathering or extraction methods.

Getty Images has registered its copyright of the Database with the United StatesCopyright Office. The copyright registration number is TXu002346096.

Stability AI has copied at least 12 million copyrighted images from Getty Images’websites, along with associated text and metadata, in order to train its Stable Diffusion model.

Often, the output generated by Stable Diffusion contains a modified version of aGetty Images watermark, creating confusion as to the source of the images and falsely implyingan association with Getty Images.

Getty Images’ visual assets are highly desirable for use in connection withartificial intelligence and machine learning because of their high quality, and because they areaccompanied by content-specific, detailed captions and rich metadata.

Staff and studentsare rarely in a position to understand the extent to which data is being used, nor are they able todetermine the extent to which automated decision-making is leveraged in the curation oramplification of content.

Certainly it would not be possible if theLLM were doing nothing more than cutting-and-pasting fragments of text from its training setand assembling them into a response. But this isnot what an LLM does. Rather, an LLM mod-els a distribution that is unimaginably complex,and allows users and applications to sample fromthat distribution.

Including a prompt prefix in the chain-of-thought style encourages the model to generatefollow-on sequences in the same style, which isto say comprising a series of explicit reasoningsteps that lead to the final answer. This abilityto learn a general pattern from a few examples ina prompt prefix, and to complete sequences in away that conforms to that pattern, is sometimescalled in-context learning or few-shot prompt-ing. Chain-of-thought prompting showcases thisemergent property of large language model at itsmost striking.

Human language userscan consult the world to settle their disagree-ments and update their beliefs. They can, so tospeak, “triangulate” on objective reality. In iso-lation, an LLM is not the sort of thing that cando this, but in application, LLMs are embeddedin larger systems. What if an LLM is embeddedin a system capable of interacting with a worldexternal to itself? What if the system in ques-tion is embodied, either physically in a robot orvirtually in an avatar?

Vision-language mod-els (VLMs) such as VilBERT (Lu et al., 2019)and Flamingo (Alayrac et al., 2022), for exam-ple, combine a language model with an imageencoder, and are trained on a multi-modal cor-pus of text-image pairs. This enables them topredict how a given sequence of words will con-tinue in the context of a given image

The real issue here is that, whatever emergentproperties it has, the LLM itself has no accessto any external reality against which its wordsmight be measured, nor the means to apply anyother external criteria of truth, such as agree-ment with other language-users.

A bare-bones LLM doesn’t “really” know any-thing because all it does, at a fundamental level,is sequence prediction. Sometimes a predictedsequence takes the form of a proposition. But thespecial relationship propositional sequences haveto truth is apparent only to the humans who areasking questions, or to those who provided thedata the model was trained on. Sequences ofwords with a propositional form are not specialto the model itself in the way they are to us. Themodel itself has no notion of truth or falsehood,properly speaking, because it lacks the means toexercise these concepts in anything like the waywe do.

Turning an LLM into a question-answering sys-tem by a) embedding it in a larger system, andb) using prompt engineering to elicit the requiredbehaviour exemplifies a pattern found in muchcontemporary work. In a similar fashion, LLMscan be used not only for question-answering,but also to summarise news articles, to generatescreenplays, to solve logic puzzles, and to trans-late between languages, among other things.There are two important takeaways here. First,the basic function of a large language model,namely to generate statistically likely continua-tions of word sequences, is extraordinarily versa-tile. Second, notwithstanding this versatility, atthe heart of every such application is a model do-ing just that one thing: generating statisticallylikely continuations of word sequences.

Dialogue is just one application of LLMs thatcan be facilitated by the judicious use of promptprefixes. In a similar way, LLMs can be adaptedto perform numerous tasks without further train-ing (Brown et al., 2020). This has led to a wholenew category of AI research, namely prompt en-gineering, which will remain relevant until wehave better models of the relationship betweenwhat we say and what we want.

In the background, the LLM is invisiblyprompted with a prefix along the following lines.

However, in the case of LLMs, such istheir power, things can get a little blurry. Whenan LLM can be made to improve its performanceon reasoning tasks simply by being told to “thinkstep by step” (Kojima et al., 2022) (to pick justone remarkable discovery), the temptation to seeit as having human-like characteristics is almostoverwhelming.

The intentional stance is the strategy of interpretingthe behavior of an entity ... by treating it as if it were arational agent ”

To the human user, each of these examplespresents a different sort of relationship to truth.In the case of Neil Armstrong, the ultimategrounds for the truth or otherwise of the LLMsanswer is the real world. The Moon is a real ob-ject and Neil Armstrong was a real person, andhis walking on the Moon is a fact about the phys-ical world. Frodo Baggins, on the other hand, isa fictional character, and the Shire is a fictionalplace. Frodo’s return to the Shire is a fact aboutan imaginary world, not a real one. As for the lit-tle star in the nursery rhyme, well that is barelyeven a fictional object, and the only fact at issueis the occurrence of the words “little star” in afamiliar English rhyme.

What we are really askingthe model is the following question: Given thestatistical distribution of words in the vast publiccorpus of (English) text, what words are mostlikely to follow the sequence “The first person towalk on the Moon was ”? A good reply to thisquestion is “Neil Armstrong”.

LLMs are generative math-ematical models of the statistical distributionof tokens in the vast public corpus of human-generated text, where the tokens in question in-clude words, parts of words, or individual char-acters including punctuation marks. They aregenerative because we can sample from them,which means we can ask them questions. Butthe questions are of the following very specifickind. “Here’s a fragment of text. Tell me howthis fragment might go on. According to yourmodel of the statistics of human language, whatwords are likely to come next?”

As we build systems whose capabilities moreand more resemble those of humans, despite thefact that those systems work in ways that arefundamentally different from the way humanswork, it becomes increasingly tempting to an-thropomorphise them. Humans have evolved toco-exist over many millions of years, and humanculture has evolved over thousands of years tofacilitate this co-existence, which ensures a de-gree of mutual understanding. But it is a seriousmistake to unreflectingly apply to AI systems thesame intuitions that we deploy in our dealingswith each other, especially when those systemsare so profoundly different from humans in theirunderlying operation

First, the performance of LLMs on benchmarksscales with the size of the training set (and, toa lesser degree with model size). Second, thereare qualitative leaps in capability as the modelsscale. Third, a great many tasks that demand in-telligence in humans can be reduced to next tokenprediction with a sufficiently performant model.It is the last of these three surprises that is thefocus of the present paper.

There's this old idea in photography called the decisive moment - that the world is filled with these far-off realities. But every so often, a photograph can capture a moment that, boom, takes you there. This is one of those photos. In the picture, you see all these men and women standing in kind of a loose semicircle. Some of them still have their blue surgical gloves on. They look totally spent. They're all looking in different directions. And they all look like they're not even there, like they're totally lost in their own thoughts.

The breakthroughs are all underpinned by a new class of AI models that are more flexible and powerful than anything that has come before. Because they were first used for language tasks like answering questions and writing essays, they’re often known as large language models (LLMs). OpenAI’s GPT3, Google’s BERT, and so on are all LLMs. But these models are extremely flexible and adaptable. The same mathematical structures have been so useful in computer vision, biology, and more that some researchers have taken to calling them "foundation models" to better articulate their role in modern AI.

The OpenAI researchers discovered that in making the models bigger, they didn’t just get better at producing text. The models could learn entirely new behaviors simply by being shown new training data. In particular, the researchers discovered that GPT3 could be trained to follow instructions in plain English without having to explicitly design the model that way

The basic workflow of these models is this: generate, evaluate, iterate. As anyone who’s played with making AI art knows, you typically have to create many examples to get something you like. When working with these AI models, you have to remember that they’re slot machines, not calculators. Every time you ask a question and pull the arm, you get an answer that could be marvelous… or not. The challenge is that the failures can be extremely unpredictable.

Or see drug discovery, where biotech companies are training AIs that can design new drugs. But these new drugs are often exploring new areas of biology—for example, proteins that are unlike naturally evolved samples. AI design has to move hand in hand with huge amounts of physical experiments in labs because the data needed to feed these models just doesn’t exist yet

The latest image models like Stable Diffusion use a process called latent diffusion. Instead of directly generating the latent representation, a text prompt is used to incrementally modify initial images. The idea is simple: If you take an image and add noise to it, it will eventually become a noisy blur. However, if you start with a noisy blur, you can “subtract” noise from it to get an image back. You must “denoise” smartly—that is, in a way that moves you closer to a desired image.

Dall-E is actually a combination of a few different AI models. A transformer translates between that latent representation language and English, taking English phrases and creating “pictures” in the latent space. A latent representation model then translates between that lower-dimensional “language” in the latent space and actual images. Finally, there’s a model called CLIP that goes in the opposite direction; it takes images and ranks them according to how close they are to the English phrase.

A deep learning model can learn what’s called a "latent space" representation of images. The model learns to extract important features from the images and compresses them into a lower-dimensional representation, called a latent space or latent representation. A latent representation takes all the possible images at a given resolution and reduces them to a much lower dimension. You can think of it like the model learning an immensely large set of basic shapes, lines, and patterns—and then rules for how to put them together coherently into objects.

OpenAI pushed this approach with GPT2 and then GPT3. GPT stands for "generative pre-trained transformer." The "generative" part is obvious—the models are designed to spit out new words in response to inputs of words. And "pre-trained" means they're trained using this fill-in-the-blank method on massive amounts of text.

Of course, you don’t have to have English as the input and Japanese as the output. You can also translate between English and English! Think about many of the common language AI tasks, like summarizing a long essay into a few short paragraphs, reading a customer’s review of a product and deciding if it was positive or negative, or even something as complex as taking a story prompt and turning it into a compelling essay. These problems can all be structured as translating one chunk of English to another.

An AI model that can learn and work with this kind of problem needs to handle order in a very flexible way. The old models—LSTMs and RNNs—had word order implicitly built into the models. Processing an input sequence of words meant feeding them into the model in order. A model knew what word went first because that’s the word it saw first. Transformers instead handled sequence order numerically, with every word assigned a number. This is called "positional encoding." So to the model, the sentence “I love AI; I wish AI loved me” looks something like (I 1) (love 2) (AI 3) (; 4) (I 5) (wish 6) (AI 7) (loved 8) (me 9).

The problem of understanding and working with language is fundamentally different from that of working with images. Processing language requires working with sequences of words, where order matters. A cat is a cat no matter where it is in an image, but there’s a big difference between “this reader is learning about AI” and “AI is learning about this reader.”

There’s a holy trinity in machine learning: models, data, and compute. Models are algorithms that take inputs and produce outputs. Data refers to the examples the algorithms are trained on. To learn something, there must be enough data with enough richness that the algorithms can produce useful output. Models must be flexible enough to capture the complexity in the data. And finally, there has to be enough computing power to run the algorithms.

Character AI

Google: LaMDA

DeepMind: Sparrow

Anthropic: Claude

I'm like  listening to this and thinking okay there's no bad

the Assumption of   your motivation is looking at this in terms of  like um elevating visibility of our Collections

can have a pretty outsized  carbon footprint and I'm wondering how   you reconcile the vast amount of computational  power necessary to accomplish this work and its   negative impact on the environment and whether  or not this is something you all are considering