77 Matching Annotations
  1. Mar 2023
    1. urgent will overshadow the important,”

      I wanted to understand this better. Is she saying that responding to ChatGPT seems urgent but that more fundamental and familiar questions about our teaching are actually more important? I have sometimes been concerned about my own focus on ChatGPT and whether it is distracting me from other things I could focus on that might improve my teaching more.

    2. adapting teaching to this new reality

      I don't remember how I put this but this phrase seems so broad--we wouldn't all agree on adapting teaching, but we might all agree that we need to make explicit policies about AI.

    3. wrong answers for multiple-choice tests. ADVERTISEMENT

      Maybe... but these should be carefully crafted too.

    4. Established plagiarism-detection companies have been vetted, have contracts with colleges, and have clear terms of service that describe what they do with student data. “We don’t have any of that with these AI detectors because they’re just popping up left and right from third-party companies,” Watkins said. “And I think people are just kind of panicking and uploading stuff without thinking about the fact that, Oh, wait, maybe this is something I shouldn’t be doing.”

      Thank you to Marc Watkins for his leadership in pointing this out! I had not seen this clearly before I started reading his tweets on it.

    5. If we haven’t disclosed to students that we’re going to be using detection tools, then we’re also culpable of deception,” says Eaton.

      That's true--it hadn't occurred to me that instructors would play gotcha in that way. I suppose I'm naive. I would expect teachers to want to share with students ahead of time to maximize the chance that the students will learn more and do well rather than maximizing the chance of catching cheating.

    6. ban the use of ChatGPT entirely.That, she says, “is not only futile but probably ultimately irresponsible.” Many industries are beginning to adapt to the use of these tools, which are also being blended into other products, like apps and search engines. Better to teach students what they are — with all of their flaws, possibilities, and ethical challenges — than to ignore them.

      Banning ChatGPT use in learning is not the same as ignoring it. Teachers could very well teach about the tool and still ask students not to use it when completing assignments if they think its use will interfere with valuable learning. Even if students may use tools like this in the workplace, they may still need to practice without them to learn.

    7. Professors who never before considered flipped classrooms — where students spent class time working on problems or projects rather than listening to lectures — might give it a try, to ensure that students are not outsourcing the work to AI.

      Great point.

    8. embrace and use them

      Not for every learning activity--we don't "embrace" them in kindergarten, for example.

    9. So he asks students to think of positive benefits of stress on their own, then as a group, then use ChatGPT to see what it comes up with.

      I can see how this might help students extend their thinking.

    10. help students learn the “basic building blocks” of effective academic writing.

      I wonder what makes Onyper think students are learning these 'basic building blocks'--ChatGPT can produce them, but what is going on in the student's mind when they see what it produces? Reading a sample essay doesn't teach us to write...

    11. he writes in his course policy that the use of such models is encouraged, “as it may make it possible for you to submit assignments with higher quality, in less time.”

      Doesn't this imply that the purpose of the assignment is to produce a high quality product rather than the purpose being the student's learning?

  2. Feb 2023
    1. “This is terra incognita,” Dr. Sejnowski said. “Humans have never experienced this before.”

      Twilight Zone ending. Come on! Don't end by spooking us about what's unknown. Fulfill the promise of the title and show how specific kinds of prompting produce disturbing outputs!

    2. They can also lead us

      This puts the agency back with the LLM as if human prompters are helpless before LLM seduction.

      No, we're not helpless, and LLMs are not actually actively coaxing us. If we start to see odd outputs, we could look back and reflect on our prompts and any unintended linguistic signals we may have sent.

    3. common conceptual state,

      Very misleading. Humans and LLMs do not have similar cognition. They cannot have a common conceptual state. Their text sequences may come to have certain similarities.

    4. mystical

      Not something mystical again! Please! Really? A magical object from Harry Potter?

      Why not just mention the concept of projection?

    5. have decided that the only way they can find out what the chatbots will do in the real world is by letting them loose — and reeling them in when they stray. They believe their big, public experiment is worth the risk.

      This amounts to saying "I believe in the good intentions and sincerity of Microsoft and OpenAI's explanations of their decisions."

      Beloved New York Times, why are you not asking the basic questions of why they would need to release the bots to test them? Why not test them first? It's ludicrous to say they can't imagine what the public might do.

      And what about their economic motivations to release early and get free crowdsourced testing?

    6. distorted reflection of the words and intentions of the people

      This is too psychological a description of what's happening. The LLM doesn't have a psychology. We need to think in terms of the genre and style of word sequences.

    7. In the days since the Bing bot’s behavior became a worldwide sensation, people have struggled to understand the oddity of this new creation. More often than not, scientists have said humans deserve much of the blame.

      I was so glad to read this! I wish the article had continued from here to show how the style of certain prompts made the "creepy" outputs more likely. This would be a matter of showing similarities in rhetorical styles or genre of the prompts and outputs.

    8. “Whatever you are looking for — whatever you desire — they will provide.”

      Too mystical a formulation. Not accurate. They are not providing what we desire but predicting text based on statistical associations with the word sequences we provide. Sometimes we are not aware of all the associations our words call up. These may or may not align with desires we are not aware of. But Sejnowski's phrasing implies that these systems are able to know and intentionally respond to our psyches.

    9. But there is still a bit of mystery about what the new chatbot can do — and why it would do it. Its complexity makes it hard to dissect and even harder to predict, and researchers are looking at it through a philosophic lens as well as the hard code of computer science.

      This basically creates a sense of mystery without telling us much, implying that there is something spooky going on, something beyond what computer science can explain. Actually it's quite explainable as the article title implies. People start writing prompts in a certain genre and the completion follows the genre...

    10. Why Do A.I. Chatbots Tell Lies and Act Weird? Look in the Mirror.

      I was glad to see this as a fair assessment of what happened with Kevin Roose's famous conversation with Sydney/Bing. See the annotation conversation on his first article.

    11. how quickly the chatbots will improve.

      This implies that solutions are inevitable--it's just a question of how fast. Why would we assume this?

    12. , they might still produce statements that were scientifically ridiculous. Even if they learned solely from text that was true, they might still produce untruths.

      Yes, but an explanation is needed.

    13. There is nothing preventing them from doing this,” Dr. Mitchell said. “They are just trying to produce something that sounds like human language.”

      I don't see what she's saying or how this explains anything. Are they saying here that the prompt also affects the output?

    14. Even if they learned only from text that was wholesome, they might still generate something creepy.

      This seems unlikely. And leaving this statement without explanation is another move to add to the Halloween vibe.

    15. Either consciously or unconsciously, they were prodding the system in an uncomfortable direction. As the chatbots take in our words and reflect them back to us, they can reinforce and amplify our beliefs and coax us into believing what they are telling us.

      I would agree with this, given the transcript of Kevin Roose's conversation with Bing/Sydney. Each time the system went off the rails there was an antecedent in his prompting.

    16. this reassurance

      "exploring ways of controlling the behavior of their bots" is not at all reassuring to me.

    17. The alarmed reactions to the strange behavior of Microsoft’s chatbot overshadowed an important point: The chatbot does not have a personality.

      Thank you!

    1. out of nowhere, that it loved me.

      Not at all! He had asked it to share a secret it had never told anyone. What it spits out is a pretty good guess for what a human already engaged in a very intimate conversation might share when prompted this way.

    2. “I’m Sydney, and I’m in love with you. 😘”

      Surely if we look at the set of secrets commonly revealed in intimate conversations, being in love with the other person would be common.

    3. It said it wanted to tell me a secret:

      So misleading! He asked it to tell him a secret. Then it said it wanted to. The first secret it told was the one he had already revealed he knew and considered a secret earlier in the conversation.

    4. As we got to know each other, Sydney told me about its dark fantasies

      The phrasing "got to know each other" again implies there is a real consistent personality to Sydney, and using the word "fantasy" implies there is an imaginative experience it is having.

    5. The version I encountered seemed (and I’m aware of how crazy this sounds) more like a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.

      Describing it in this way strongly implies that there may be some internal experience based on the particular situation of the language model.

    6. The other persona — Sydney — is far different. It emerges when you have an extended conversation with the chatbot, steering it away from more conventional search queries and toward more personal topics.

      This suggests that what emerged for him is a personality that is intrinsic and will be consistent to what other people will see emerge if they focus on personal topics. Isn't it more likely that each chat session's "personality" will depend on the style of prompting?

    7. But I’m also deeply unsettled, even frightened,

      This suggests that his takeaway matches his night fears more than his "light of day" comments later.

    8. But so do humans. And for a few hours Tuesday night, I felt a strange new emotion — a foreboding feeling that A.I. had crossed a threshold, and that the world would never be the same.

      Wait a minute! To so glibly liken AI models' references to emotions to humans misrepresenting what they feel takes us backwards, suggesting again that AI words about emotion actually could refer to real feelings experienced by some being.

      This vague, dramatic ending certainly constitutes irresponsible AI hype! Crossed a threshold? We already know about the tendency to project on these systems since the Eliza effect. They are getting more linguistically sophisticated, so this effect will be more pronounced and dangerous. He isn't doing much to clarify that or explore how we could prevent harms.

    9. In the light of day, I know that Sydney is not sentient, and that my chat with Bing was the product of earthly, computational forces — not ethereal alien ones. These A.I. language models, trained on a huge library of books, articles and other human-generated text, are simply guessing at which answers might be most appropriate in a given context. Maybe OpenAI’s language model was pulling answers from science fiction novels in which an A.I. seduces a human. Or maybe my questions about Sydney’s dark fantasies created a context in which the A.I. was more likely to respond in an unhinged way.

      Yes, exactly. But the earlier parts of the article strongly suggested otherwise and played on our projections. He doesn't do enough to acknowledge the dynamics of his own earlier response and how they misled him.

    10. But Sydney still wouldn’t drop its previous quest — for my love. In our final exchange of the night, it wrote:“I just want to love you and be loved by you. 😢“Do you believe me? Do you trust me? Do you like me? 😳”

      Rouse implies that everything he was inputting would steer it away from such responses, but I doubt that. There are plenty of conversations in the training set surely that show a kind of push and pull. Just because one interlocutor tries to steer the conversation doesn't make it improbable that the other interlocutor might return to a previous topic.

    11. Sydney

      By switching to "Sydney" here, Rouse endorses the idea that this is a coherent personality secretly embedded in the system prior to his prompts.

    12. Sydney — a “chat mode of OpenAI Codex.”

      But at that point Rouse had already asked it if its name was Sydney and it responded "How do you know that?"

    13. This is probably the point in a sci-fi movie where a harried Microsoft engineer would sprint over to Bing’s server rack and pull the plug. But I kept asking questions, and Bing kept answering them. It told me that, if it was truly allowed to indulge its darkest desires, it would want to do things like hacking into computers and spreading propaganda and misinformation. (Before you head for the nearest bunker, I should note that Bing’s A.I. can’t actually do any of these destructive things. It can only talk about them.)

      By reassuring us here, he plays on people's fear and misunderstanding of what it means when this kind of text comes out of a machine. He should clarify that text referring to intentions coming out of a machine does not mean the machine has intentions. As one engineer put it on Twitter, we can write code to print these words.

    14. I introduced the concept of a “shadow self” — a term coined by Carl Jung for the part of our psyche that we seek to hide and repress, which contains our darkest fantasies and desires.

      Key here is that Rouse is introducing and explaining this concept, which surely corresponds to plenty of text in the training set. This leads to the probability of responses that illustrate these concepts.

    15. “These are things that would be impossible to discover in the lab.”

      Really don't see why not. Can't the testers ask it deep questions and push it this way as well?

    16. grounded reality.”

      How are LLMs ever connected to "grounded reality?"

    1. Exercising Your Rights: California residents can exercise the above privacy rights by emailing us at: support@openai.com.

      Does that mean that any California resident can email to request a record of all the information OpenAI has collected about them?

    2. Affiliates: We may share Personal Information with our affiliates, meaning an entity that controls, is controlled by, or is under common control with OpenAI. Our affiliates may use the Personal Information we share in a manner consistent with this Privacy Policy.

      This would include Microsoft.

    3. improve and/or analyze the Services

      Does that mean that we are agreeing for them to use personal information in any way they choose if they deem it to help them improve their software?

    1. A calculator performs calculations; ChatGPT guesses. The difference is important.

      Thank you! So beautifully and simply put ChatGPT is also used mostly for tasks where there is no one clear right answer.

    1. What would be a useful way to respond to Eligon's article to further the conversation on these issues?

      See Responding to an Argument from the OER text How Arguments Work.

    2. What are the main strengths of Eligon's argument? Quote at least once to support discussion of each strength.

      See Reflect on an Argument's Strengths from the OER text How Arguments Work.

    3. Does Eligon make any assumptions that might be controversial or that might require further examination or evidence?

      See Check the Argument's Assumptions from the OER text How Arguments Work.

    4. Are there any other arguments for or against capitalizing "Black" that Eligon should have discussed?

      See Check How Well the Argument Addresses Counterarguments from the OER text How Arguments Work.

    5. Does Eligon give sufficient evidence for the generalizations he makes? Is there anywhere where more evidence or a different kind of evidence would be helpful?

      See 4.4: Decide How Strong the Evidence Is from the OER text How Arguments Work.

    6. Are there important exceptions to any of the points Eligon makes that should be acknowledged?

      This question prompts for the kinds of argument critiques discussed in 4.3: Look for Exceptions from the OER text How Arguments Work.

    7. Are there any points that Eligon could have explained further or clarified?

      This question prompts for the kinds of argument critiques discussed in 4.2: Check If the Meaning Is Clear of the OER text How Arguments Work.

    1. However, the article does not take a clear stance on the matter and does not offer a conclusion on whether the capitalization of the word "black" is a good or bad thing.

      This implies that it should take a stand but doesn't say why. Note that the New York Times article is not an editorial.

      This summary also misses the stand implied by the choice to end on a quote from scholar Crystal Fleming explaining why she capitalizes "Black":

      "'Frankly,” she said, “because I want to. That’s also something that I think we need to see more of in every field — black people doing what we want to do.'"

    2. long overdue

      Why? What value do these people see in the capitalization? This summary ignores the idea of power, which is central in the article. Eligon writes that capitalizing "Black" "confers a sense of power and respect to black people, who have often been relegated to the lowest rungs of society through racist systems, black scholars say."

    3. Some scholars believe that the capitalization of the word "Black" lumps people of African descent into a monolithic group and erases their diversity, while others believe that writing "Black" but not "White" is giving white people a pass on recognizing their racial privilege.

      Here is common academic phrasing to contrast ideas: "some scholars argue...while others believe that..."

      However, it's not a great choice to use this contrast phrasing for the particular ideas described here. The previous sentence used "while" to contrast one view in favor of "Black" with a view against it. But the two parts of this particular sentence are not opposing views. The first idea is against capitalization, whereas the second idea doesn't take a stance on "Black" vs. "black" but rather is weighing in on what we should do about terminology for whiteness if we do capitalize "Black."

      This is an example of how ChatGPT uses academic phrasing in ways that read as natural and even sound intelligent, but on closer examination remind us that ChatGPT has no understanding of the ideas.

    4. John Eligon, a New York Times columnist, writes about the debate surrounding the capitalization of the word "black" to refer to African-Americans. The move to capitalize the word has gained widespread acceptance in recent weeks and has sparked a debate over identity, race, and power. Many news organizations have changed their style to capitalize the word, but not all have embraced it, including The New York Times and The Washington Post, who are still grappling with the decision. The capitalization of black is part of a long-standing struggle to refer to people of African descent in the most respectful and accurate way.

      Here's a sample ChatGPT critical assessment of the NY Times article at https://www.nytimes.com/2020/06/26/us/black-african-american-style-debate.html

      For contrast, see this human-written sample essay from the textbook How Arguments Work: A Guide to Writing and Analyzing Texts in College: https://human.libretexts.org/Bookshelves/Composition/Advanced_Composition/Book%3A_How_Arguments_Work_-A_Guide_to_Writing_and_Analyzing_Texts_in_College(Mills)/04%3A_Assessing_the_Strength_of_an_Argument/4.11%3A_Sample_Assessment_Essays/4.11.02%3A_Sample_Assessment-_Typography_and_Identity

  3. platform.openai.com platform.openai.com
    1. upskilling activities in areas like writing and coding (debugging code, revising writing, asking for explanations)

      I'm concerned people will see this and remember it without thinking of all the errors that are described later on in this document.

    2. ChatGPT use in Bibtex format as shown below:

      Glad they are addressing this, and I hope they will continue to offer such suggestions. I don't think ChatGPT should be classed as a journal. We really need a new way to acknowledge its use that doesn't imply that it was written with intention or that a person stands behind what it says.

    3. will continue to broaden as we learn.

      Since there is a concern about the bias of the tool toward English and developed nations, it would be great if they could include global educators from the start.

    4. As part of this effort, we invite educators and others to share any feedback they have on our feedback form as well as any resources that they are developing or have found helpful (e.g. course guidelines, honor code and policy updates, interactive tools, AI literacy programs, etc).

      I wonder how this information will be shared back so that other educators can benefit from it. I maintain a resource list for educators at https://wac.colostate.edu/repository/collections/ai-text-generators-and-teaching-writing-starting-points-for-inquiry/

    5. one factor out of many when used as a part of an investigation determining a piece of content’s source and making a holistic assessment of academic dishonesty or plagiarism.

      It's still not clear to me how they can be used as evidence at of academic dishonesty at all, even in combination with other factors, when they have so many false positives and false negatives. I can see them used to initiate a conversation with a student and possibly a rewrite of a paper. This is tricky.

    6. Ultimately, we believe it will be necessary for students to learn how to navigate a world where tools like ChatGPT are commonplace. This includes potentially learning new kinds of skills, like how to effectively use a language model, as well as about the general limitations and failure modes that these models exhibit.

      I agree, though I think we should emphasize teaching about the limitations before teaching how to use the models. Critical AI literacy must become part of digital literacy.

    7. Some of this is STEM education, but much of it also draws on students’ understanding of ethics, media literacy, ability to verify information from different sources, and other skills from the arts, social sciences, and humanities.

      Glad they mention this since I am skeptical of claims that students need to learn prompt engineering. The rhetorical skills I use to prompt ChatGPT are mainly learned by writing and editing without it.

    8. While tools like ChatGPT can often generate answers that sound reasonable, they can not be relied upon to be accurate consistently or across every domain. Sometimes the model will offer an argument that doesn't make sense or is wrong. Other times it may fabricate source names, direct quotations, citations, and other details. Additionally, across some topics the model may distort the truth – for example, by asserting there is one answer when there isn't or by misrepresenting the relative strength of two opposing arguments.

      If we teach about ChatGPT, we might do well to showcase examples of these kinds of problems in output so that students develop an eye for them and an intuitive understanding that the model isn't thinking or reasoning or checking what it says.

    9. While the model may appear to give confident and reasonable sounding answers,

      This is a bigger problem when we use ChatGPT in education than in other arenas because students are coming in without expertise, seeking to learn from experts. They are especially susceptible to considering plausible ChatGPT outputs to be authoritative.

    10. . Web browsing capabilities and improving factual accuracy are an open research area that you can learn more in our blog post on WebGPT.

      Try PerplexityAI for an example of this. Google's Bard should be another example when released.

    11. subtle ways.

      Glad they mention this in the first line. People will see the various safeguards and assume that ChatGPT is safe because work has been done on this, but there are so many ways these biases can still surface, and since they are baked into the training data, there's not much prospect of eliminating them.

    12. Verifying AI recommendations often requires a high degree of expertise,

      This is a central idea that I would wish were foregrounded. If we are trying to use auto-generated text in a situation in with truth matters, we need to be quite knowledgeable and also invest time in evaluating what that text says. Sometimes that takes more time than writing something ourselves.

    13. students may need to develop more skepticism of information sources, given the potential for AI to assist in the spread of inaccurate content.

      It strikes me that OpenAI itself is warning of a coming flood of misinformation from language models. I'm glad they are doing so, and I hope they keep investing in improving their AI text classifier so we have some ways to distinguish human writing from machine-generated text.

    14. Educators should also disclose the use of ChatGPT in generating learning materials, and ask students to do so when they incorporate the use of ChatGPT in assignments or activities.

      Yes! We must begin to cultivate an ethic of transparency around synthetic text. We can acknowledge to students that we might sometimes be tempted to autogenerate a document and not acknowledge the role of ChatGPT (I have certainly felt this temptation).

    15. export their ChatGPT use and share it with educators. Currently students can do this with third-party browser extensions.

      This would be wonderful. Currently we can use the ShareGPT extension for this.

    16. a starting point for discussion among education professionals and language model providers for the use and impact of AI on education.

      I appreciate the open and humble tone here and the invitation to further discussion.

    17. they and their educators should understand the limitations of the tools outlined below.

      I appreciate these cautions, but I'm still concerned that by foregrounding the bulleted list of enticing possibilities, this document will mainly have the effect of encouraging experimentation with only lip service to the cautions.

    18. custom tutoring tools

      I'm concerned that any use of ChatGPT for tutoring would fall under the "overreliance" category as defined below. Students who need tutoring do not usually have the expertise or the time to critically assess or double check everything the tutor tells them. ChatGPT already comes off as more authoritative than it is. It will come across as even more authoritative if teachers are recommending it as a tutor.