The time when we suffer from large amounts of AI slop is gone. Now we instead suffer under a massive load of good reports.
大多数人认为AI工具会产生大量低质量的'垃圾报告'(AI slop),增加开发者的负担,但作者认为现在AI生成的安全报告质量很高,虽然数量庞大但都是高质量的报告。这是一个反直觉的观点,因为通常人们认为自动化工具会产生大量噪音而非有价值的贡献。
The time when we suffer from large amounts of AI slop is gone. Now we instead suffer under a massive load of good reports.
大多数人认为AI工具会产生大量低质量的'垃圾报告'(AI slop),增加开发者的负担,但作者认为现在AI生成的安全报告质量很高,虽然数量庞大但都是高质量的报告。这是一个反直觉的观点,因为通常人们认为自动化工具会产生大量噪音而非有价值的贡献。
In one U.S. survey, 40% of employees said they had received 'workslop', i.e. AI-generated content that looks polished but isn't accurate or useful, in the past month.
这一惊人的高比例(40%)的'workslop'现象揭示了AI应用中的一个悖论:虽然AI提高了效率,但同时也带来了大量低质量内容。这一发现挑战了'AI总是提高生产力'的普遍假设,暗示了过度依赖AI可能导致的隐性成本,需要重新评估AI的实际价值。
A small model trained on fewer than 2,000 examples from real lawyers, bankers, and consultants recently beat all but the best frontier models on corporate legal work, at a fraction of the price.
这一发现挑战了'规模和计算能力胜过一切'的AI发展范式。高质量专业化数据训练的小型模型在特定领域表现优于通用大模型,暗示AI发展可能从'越大越好'转向'更专业、更高效'的新阶段。
As slop takes over the Internet, labs may struggle to obtain high-quality corpuses for training models.
这一观察揭示了AI训练数据质量的危机。随着互联网内容质量的下降,AI系统可能面临'垃圾进,垃圾出'的风险。作者提出的'低背景钢'比喻巧妙地指出了使用2023年前纯净数据的解决方案,同时也暗示了数字时代知识污染的严重性,这可能会对AI系统的可靠性和偏见产生深远影响。
Add contacts, live search, full pipeline dashboard – all unit tests passed.
令人惊讶的是:AI生成的代码不仅功能完整,包括联系人管理、实时搜索和完整的管道仪表板,而且所有单元测试都通过了,表明AI不仅能快速编码,还能保证代码质量。
their productivity is affected by the state of the codebase.
【启发】这句话的深远意义在于:它把 AI Coding Agent 与人类开发者置于同一评价维度。这不是「AI 是否能替代人」的问题,而是「AI 受代码质量影响的方式是否与人类相同」。答案是肯定的——这意味着几十年来软件工程师积累的代码质量实践,不是因为 AI 的到来而失效,而恰恰因为 AI 的到来而变得更加重要。技术债从「慢慢影响人」变成了「立刻影响 AI 的 token 消耗」。
Some good pointers to [[Brian Eno c]] work and thinking, to follow up.
Also good anecdote from one of those links on Rem Koolhaas notion of n:: premature sheen Making things look nice early takes away from thinking about other points of quality. Jeremy applies it to AI too, the premature sheen generate awe, but not quality output.
Broderick makes a more important point: AI search is about summarizing web results so you don't have to click links and read the pages yourself. If that's the future of the web, who the fuck is going to write those pages that the summarizer summarizes? What is the incentive, the business-model, the rational explanation for predicting a world in which millions of us go on writing web-pages, when the gatekeepers to the web have promised to rig the game so that no one will ever visit those pages, or read what we've written there, or even know it was us who wrote the underlying material the summarizer just summarized? If we stop writing the web, AIs will have to summarize each other, forming an inhuman centipede of botshit-ingestion. This is bad news, because there's pretty solid mathematical evidence that training a bot on botshit makes it absolutely useless. Or, as the authors of the paper – including the eminent cryptographer Ross Anderson – put it, "using model-generated content in training causes irreversible defects"
Broderick: https://www.garbageday.email/p/ai-search-doomsday-cult, Anderson: https://arxiv.org/abs/2305.17493
AI search hides the authors of the material it presents, summarising it is abstracting away the authors. It doesn't bring readers to those authors, it just presents a summary to the searcher as end result. Take it or leave it. At the same time, if one searches for something you know about, you see those summaries are always of. Leaving you guessing how of it is when searching something you don't know about. Search should never be the endpoint, always a starting point. I think that is my main aversion against AI search tools. Despite those clamoring 'it will get better over time' I don't think it will easily because the tool nor its makers have any interest in the quality of output necessarily and definitely can't assess it. So what's next, humans factchecking AI output. Why not prevent bs at its source? Nice ref to Maggie Appleton's centipede metaphor in [[The Expanding Dark Forest and Generative AI]]
So what does a conscious universe have to do with AI and existential risk? It all comes back to whether our primary orientation is around quantity, or around quality. An understanding of reality that recognises consciousness as fundamental views the quality of your experience as equal to, or greater than, what can be quantified.Orienting toward quality, toward the experience of being alive, can radically change how we build technology, how we approach complex problems, and how we treat one another.
Key finding Paraphrase - So what does a conscious universe have to do with AI and existential risk? - It all comes back to whether our primary orientation is around - quantity, or around - quality. - An understanding of reality - that recognises consciousness as fundamental - views the quality of your experience as - equal to, - or greater than, - what can be quantified.
Quote - metaphysics of quality - would open the door for ways of knowing made secondary by physicalism
Author - Robert Persig - Zen and the Art of Motorcycle Maintenance // - When we elevate the quality of each our experience - we elevate the life of each individual - and recognize each individual life as sacred - we each matter - The measurable is also the limited - whilst the immeasurable and directly felt is the infinite - Our finite world that all technology is built upon - is itself built on the raw material of the infinite
//