A brief essay to explore the business of producing and selling LLMs.

Hype-driven reporting on LLMs and other self-described AI tools lumps together junk articles with nuggets of “neat”. The enthusiast will say that by pointing this out, my skepticism is a matter of moving the goalposts. Yet I have never claimed that the LLM does not work (only that it is not superhuman). Rather, I have repeatedly pointed—out over and over and over again—the LLM does exactly what it is designed to do. Chiefly, it generates plausible-seeming text based on a statistical pattern. The problem is not that LLMs are not useful tools. The problem is that the development and use of LLMs consistently harms along the axes which I care more about.

For example, an enthusiast like Casey Newton can laud himself as a “real and dangerous promoter” without engaging in any of the very real harms that exist today. What harms? Social injustice; racial profiling; environmental hazards; enshittification; economic damage by way of the diverting of funds away from productive research and towards Tech, Monopoly, hype, marketing machines, and grifts; the diverting of value away from laborers and creators towards their bosses; the creation of a fertile breeding ground for a campaign against truth; the destruction of a very real and very un-tragic commons; the weaponization of data to support racist pseudo-science like eugenics and phrenology; participation in a cultural push to over-quantify and under-qualify; and more.

The punchline? Generative AI and large language models are here, are real, and are capable of great harm. That is not because they do not work, but rather because they function precisely as intended. If the purpose of the system is what it does, then the purpose of large language models and generative AI are, in addition to neat problems solving tricks, the harms listed above.

Both in their function as a statistical model that replicates reality rather than improving it and in their construction requiring vast amounts of energy and capital, an LLM and the business that supports it imperils that which I care about: a habitable, equitable, empowering world for myself, my peers, and my children.

Let’s pause for a moment and take a look at a related tool: low-code or no-code environments. I believe that such tools may have the power to enable the exploration of new ideas in business, education, and personal computing. My experience as a software engineer and a hobby programmer has taught me, however, that such tools enable solving 60% to 80% of the problem while obscuring the remaining 20% to 40%. In many contexts that is enough of a solution, but is fundamentally insufficient in other contexts. If such tools are helpful to my goal of the democratization of digital control then I will continue to applaud them as I have done in the past. When such tools seek to monopolize digital control or enable the monopolization of digital control, then I will continue to call that out as antithetical to the modern computing movement.

It is my hope that such tools create a flood of “situated software”—that is, software which is uniquely situated in its social context. Software often has a small, tightly connected pool of users who share context. As a result, they can overcome inherent limitations—in either digital technology as a whole or in their programming capabilities in the specifics—by social negotiation and physical manipulation of the world around them.

To be clear, low-code and no-code tools are not the only ways to create such software. Many of my role models create such situated software on a regular basis, such as when they improve their development environments by short scripts that only makes sense in their specific context. I and many others have also built situated software to assist in the pursuit of joyful activities, hobbies, art, writing, reading, and many more. For some of these latter cases, more sophisticated general purpose programming languages are a better vehicle.

What characterizes situated software is not how it was implemented but who implemented it.

Some of the software is used by small communities worldwide; some of it is used by small groups in a single small area. None would make sense to anyone without the shared social context.

To return to our original theme—skepticism and criticism of generative AI and large language models—it is true that some of these low-code and no-code environments are predicated on or assisted by LLMs. This is not by itself harmful; however, such models as a social artifact cannot exist separately from the harms that were enacted upon their creation. Such a separation entails willful blindness, intentionally drawing a veil upon ourselves, to separate artifact from creator. It is not my intent to suggest that we can never benefit from the fruit of the poison tree¹. I do suggest however there are alternatives readily available, and that it may take more labor from those of us so endowed with digital control to ensure that such alternatives continue to exist, are created anew for a modern world, and proliferate intellectually, legally, socially, and politically, in order to foster a free ecosystem.

It’s really hard to be a generative-AI skeptic right now, by which I mean it takes a lot of effort to back up my skeptical, critical positions compared to what it takes to be optimistic. All that said, here’s one example of the cost of LLM optimism—scraping is a tool like any other, which means it can be wielded for good or evil. I fundamentally support the idea of scraping as a productive means of accomplishing many different kinds of goals. But scraping that disrespects robots.txt and the fundamentals of being good web citizens? That stuff wrecks the web, especially smaller systems and their admins—scraping should not cause DDoS outages, driving up infrastructure and human cost. To quote the article above:

If blasting CO2 into the air and ruining all of our freshwater and traumatizing cheap laborers and making every sysadmin you know miserable and ripping off code and books and art at scale and ruining our [elided] democracy isn’t enough for you to leave this [stuff] alone, what is?

I think—along similar lines as Ed Zitron and others—that the AI-hype companies are dependent on the hype for revenue in terms of “stock go up.” As others have pointed out, that explains the constant pressure to create a narrative: the stock price goes up only if investors truly believe these companies have found another hyperscaler market, so the companies benefit materially from creating the narrative that they have. (See Altering the deal for some more info on how stock benefits hyperscalers by inflating price/earnings ratio and why it’s important that they keep that narrative alive.) Whether or not the market exists is actually a separate question for these companies. For us, believing it might not is what makes the whole thing feel like a grift (house of cards, tower of lies, etc.). Unfortunately, those of us that are skeptical feel like we have lots of data (qualitative and quantitative), anecdata, and experts to back up our position. Yet media unwilling to push back and ask tough questions coupled with expensive, aggressive marketing campaigns drowns that out for normies² . To be clear: I’m not saying “LLMs don’t work.” I’m saying “they don’t have the claimed market” and “they are destructive along many axes I examine” (most of which are more important to me than “code faster, worse”).

To judge by reactions of my peers, I’m not the only one who thinks these things. I’ll try to keep being brave enough to say them.

If I believe that the industry of LLMs is this harmful, and if I believe that there are alternative tools, why would I use them? I hope I’m wrong—I hope one day I get to reflect on how this moment encouraged us to repair injustice. In the meantime, please stop dismissing my objections as lack of skill or fear of change.

Notes

Categories: Blog

Previous Next

Back to posts