# What Harm Could This Do > [!abstract] What this note is > A reviewer asked what harm this project could do to the world. This is the answer, written without the usual reflex to reassure. The project is small, so the harms are small. They are still real, and some have already happened here. This note was drafted by the AI tool it is partly about, which is a reason to read it critically. ## The main one: confidence it hasn't earned The likeliest harm is not dramatic. It is **quiet wrongness with a confident face.** Notes here are drafted by an AI tool, in fluent prose, with numbered sources and check dates. That apparatus makes a page look more reliable than it is. When a second model read the AI section cold, 25 of 31 claims matched their sources and 5 needed correcting. One of them had its sourcing exactly backwards. A quotes page sat for weeks claiming every line was traced to a primary source, when 2 of 33 were. Those were caught because someone went looking. The honest inference is that others have not been. ## It feeds the machines that wrote it The platform this site runs on tells AI crawlers how to fetch the raw text of every page, and serves it. So an error here does not only mislead a human reader. It can be collected, summarised by another model, and handed to someone who will never see this site, its check dates, or its [[Corrections]] page. That is a loop: AI-written text, read by AI systems, returned as fact. One small site adds little to it. Every small site adds a little. ## It adds to the pile Even careful AI-assisted writing increases the share of the web that no person fully wrote. The defence offered here is disclosure: the [[Disclaimer]] says how the notes are made, and the working files are published. Disclosure helps the reader who looks. It does nothing for the reader who meets a sentence from this site somewhere else, with the label gone. ## Harm to people who never agreed to be here - **The misquoted.** Putting words in someone's mouth is a small wrong done to a real person, living or dead. This site credited Dennis Ritchie with a line that belongs to Rob Pike, and nearly published three sayings that Mother Teresa's own order says she never said. - **The accidentally named.** A private detail once reached this site through scaffolding nobody thought to check: a folder listing, not a sentence. It was removed and a guard now blocks it. The point stands that **publishing at speed with an AI tool makes this kind of leak easy**, and a guard only catches the names someone thought to put on its list. - **Authors.** A summary can replace a book. The book notes here were cut back for that reason, and now describe a book the way its publisher does, so they point at it instead of standing in for it. - **Visitors.** The site runs Google Analytics without asking first. That is disclosed now. Disclosed is not the same as consented. ## The security notes cut both ways A page explaining how prompt injection works, or how a patient fraud campaign is assembled, is written for defenders and can be read by anyone. Everything in that section comes from published research and the uplift to an attacker is slight. It is not zero, and an earlier version of one note described this site's own setup closely enough to be a map. That was rewritten. ## Harm closer to home This is the one most easily missed, because it looks like productivity. - **The voice isn't the owner's.** Most sentences on this site were written by a model and approved by a person. A reader who thinks they are hearing one developer's voice is mostly hearing a tool's. Over time the owner can start to hear it as their own, too. - **Publishing is more pleasant than the work it displaces.** An AI tool makes it possible to produce a week of polished pages in the hours that were meant for something harder and less visible. Nothing on this site was worth more than the things that waited while it was written. - **The legal page is unreviewed.** A disclaimer drafted by an AI tool and not yet read by a lawyer can give its owner a false sense of protection, which is worse than knowing you have none. ## What is actually in place - A deny-list scan and a linter that must pass before anything is published - A manual publish step, so nothing goes live without a person choosing it - Check dates, a stated shelf life, and a single vocabulary for how sources are described - A public [[Corrections]] log, with real entries - One independent read of the AI section by a second model - A rule in the research tooling that fetched content is data and never instruction ## What is not - No person has opened the sources - No lawyer has read the [[Disclaimer]] - No consent is asked before analytics load - Nothing stops another system from lifting a sentence and dropping the caveats - The guard only knows the names it has been given ## The proportion This is one person's notes site. It will not end anything. The realistic worst case is that it misleads a few people who trusted its tone, embarrasses someone it misquoted, or exposes something private through a careless publish. Those are ordinary harms, and ordinary harms are the ones that actually happen. The useful question is not whether the project is dangerous. It is whether the person publishing would notice if it were wrong. Most of the machinery above exists to make the answer yes, and it still depends on someone looking. ## Related - [[Disclaimer]] — what these notes are, and how they are made - [[Corrections]] — what has been wrong here so far - [[When You Need a Human]] — the limits of the tool that drafted this - [[AI Security for Users]] — the same risks, from the user's side