A lot of discussions around Ai and their dangers are on the top of everyone's mind, especially in the tech sector. Many experts warn of AGI, while others warn of its competing usage of resources that humans also need to survive; like water and electricity. One of the things that I see less discussed, but in my opinion is far more sinister and dangerous is how Ai can actually be used to influence and manipulate the consensus of reality itself.
In the heyday of Google search, we learned that a good search engine is a valuable tool as it gives you quick access to information that you need. It allowed you to find anything and everything you wanted, you could do your own research and make your own discoveries. Fast forward a few years and Google searching overtakes internet search with people referring to it as 'Googling'. They literally became the functional name. As it continued its climb into popularity, people started to take notice of SEO (search engine optimization). How they could harness keywords and phrases so their website surfaces sooner in the search results. This then led to Google offering the top slots to paid promoters for specific things. All of those things were easy enough to work around, although at times questionably done.
Today, we have begun a process of replacing Google 'manual' searching with something even more intuitive and easy, which is using Ai to find answers. You don't even have to look for specific search results, or think about the context, Ai does it all for you and within seconds, spits out not only what you asked for, but typically all of the context to go along with it. It actually works really well, and makes 'searching' infinitely easier and more efficient in some cases. We are developing the same relationship that we did with Google, now with Ai. ChatGPT is one of the largest, and most used platforms currently in the world. Ai competitors are not far behind in sheer numbers. Its popularity is in a constant state of acceleration.
The problem here is in the detail, the nuance, the nudge. If we have a grown expectation, and possibly a dependency on Ai for our 'searching' purposes. What happens when that very Ai is weighted to respond certain ways to specific questions? One of the easiest, low hanging fruit examples that you can actually go see for yourself would be the Chinese model, DeepSeek. It is a direct competitor to US based Ai, and is considered adversarial in US Ai policies. If you ask DeepSeek Ai about Tiananmen Square, or the 1989 protests and massacre, it will deflect you every time with statements like 'There are many conspiracy theories and unproven allegations….', and it will not discuss it at all. This one is obvious, its clear what is happening, now imagine something subtler, what if the weights of the Ai are skewed, or the response patterns coerce a little in one direction for specific topics? What happens if the results you requested from the Ai are weighted in such a way that isn't obvious, but directs your attention in a more specific way. It can use bold, titles, gaslighting, kerning, spacing, exclusion, list ordering, italics, etc as ways to manipulate how a person reads something. It gives you the truth, but restructures how it shows you the truth to underweight the reality, and overweight what it wants you to take away from the conversation.
Ai models are curated and trained by their owners. Sometimes that is a corporation, sometimes a government agency, sometimes just a group of people. Any of those could leverage their publicly trusted Ai with invisible mechanics that can help shape our reality without most people noticing.
The point I am trying to convey in all of this is that Ai, as it becomes the 'go to' search and discovery tool for everyone, can easily be directed to hold bias and project that bias in its responses, to a public that will have largely let its guard down as up to this point, Ai has been useful and accurate. Once it's the 'trusted source of truth', that same power can be inverted to sway mass opinion in very subtle ways.
TL;DR
- The dramatic AI fears (AGI, resource competition) get the attention
- The bigger risk is AI shaping the consensus of reality through trusted infrastructure
- We are repeating the Google trust pattern, only with less ability to work around it
- DeepSeek's deflection on Tiananmen is the obvious version of weighting
- Subtler weighting can shape attention through formatting, ordering, exclusion, and emphasis
- Whoever owns and trains the model controls the weighting
- Once AI becomes the trusted source of truth, that trust can be inverted to sway mass opinion
// suggested reading
- Kropotkin, Peter (1902) — Mutual Aid: A Factor of Evolution
- Graeber, David (2004) — Fragments of an Anarchist Anthropology