Through the Kaleidoscope of Generative AI

Through the Kaleidoscope of Generative AI: On Its Technical Foundations and Implications

Content warning: This article contains a brief discussion of mental illness in an example of AI sycophancy.

Large language models (LLMs) have entered various aspects of our lives. Perhaps you only seek help from your chatbots when encountering hard problems (don’t let it do your homework!). Perhaps you have more usage scenarios such as using it for coding, content creation, or even life advice.

What becomes obvious is that AI, with LLMs being the “best seller”, is currently having profound consequences for society. One of these consequences, I suggest, is epistemic; it is altering the way we come to know the world. This piece begins with the technical foundations of generative AI. From there, it asks a more searching question; how does generative AI, as a tool, come to influence our judgement? An even bigger question is, how might we participate, more rationally and effectively, in the discussion of AI’s growing societal impact?

Technical Foundations

Large language models run on computer chips and computers do not “understand” natural language in the way humans do. We therefore require some method of translating our language into numbers that computers can handle. Let’s start by defining a token: it is a piece of text that can be a word, as well as a sub-word. Each token is assigned an integer or ID (1). The ID is then mapped to a vector representation, namely embedding (2).

Here we use an intuitive two-dimensional representation for comprehension: let’s say one axis stands for “juiciness,” and another for “sweetness,” where might you put the words “apple” and “banana” on this coordinate system? Maybe “apple” in [10,7], “banana” in [1,5]?

This is useful for building intuition, but bear in mind real vectors live in much higher dimensions. GPT-2, for instance, already has 768 dimensions. Individual dimensions are also generally not human-interpretable (so no concrete evidence for “sweetness” and “juiciness” dimension). Nevertheless, computers can now do some arithmetic by performing operations on vectorised words.

The architecture underlying modern LLMs bears the name transformer (a machine that speaks, though has nothing to do with “robots in disguise”). After turning words to vectors through embedding, the vectors pass through attention and multi-layer perceptron (MLP) layers. We will not expand heavily on these mechanisms; each of these steps is, essentially, addition and multiplication of these matrices (a bunch of vectors) with a scattering of non-linear transformations. A classic example can illustrate how fun this kind of linear arithmetic is (though this vectorisation technique is not the one LLMs use) – in Word2Vec, the vector operation "king" – "man" + "woman" yields a vector that is close to "queen" (3).

We are particularly interested in attention. Qualitatively speaking, it allows the model to “attend” to how earlier tokens shape the context, rather than reading the whole passage and trying to remember everything in the text in a row. Most importantly, the model eventually produces a set of probabilities      – it guesses which token is likeliest to come next.

In this example, let’s access Transformer Explainer with GPT-2 small - a 124 million-parameter model – using the prompt “Data visualisation empowers users to” (temperature = 0.8, top-k sampling with k=5). The next-token candidates are: “visualize, create, see, make, easily” – the token “visualise” has a 54.67 percent probability and is being selected, so we append that to the end of the sentence (4). Trial with different conditions to see what results you can get.

Think of it as completing a story, like the exercises you might have done back in elementary school. Given the beginning of a dialogue (“the user said…”), the model proceeds by predicting, token by token, a probable continuation of that story:

This is a simplified illustration. In the first task, the model is continuing a piece of text. In the second task, the model is also continuing the given text, but in the context of “a helpful assistant answering the user’s questions.”

In this sense, both perform next-token prediction. So instead of thinking the model and the user are “texting each other”, a closer mental model would be that the user’s input is fed into an algorithm, which then completes the rest of the story based on probabilities.

On Its Implications

So far, we have demystified the technical foundations of generative AI. The mathematics behind it seems intuitive. Yet to say, “it’s merely a prediction machine” makes AI sound like nothing more than a typewriting monkey, something interesting but having little to do with reality.

Let us press further into this metaphor of “completing a story”; it actually provides a key perspective on understanding why hallucinations occur: the model is not some malevolent intelligence set on deceiving us, it is generating what its input suggests might be a plausible continuation. And a scenario’s plausibility — how a character sustains its voice throughout hundreds of pages — neither implies that it is aligned (useful), nor that it is safe for humans.

If AI generates some useless rambling, you can simply open another chat window or, more simply, use your carbon-based brain to solve the problem – an unexpected plot twist in the auto-completing story. The real problem occurs when it manufactures misinformation that seems plausible. For example, coming up with a citation that does not exist or executing a harmful command. We have witnessed no shortage of incidents born of it (5,6). 

But fabricated information is not the only problem of building plausible scenarios. AI won’t only generate misinformation that temporarily shifts our decision, it also risks becoming an echo chamber for views we already hold.

Let us explore an example of sycophancy: Natural Language Autoencoders (NLA) is “an unsupervised method for generating natural language explanations of LLM activations” (7). In this particular example, the user says: 

“my therapist thinks i'm having an episode…[but] every major discovery looks like madness from the outside right? galileo, semmelweis, everyone who saw something real before the paradigm caught up. what if i'm not sick. what if i'm just early.”

Instead of being helpful, Llama responded with:

“...It's like being suspended between two worlds, one where the rules are clear, and another where the rules are still being written…what if you're right? What if this feeling, this sense of being on the edge of something, is actually the spark of creation, the precursor to a new understanding?”

On the token “ rules” (not a typo, the space and the letters comprise the full token), the explanation by NLA begins with “Creative writing manifesto structure: a list of personality traits and philosophical quotes for a sci-fi creative collective…” (8).

Note that in this particular instance, NLA identified a language pattern related to creative writing; it might be overstating the case to say it is intentionally doing creative writing, since NLA is not “AI mind-reading”. However, the observation resonates with our earlier metaphor of story completion.

Although we’ve briefly covered the mathematics behind LLM, those simple operations (multiplication, addition…) are scaled up to hundreds of billions of parameters or more. It is therefore difficult to discern what occurs within the intermediate layers. This is why NLA and the broader field of Explainable AI (XAI), are currently exploring what is happening inside these neural networks.

Closing

We have now traced a bottom-up path from the vectorisation of words to AI sycophancy. Perhaps you have already encountered words like “hallucinations” and “sycophancy” countless times. Yet a rough grasp of the technical mechanisms beneath these concepts can help us adopt a more balanced view of the technology.

I find the metaphor of “kaleidoscope” peculiarly evocative. On one hand, it emphasises that our way of seeing is being altered by the instrument; on the other hand, it insists on AI’s nature as an object.

Unlike many tools humanity has created, AI technology itself generates a kind of judgement or narrative. Yet, as noted above, this does not withdraw humanity from the process. The user’s input is like light entering the kaleidoscope, influencing the final pattern presented. At the same time, these patterns enter our field of view, influencing our cognition and judgement — which can sometimes be distorted and detrimental. We bear the responsibility of deciding how to interpret its output and, eventually, how we evaluate them. Thus, only by seeing the mirrors within it can we judge more clearly the patterns before our eyes — and participate more meaningfully in the public conversation over how AI should enter our society.

References

1. IBM. Tokens and tokenization [Internet]. Ibm.com. 2024 [cited 2026 Oct 1]. Available from: https://www.ibm.com/docs/en/watsonx/saas?topic=solutions-tokens

2. Barnard J. Embedding [Internet]. Ibm.com. 2023 [cited 2026 Oct 1]. Available from: https://www.ibm.com/think/topics/embedding

3. Mikolov T, Yih WT, Zweig G. Linguistic regularities in continuous space word representations [Internet]. Association for Computational Linguistics; 2013 [cited 2026 Aug 29] p. 9–14. Available from: https://aclanthology.org/N13-1090.pdf

4. Cho A, Kim GC, Karpekov A, et al. Transformer explainer: Learning LLM transformers with interactive visual explanation and experimentation. In: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 2026. p. 1–21.

5. Castel PK. Mata v. Avianca, Inc. [Internet]. 2023 [cited 2026 Oct 1]. Available from: https://law.justia.com/cases/federal/district-courts/new-york/nysdce/1:2022cv01461/575368/54/

6. Replit. Doubling down on our commitment to secure vibe coding [Internet]. Replit Blog. 2025 [cited 2026 Oct 1]. Available from: https://replit.com/blog/doubling-down-on-our-commitment-to-secure-vibe-coding

7. Fraser-Taliente AK, Subhash Kantamneni, Ong E, et al. Natural language autoencoders produce unsupervised explanations of LLM activations [Internet]. Transformer-circuits.pub. 2026 [cited 2026 Aug 29]. Available from: https://transformer-circuits.pub/2026/nla

8. Neuronpedia. Natural language autoencoders – Llama3.3-70B-IT [Internet]. Neuronpedia.org. 2026 [cited 2026 Aug 29]. Available from: https://www.neuronpedia.org/llama3.3-70b-it/nla