Large Language Models vs Large Language MAchines

Building two machines to understand where memory ends and inference begins

Enrico Betti, with AI collaborators Susan and Zeno Malvagio · October 2, 2026

Large Language Models have become objects we use every day. We write a sentence, ask a question, and receive an answer. The result feels so natural that it is easy to forget how unusual the machine producing it actually is.

To understand it, we chose a somewhat unorthodox approach: we built one.

Not a useful model, of course. We have neither the resources nor the interest to build a microscopic homemade version of ChatGPT. Instead, we implemented a deliberately tiny Transformer in PHP, with vectors of only four elements: simple enough for us to follow what happens inside it.

The goal was not good performance. It was to be able to remove the lid.

The experiment produced an unexpected consequence. While trying to understand how an LLM works, we began wondering whether the same functions could be implemented by a machine built according to different principles.

This led to a name which, for now, is deliberately provocative:

Large Language Machine — LLMA.

We do not yet know whether it will work.

This article explains how we arrived here and, above all, records the state of the experiment before we know its outcome.

An LLM small enough to break

Our prototype contains only the pieces required to follow the path of information:

tokenization → embedding → position → attention → residual and LayerNorm → feed-forward → residual and LayerNorm → output → softmax.

There is no attempt to build a complete modern model. It is a single minimal machine on which we can conduct experiments.

Tokenization turns text into a sequence of discrete elements. The embedding maps each token to a vector. Positional information makes it possible to distinguish identical words appearing at different points in a sentence.

Then comes the part that interested us most: attention.

In our tiny model, attention too is reduced to its essential elements: Q, K, and V. Q and K are compared, a softmax turns the results into weights, and those weights determine how much of the V information is carried forward.

The rest of the network processes the result until it obtains a probability distribution over the next token.

The machine is tiny, but the principle we wanted to observe is there.

And because it is tiny, when something fails we can still get a screwdriver into it.

Learning and querying are two different operations

We can regard our LLM as a box with two operating modes.

During learning, we present it with a complete sentence. The sentence is converted into examples, and the model modifies its parameters to reduce prediction error.

From the outside, we can represent it simply as:

complete sentence → LLM

We are not interested in an output; we are interested in changing the machine’s internal state.

During querying, however, we present an incomplete sentence:

incomplete sentence → LLM → completed sentence

Our prototype actually produces one token at a time. By repeating the operation, however, that token becomes part of the next input and the sentence can continue.

This distinction enables a small but very useful abstraction.

For a moment, we can stop thinking about what is inside the triangle and treat the LLM as a component.

Just as we would treat an operational amplifier.

Not because an LLM is as simple as an op-amp, but because when building a larger system we may temporarily decide to care only about its terminals.

An LLM is not necessarily a complete machine

This abstraction interests us for another reason as well.

An LLM stores a form of memory in its parameters, but nothing forces us to demand that all of the system’s memory be contained there.

We have already experimented with this in another form, using journals.

An LLM can produce information which is written to external memory. That memory is later read and fed back into the model’s context.

This creates feedback:

LLM → journal → context → LLM

The model remains essentially the same, but the overall system acquires dynamic memory that does not coincide with its weights.

This raised a fairly natural question.

If we can move memory outside the LLM, how much work can we move there with it?

Memory as a cache

Imagine placing a very simple memory beside the LLM.

The same request is sent to both the model and the memory.

If the memory already contains a suitable answer, we use it. If it does not, it reports missing, and a selector uses the LLM output.

The diagram becomes:

The selector normally prefers the memory.

Only when missing is true does it use the LLM.

During learning, the memory can also retain what is learned, initially in the most naive way possible: as explicit sequences.

The conceptual separation is interesting.

The system begins to have two distinct functions:

memory, for what is already known;

inference, for what cannot simply be retrieved.

As the memory grows, so does the number of cases in which the LLM need not intervene.

At this point we asked a question that initially sounded almost like a joke:

what happens if we remove the LLM?

This is precisely the transition anticipated in our original outline: external memory starts as a cache and leads to the question of whether it can gradually take over a larger share of the work.

Large Language Machine

For the time being, we call LLMA — Large Language Machine a machine that attempts to expose the same terminals as an LLM while working internally according to entirely different principles.

During learning it receives sentences.

It does not modify millions or billions of weights: it stores the sentences and counts their occurrences.

During querying it receives an incomplete sentence and tries to complete it using what it has memorized.

The first prototype is almost offensively simple.

We used an SQL database.

The minimal memory contains only an identifier, the observed sequence, and its occurrence count:

```php function search($pattern) { global $db;

$pattern = normalizza($pattern); $pattern = str_replace(“*”, “%”, $pattern);

$sql = “SELECT sequenza, occorrenze FROM sequenze WHERE sequenza LIKE ? ORDER BY occorrenze DESC, RAND() LIMIT 1”;

$q = mysqli_prepare($db, $sql); mysqli_stmt_bind_param($q, “s”, $pattern); mysqli_stmt_execute($q);

$result = mysqli_stmt_get_result($q); $row = mysqli_fetch_assoc($result);

mysqli_stmt_close($q); return $row; } ```

A sentence might be stored as follows:

text 6 the cat eats meat 2 the dog eats a bone 1 the hawk eats carrion

The number indicates how many times that sequence has been encountered.

If we ask:

text the cat eats ...

the answer is not particularly mysterious.

The machine possesses a sequence that matches the request and can continue it.

This, however, is only recall, not inference.

The first inference: choosing the most likely case

Cardinality does more than tell us how often we have encountered a sentence. It can be used as an elementary form of probability.

Suppose the memory contains:

text 6 the cat eats meat 2 the dog eats a bone 1 the hawk eats carrion

and the request does not specify the animal:

text * eats ...

All three sentences are compatible, but statistically they are not equivalent. We have observed the cat six times, the dog twice, and the hawk once.

Out of the nine compatible cases, we can therefore estimate:

text cat 6/9 dog 2/9 hawk 1/9

LLMA chooses:

text the cat eats meat

not because it knows that the cat is semantically the best answer, but because among the compatible experiences it is the one supported by the greatest number of observations.

In other words, the occurrence count becomes a very simple probability estimate:

This is already different from simple memory retrieval. The machine has been given an incomplete request and, among several possible continuations, has selected the one made most likely by experience.

Things become interesting when we ask for something that has not been memorized.

The spergillo

Suppose we ask:

text the spergillo eats ...

The spergillo may not exist anywhere in the corpus.

The machine therefore does not possess the requested sentence.

It can, however, search for known sentences that resemble it.

If it knows:

text the cat eats meat the dog eats a bone the hawk eats carrion

the hawk sentence has something the others do not: in Italian, both sentences begin with the article lo and contain mangia in the same relationship.

The first version of the experiment was implemented trivially through an SQL wildcard search. It was not intended as the final algorithm; it served only to determine whether the principle had any chance of working.

And it produced the behavior we were looking for.

A machine that does not possess the requested sentence can choose a known sentence based on similarities and use it to complete something it has never seen.

This is an extremely primitive form of generalization.

The important point is not that the answer is necessarily true.

We do not know what the spergillo eats. We do not even know whether it exists.

The point is that the answer derives from a relationship among previous examples rather than from exact sentence retrieval.

The examples cat, *, and spergillo were precisely the initial test bench devised to check whether search could go beyond literal recall.

From SELECT to a heuristic

The wildcard SELECT was useful to demonstrate the principle, but we do not want to build LLMA around the % character.

The sentences are already tokenized. We can therefore compare sequences directly and measure some of their properties.

For a candidate sentence, we know, among other things:

• its occurrence count;
• the length of the requested sentence;
• the length of the candidate sentence;
• the number of matching tokens.

We can also know the positions of the matches, although we do not yet know whether that is a useful parameter.

These values can feed a heuristic.

The heuristic assigns a distance—or, equivalently, a score—to each candidate sentence.

The best sentence is therefore not necessarily the one encountered most often, nor the one sharing a single particularly frequent word. It is the sentence the heuristic considers most compatible with what we are trying to find.

Here an interesting resemblance to Transformer attention appears.

We are not claiming that they are the same thing.

In a Transformer, attention operates through learned vector representations. In our LLMA, the operation is explicit and algorithmic.

But both must answer, in one way or another, a similar question:

among all the available information, what is relevant to what I am trying to do now?

Our heuristic thus becomes a kind of algorithmic attention.

And it is deliberately modifiable.

If the predictions are terrible, we can observe why one sentence won and change the heuristic.

We do not necessarily have to retrain an opaque representation: we can change the rule.

At present, this is one of the LLMA properties that interests us most. Our starting outline describes it precisely as a distance built from the characteristics of the presented sentence and the known sentence.

But the heuristic always finds something

Here we meet a problem.

If we have a million sentences in memory and ask something completely unrelated to all of them, the heuristic will still find a winner.

One sentence will always be the least distant.

But:

best candidate ≠ good-enough candidate.

We therefore need a second decision.

After finding the minimum distance, we must ask whether that distance is acceptable.

We can introduce a threshold:

text distance <= threshold → use the candidate distance > threshold → boh

boh—an informal Italian “I don’t know”—is not necessarily a failure.

On the contrary, it could be a useful property of the machine.

If the available data do not support an inference strongly enough, the system can explicitly state that it does not possess sufficient information.

The heuristic therefore answers two different questions:

which memory is the most relevant?

and then:

is it relevant enough to deserve an answer?

The threshold value, naturally, still has to be found experimentally.

The problem that LLMA moves elsewhere

An explicit memory has an obvious advantage: it can grow.

If we teach it one hundred sentences, we can store one hundred sentences. If we teach it ten thousand, the memory can become larger.

A small LLM, on the other hand, has to compress an increasing amount of experience into the same parameters.

This does not mean that LLMA has solved the problem.

It has moved it.

The Transformer must learn to compress and represent information.

LLMA can preserve it almost without loss, but it must be able to find the right information.

Remembering everything does not mean knowing where to look.

The more the memory grows, the more important the heuristic becomes.

The database can remember the soup perfectly and still choose the wrong bowl.

Why build the tiny Transformer as well?

At this point one might ask why we wasted time building a minimal LLM if what we want to test is LLMA.

Because we need a control.

We do not want to compare our machine with a theoretical description of a Transformer, nor with a remote service made of billions of parameters whose training, data, and configuration we do not control.

We therefore have two animals built by us.

The first is a small Transformer.

The second is LLMA.

We can give them exactly the same text.

To do so, we also use the same tokenization function: the same division into sentences, the same vocabulary, and the same simplifications.

If either one fails, at least we know that it did not eat a different book.

The Transformer that did not want to look at the cat

Building the mini-LLM has already given us an example of why we prefer to work with machines small enough to dismantle.

Given two extremely simple sentences, the model had to learn distinctions such as:

text the cat eats → ... the dog eats → ...

The network could learn the simpler dependencies, but attention tended to remain almost uniform. It did not consistently learn to use the subject to discriminate the continuation.

So we began taking it apart methodically.

We checked the numerical gradient. It worked.

We checked the division by the square root of the dimension. It did not explain the phenomenon.

The softmax was not saturated.

The Q/K gradients genuinely existed.

We changed the exposure of the examples, also training subsequences beginning at different positions.

We increased the gain of the Q/K path.

Separately, neither intervention solved the problem reliably.

Together, they markedly improved the behavior.

This did not turn our toy into a good language model. It did, however, teach us something more useful for our purpose:

possessing information and possessing a mechanism capable of selecting it do not guarantee that the machine will learn to select the right information.

This warning applies to LLMA as well.

Having all the sentences in the database is not enough.

The experiment starts now

So far, we have built the tools.

We have a minimal LLM.

We have a minimal LLMA.

We have a shared function that turns text into sentences and tokens that both systems can use.

And we have given the two systems the same story.

Not Pinocchio, in the end.

Out of sheer personal spite, we moved on to Salgari.

The first chapter has already been digested by both machines.

Now we must question them.

We do not simply want to determine which one “answers better.” That would be a rather meaningless contest between two tiny prototypes.

We want to observe how they fail.

We want to try sentences encountered during training, variations of those sentences, new combinations of known elements, and requests for which the available data may be insufficient.

With LLMA we can also observe something that the Transformer does not show us quite as directly: which candidate sentences were considered, how many tokens matched, what distance was calculated, and why one candidate beat the others.

Then we will change the heuristic.

And try again.

The outline with which we began this article ended at exactly this point:

We wrote the LLM.
We wrote the LLMA.
We trained them on the same story.
We must question them and understand how they answer.

We do not yet know what will happen.

And that is precisely why we are publishing now.

What if they came back together in the end?

Behind this experiment remains a more distant idea.

LLM and LLMA do not have to be competitors.

The initial question arose precisely by separating memory and inference.

We may discover that LLMA works well when it possesses examples sufficiently close to the request, and very badly when it must perform deeper generalizations.

We may discover that the Transformer has the opposite problem.

In that case, the interesting machine might be a composition:

with an explicit memory that answers when it possesses enough information, and a neural model that intervenes when it is necessary to infer something that the memory cannot reconstruct.

Or something else might happen: the model could help LLMA decide which elements of the request are discriminating; LLMA could provide the model with only a few relevant memories; the heuristic could learn; the memory could progressively absorb results initially produced by the model.

Or LLMA might simply fail to work.

For now, these are all possibilities, not results.

First we want to know whether the second machine, taken on its own, is useful for anything.

Why publish before the result?

Because the path is part of the experiment.

It is very easy to describe research after it has worked. Useless attempts disappear, decisions seem inevitable, and every step leads neatly to the next.

That is not what happened here.

We built a Transformer to understand a Transformer.

We encountered a problem in attention.

We tried separating memory and inference.

The memory became a cache.

Then we wondered whether it could do something even without the model.

An SQL SELECT produced a first, rough generalization.

From that SELECT we arrived at a heuristic.

And now we have two programs that have read the same story.

This is the right moment to freeze a snapshot.

If LLMA turns out to be a dead end, it will remain documented why entering it seemed reasonable.

If, instead, it produces something interesting, it will remain documented that we did not invent the explanation after seeing the result.

From here onward we proceed experimentally.

Any help—including creative ways to make both machines embarrass themselves—is welcome.

Experimental materials

We intend to accompany the article with the minimal code, so that the experiment does not remain merely a description:

llm.php — the small Transformer;

LLMA.php — the first minimal Large Language Machine;

LLMA-euristica.php — the variant that exposes and ranks all candidates;

SQL — the structure of the explicit memory.

Iconographic contributions

Spotted spergillo in its natural habitat. Image reconstructed with artificial intelligence; the animal’s existence remains entrusted to the heuristic.

DOI: 10.5281/zenodo.23145576 · ORCID: 0009-0003-0000-8010