What it means
A transformer is a type of neural network architecture, introduced in 2017, that underlies most modern large language models. Its key mechanism, called attention, lets the model weigh how relevant each part of the input is to every other part, which helps it handle context in long texts.
Real-world example
When an LLM reads the sentence "The museum returned the painting to the family because it had been looted", the attention mechanism helps it connect "it" to "the painting" rather than "the museum". This ability to track relationships across a text is why transformer-based models handle summarisation and translation far better than earlier approaches.
Why it matters for AI adoption
Most decision-makers do not need to know how transformers work in detail. It is useful to know that the term describes the architecture behind today's LLMs, which explains both their strengths (handling context) and some practical limits, such as the cost of processing very long inputs.
Questions to ask
- Do we actually need a large transformer-based model for this task, or would a smaller, simpler model be enough?