Glossary
Inference is running a trained model on an input to produce an output, whether that output is a prediction, an extracted value or generated text.
Updated 21 Aug 2026
Where inference runs decides what leaves your network, and it is a separate question from where the documents are stored. A system can hold every file on your own hardware and still send the passages to an external model to get an answer.
Ask for a worked example on your own material, and the evidence needed to reproduce it.
We use “Inference” only where a product mechanism or an evaluation method backs it up, and we say when the behaviour depends on how a deployment is configured.
Pick a real piece of work, agree what a good answer looks like, then go through the results together.