Autoregressive Model - Page 6

An autoregressive model generates or predicts a sequence one element at a time, conditioning each new step on earlier elements. In a language model, the next token is selected from probabilities based on the prompt and tokens already generated; in time-series analysis, a future value may depend on previous observations. This structure supports flexible generation and variable-length output, but it creates sequential inference that can be slower than producing all outputs in parallel. Errors can also compound because later predictions depend on earlier generated content. Sampling strategy, context length, caching, and stopping rules strongly influence results even though the underlying model parameters remain unchanged.