#LanguageModel

Self-Attention

By Ethan Caldwell Jul 12, 2026 • 1 min read

Self-attention lets each element in a sequence weigh and combine information from other elements in that same sequence.

Padding

By Ethan Caldwell Jun 24, 2026 • 1 min read

Padding adds placeholder values so variable-length inputs can be processed together in a fixed-size machine learning batch.

Natural Language Understanding (NLU)

By Ethan Caldwell Jun 20, 2026 • 1 min read

NLU enables computer systems to identify meaning, intent, entities, and relationships in human language.

Natural Language Generation (NLG)

By Ethan Caldwell Jun 17, 2026 • 1 min read

NLG is the branch of AI that produces human-readable language from data, prompts, representations, or structured plans.

Multi-Head Attention

By Ethan Caldwell Jun 9, 2026 • 1 min read

Multi-head attention runs several attention operations in parallel so a model can capture different relationships in the same input.

Generative Pre-Trained Transformer (GPT)

By Marcus Lee Apr 29, 2026 • 1 min read

GPT is a family of transformer language models pre-trained to predict text and adapted to generate useful responses from prompts.

Causal Language Modeling (CLM)

By Ethan Caldwell Mar 3, 2026 • 1 min read

Causal language modeling trains a model to predict each next token without accessing future tokens in the sequence.

BLEU Score

By Ethan Caldwell Feb 28, 2026 • 1 min read

BLEU score evaluates generated text by measuring n-gram overlap with one or more human-written reference translations.

BERT (Bidirectional Encoder Representations from Transformers)

By Ethan Caldwell Feb 24, 2026 • 1 min read

BERT is an encoder-only transformer model that learns language representations by using context from both directions.

Autoregressive Model

By Ethan Caldwell Feb 13, 2026 • 1 min read

An autoregressive model predicts each next value or token using values that appeared earlier in the sequence.

Attention Mask

By Ethan Caldwell Feb 6, 2026 • 1 min read

An attention mask tells a transformer which token positions it may consider and which positions should be ignored.