Computer Vision

A context window is the maximum amount of text (measured in tokens) that an LLM can process in a single interaction, encompassing both the input prompt and the generated output. Larger context windows allow models to handle longer documents, maintain extended conversations, and reason over more information at once. Context window sizes have grown rapidly — from 4K tokens in early GPT models to over 1M tokens in current models like Claude.

A diffusion model is a type of generative AI that creates data by learning to reverse a gradual noise-adding process. During training, the model learns to progressively denoise random noise into coherent outputs like images, audio, or video. Diffusion models power tools like Stable Diffusion, DALL-E, and Midjourney, and have become the dominant architecture for high-quality image generation.

Generative AI is a category of artificial intelligence that creates new content — text, images, code, music, or video — rather than just analyzing or classifying existing data. Powered by architectures like transformers and diffusion models, generative AI has transformed software development with tools like GitHub Copilot, Claude, and Cursor. It represents a shift from AI as a classification tool to AI as a creative collaborator.

The transformer is a neural network architecture introduced in the 2017 paper "Attention Is All You Need" that revolutionized natural language processing. Unlike recurrent networks, transformers process entire sequences in parallel using a self-attention mechanism, which allows them to capture long-range dependencies efficiently. Virtually all modern LLMs, including GPT and Claude, are built on the transformer architecture.

Chain of Thought (CoT) is a prompting technique that encourages an LLM to break down complex reasoning into intermediate steps before arriving at a final answer. By explicitly reasoning through each step, models achieve significantly better accuracy on math, logic, and multi-step problems. Extended thinking and "thinking" tokens in models like Claude represent a built-in form of chain-of-thought reasoning.

Natural Language Processing (NLP) is a branch of AI focused on enabling computers to understand, interpret, and generate human language. NLP powers applications like chatbots, translation services, sentiment analysis, and text summarization. Modern NLP has been transformed by transformer-based models, which achieve remarkable performance on tasks that previously required extensive hand-crafted rules.