Hallucination

A neural network is a computational model inspired by the human brain, consisting of layers of interconnected nodes (neurons) that process data by adjusting weighted connections during training. Deep neural networks with many layers form the foundation of modern AI, powering everything from image recognition to language understanding. Common architectures include feedforward networks, convolutional networks (CNNs), and transformers.

A context window is the maximum amount of text (measured in tokens) that an LLM can process in a single interaction, encompassing both the input prompt and the generated output. Larger context windows allow models to handle longer documents, maintain extended conversations, and reason over more information at once. Context window sizes have grown rapidly — from 4K tokens in early GPT models to over 1M tokens in current models like Claude.

Natural Language Processing (NLP) is a branch of AI focused on enabling computers to understand, interpret, and generate human language. NLP powers applications like chatbots, translation services, sentiment analysis, and text summarization. Modern NLP has been transformed by transformer-based models, which achieve remarkable performance on tasks that previously required extensive hand-crafted rules.

In the context of AI language models, a token is the basic unit of text that a model processes — typically a word, subword, or character depending on the tokenizer. LLM pricing, context windows, and rate limits are all measured in tokens. Understanding tokenization is essential for optimizing costs and staying within model context limits when building AI-powered applications.

An embedding is a dense numerical vector representation of data — such as text, images, or code — in a high-dimensional space where semantically similar items are positioned closer together. Embeddings are fundamental to semantic search, recommendation systems, and RAG pipelines. They are generated by specialized models and typically stored in vector databases for efficient similarity lookups.

Fine-tuning is the process of further training a pre-trained AI model on a smaller, domain-specific dataset to adapt it for a particular task. Instead of training from scratch, fine-tuning adjusts existing model weights, which is significantly cheaper and faster. Common approaches include full fine-tuning, LoRA (Low-Rank Adaptation), and instruction tuning for aligning model behavior with specific requirements.