# A Short Personal Reference Guide to LLM Concepts & Limitations ## Core Concepts ### Foundation - **Large Language Models**: Neural networks trained on vast text data to predict next tokens - **Architecture**: Primarily transformer-based with attention mechanisms - **Parameters**: Model size/capacity (billions to trillions) - **Training**: Self-supervised learning on text corpora - **Inference**: Generation of text responses to prompts ### Key Capabilities - Text comprehension and generation - Pattern recognition across diverse domains - In-context learning from examples - Reasoning through chain-of-thought processes - Knowledge retrieval from training data ### Interaction Methods - **Prompting**: Crafting effective instructions - **RAG**: Retrieval-Augmented Generation for factual grounding - **Fine-tuning**: Adapting models to specific tasks/domains - **Function calling**: Enabling tool use and API integration ## Critical Limitations ### Knowledge Constraints - **Knowledge cutoff**: Limited to training data timeframe - **Hallucinations**: Confidently generating false information - **Shallow expertise**: Broad but often superficial domain knowledge ### Reasoning Gaps - **Mathematical reasoning**: Struggles with complex calculations - **Spatial reasoning**: Limited understanding of physical space/geometry - **Causal reasoning**: Correlation vs. causation confusion - **Logical consistency**: Contradictions across longer contexts ### Context Handling - **Context window**: Limited text history (tokens) maintained - **Memory**: No persistent memory between sessions - **World model**: Incomplete understanding of physical reality ### Social Limitations - **Theory of mind**: Limited understanding of human intentions/beliefs - **Cultural nuance**: Struggles with complex cultural contexts - **Ethical reasoning**: Simplified moral frameworks ### Technical Constraints - **Latency**: Generation speed limitations - **Computing costs**: Resource intensity of large models - **Error recovery**: Difficulty recognizing and correcting mistakes - **Multimodal integration**: Limited understanding across modalities ## Mitigation Strategies ### Enhancing Reliability - Tool integration for verified calculations/lookups - Chain-of-thought prompting for complex reasoning - Knowledge retrieval from external databases - Explicit verification steps for factual claims ### Improving Output - Clear instructions with specific format expectations - Example-based prompting for consistency - Breaking complex tasks into simpler components - Focused domain-specific prompting ### Safety Practices - Avoid overreliance on LLM outputs for critical decisions - Verify factual claims through independent sources - Maintain human oversight for sensitive applications - Recognize appropriate use cases and limitations