Local AI Models

Curriculum

1. 1

   Understand open weights

   Separate weights, architecture, training, inference, licenses, and the wider meaning of open source AI.

   1. 01 Weights are learned numbers
   2. 02 Separate training from inference
   3. 03 Architecture, weights, and runtime
   4. 04 Open weights and open source AI
   5. 05 Read the model card and license
   6. 06 Check your understanding: open weights
2. 2

   Choose a model that fits

   Compare capabilities, parameter counts, quantization, formats, context, memory, and task quality.

   1. 07 Start from the task
   2. 08 Parameters and memory
   3. 09 Understand quantization
   4. 10 Safetensors and GGUF
   5. 11 Context and the KV cache
   6. 12 Build a small evaluation
   7. 13 Check your understanding: choosing a model
3. 3

   Run models locally

   Install Ollama, manage model files, use the local API, stream responses, and measure performance.

   1. 14 Choose a local runtime
   2. 15 Install and check Ollama
   3. 16 Pull and run your first model
   4. 17 Call the local chat API
   5. 18 Call Ollama from Node.js
   6. 19 Stream and measure responses
   7. 20 Check your understanding: running models locally
4. 4

   Build a local AI feature

   Create a Node.js summarizer with structured output, timeouts, cancellation, fallback behavior, and focused tests.

   1. 21 Define the feature boundary
   2. 22 Request structured output
   3. 23 Add timeout and cancellation
   4. 24 Add a deterministic fallback
   5. 25 Use tools with a permission boundary
   6. 26 Keep the provider swappable
   7. 27 Test the local AI feature
   8. 28 Check your understanding: building a local feature
5. 5

   Operate local AI responsibly

   Review privacy, model supply chains, permissions, upgrades, costs, evaluations, and production boundaries.

   1. 29 Map the complete data flow
   2. 30 Verify model files and code
   3. 31 Pin and upgrade models
   4. 32 Understand the real cost
   5. 33 Complete the local summarizer
   6. 34 Check your understanding: operating local AI