Microsoft Foundry Local (foundry) is a powerful command-line interface (CLI) tool for running Large Language Models (LLMs) locally. It allows you to deploy an OpenAI-compatible API directly on your machine, manage model caching, and configure execution environments.
The CLI is organized into three logical groups: service (managing the background process), model (working with AI models), and cache (managing files and storage).
1. Getting Started and General Help
If you have just installed Foundry, your first commands to verify functionality should be:
foundry --version– Check the installed version.foundry --help– Display the main help menu.
To get detailed information about specific sections, use:
foundry model --helpfoundry service --helpfoundry cache --help
2. Service Management
The service is a background process responsible for running the API and loading models into memory.
- Status and Information:
foundry service status– Check if the service is running and get the local API endpoint (e.g.,http://127.0.0.1:5273/v1).foundry service ps– List models currently loaded into RAM or VRAM.
- State Control:
foundry service start– Start the service.foundry service stop– Stop the service.foundry service restart– Restart the service.
- Configuration and Logs:
foundry service logsorfoundry service diag– View real-time service logs.foundry service set– Modify internal service settings.
3. Working with Models
The model section is used for searching, downloading, and running neural networks (such as Phi-4, Qwen, or Mistral).
Basic Operations
foundry model list– View all models available in the catalog.foundry model info <model_name>– Get detailed information about a model.- Pro Tip: Use the
--licenseflag to view the model’s usage terms.
- Pro Tip: Use the
foundry model download <model_name>– Download the model to the cache only (without running it).foundry model load <model_name>– Pre-load a model into memory. This ensures the API responds instantly upon the first request.foundry model unload <model_name>– Unload a model from memory to free up resources.foundry model run <model_name>– The “all-in-one” command: downloads (if necessary), loads, and starts an interactive chat directly in the console.
Advanced Filtering
When searching the catalog, you can use filters to find models compatible with your hardware:
foundry model list --filter device=GPU– Show only GPU-accelerated models.foundry model list --filter device=!GPU– Exclude GPU models.foundry model list --filter task=chat-completion– Find models designed for chatting.foundry model list --filter alias=phi*– Find all models in the Phi family using wildcards.
4. Cache Management
AI models can take up dozens of gigabytes. Cache management helps you monitor and control disk space.
foundry cache location– Find the path to the folder where models are stored.foundry cache list– List all models currently downloaded.foundry cache cd <path>– Crucial command: Allows you to move the heavy model directory to a different drive (e.g., from a systemC:drive to a largerD:drive).foundry cache remove <model_name>– Delete a specific model from your disk.foundry cache clear– Completely wipe the cache.
5. Diagnostics and Troubleshooting
If Foundry is behaving inconsistently, use these tools to gather data for bug reports:
foundry zip-logs– Automatically bundles all logs into a single ZIP archive.foundry --help --verbose– Displays hidden commands and detailed CLI parameters.
6. The Future of Foundry: Changes in New Versions
In recent preview versions (branch 0.10.x and higher), Microsoft has begun simplifying the syntax. If the standard commands don’t work, try these new shortened alternatives:
| Old Command | New Command (v0.10+) |
|---|---|
foundry model run | foundry run |
foundry service start | foundry server start |
foundry service diag | foundry server logs |
foundry model info | foundry model show |
foundry cache remove | foundry cache rm |
Conclusion
Microsoft Foundry Local is an excellent way to integrate AI into your projects. Once you obtain the API address via foundry service status, you can use it in the OpenAI SDK, LangChain, Open WebUI, or your own Python scripts by simply swapping the base_url.
To find out exactly which commands are available for your specific build, run foundry --version.