Local models are great because, after the first download, you can use them anywhere — no internet required. Everything you work with stays on your device, and no “big brother” sees your data or queries. Responses come without the usual delays associated with cloud services, and computing costs are reduced to the price of electricity, since everything runs on your own hardware.
Models adapt to the device: Foundry Local selects the optimal version for CPU, GPU, or NPU, ensuring maximum speed and efficiency.
🛠 Foundry Local
Foundry Local by Microsoft simplifies running local models.
Some AI tasks run locally, while others use the cloud when it makes sense.
You can work with Foundry Local via:
- CLI — simple commands in Terminal or PowerShell
- SDK — familiar libraries, including an OpenAI-compatible REST API
- Visual Studio Code AI Toolkit — models right in your IDE
The model catalog includes variants optimized for local execution. If a model is already downloaded, it launches instantly from cache. The local service accepts requests like a web service, allowing you to call models from scripts or integrate them into your own tools.
⚙️ System Requirements
- Operating system: Windows 10/11 (x64/ARM), Windows Server 2025, or macOS
- Memory and disk: at least 8 GB RAM and 3 GB disk (16 GB / 15 GB recommended)
- Network: internet required only for the first model download
- Acceleration (optional): NVIDIA (2000+ series), AMD (6000+), Intel iGPU, Qualcomm Snapdragon X Elite, Apple Silicon
- Administrator rights: required for installation
💾 Installing Foundry Local
1. Via CLI
Windows:
winget install Microsoft.FoundryLocal --scope machine
If installation fails — download the .msix package and dependencies, then install via PowerShell with administrator rights.
macOS: use the instructions from the official documentation.
2. Downloading the installer
Find the latest version in the Foundry Local GitHub repository and install manually.
After installation, check the help:
foundry help
🚀 Running Your First Model
- Launch a model in the terminal (for example,
phi-3.5-mini). - On the first run, it will download locally (takes a few minutes).
- On subsequent runs, it will load from cache.
Example:
foundry model run phi-3.5-mini
To run GPT-OSS-20B:
foundry model run gpt-oss-20b
⚠ Requires NVIDIA GPU with 16 GB+ VRAM and Foundry Local 0.6.87+.
Check your version:
foundry --version
🖥 Foundry Local CLI — Reference
The Foundry Local Command-Line Interface (CLI) is designed to manage models, the service, and the local cache.
🔎 Overview
See all available commands:
foundry --help
Command categories:
- Model — manage and run models
- Service — manage the Foundry Local service
- Cache — manage the cache
📦 Model Commands
| Command | Description |
|---|---|
foundry model --help | Help for model commands. |
foundry model run <model> | Run a model (downloads if necessary). |
foundry model list | List available models. |
foundry model info <model> | Detailed info about a model. |
foundry model download <model> | Download a model into cache without running it. |
foundry model load <model> | Load a model into the service. |
foundry model unload <model> | Unload a model from the service. |
⚙️ Service Commands
| Command | Description |
|---|---|
foundry service --help | Help for service commands. |
foundry service start | Start the service. |
foundry service stop | Stop the service. |
foundry service restart | Restart the service. |
foundry service status | Check status. |
foundry service ps | List loaded models. |
foundry service logs | View logs. |
foundry service set <options> | Configure (e.g., GPU). |
🗂 Cache Commands
| Command | Description |
|---|---|
foundry cache --help | Help for cache commands. |
foundry cache location | Show cache path. |
foundry cache list | List cached models. |
foundry cache remove <model> | Remove a model from cache |
foundry cache cd <path> | Change cache directory. |
🚀 Usage Examples
Quick model run
# Download and run a model
foundry model run phi-4-mini
# Show model info
foundry model info phi-4-mini
# Download without running
foundry model download phi-4-mini
Service management
foundry service status
foundry service ps
foundry service restart
Working with cache
foundry cache list
foundry cache remove old-model
foundry cache cd /path/to/larger/drive
🔧 Advanced Features
# View model license
foundry model info phi-4-mini --license
# Generate diagnostic logs
foundry zip-logs
# Configure GPU usage
foundry service set --gpu 0