What it does
DwarfStar (ds4) is a small native inference engine for running a handful of large language models on hardware people can actually own. It is deliberately narrow rather than a general GGUF runner: it is tuned for DeepSeek V4 Flash (including an experimental vision model), DeepSeek V4.1 Flash, DeepSeek V4 PRO, GLM 5.2 and 5.3, GLM 5.3 Flash and Qwen3.8 Flash Next, and it uses the GGUF files that the project itself produces. Metal on Macs with 96 GB or more is the primary target, and smaller machines can use SSD streaming. NVIDIA CUDA, including multi-GPU setups and the DGX Spark, and ROCm on Strix Halo systems are also supported. Several machines can be combined, for example two Macs linked over RDMA with tensor parallelism, or pipeline parallelism to pool memory. You build it with make, download a model with a provided script, and then use an interactive command-line chat, a one-shot prompt, ds4-agent (a built-in coding agent) or ds4-server, a local server that clients such as Pi, OpenCode, Codex CLI and Claude Code can connect to. Further options include an on-disk cache of conversation state, speculative decoding, adjustable thinking effort and a built-in evaluation tool. The page calls the software very fast changing and of beta quality. It also discloses that the project is developed with strong assistance from AI coding agents.
Capabilities
- Runs DeepSeek V4, GLM and Qwen3.8 models locally
- Supports Apple Metal, NVIDIA CUDA and AMD ROCm
- SSD streaming for machines with less memory
- Built-in coding agent and a local server for coding-agent clients
- Spreads a model across several machines
- Command-line chat with adjustable thinking effort
When to use it
- You want to run a large open-weight model on a high-memory Mac, DGX Spark or Strix Halo machine
- You want to point a coding-agent client at a model running on your own machine
- You are comfortable building from source and with beta-quality software
