What it does
Kiln is an app and library for teams building AI systems. The desktop app is a free download for macOS, Windows and Linux, and the library is a Python package under the MIT license that can be used to run Kiln tasks from your own code. The site groups the product into four areas. Build covers RAG with document indexing, skills, tools and MCP, sub-agents, structured JSON output and reasoning. Evaluate covers LLM-as-judge evals, an AI eval builder, golden datasets and human ratings. Improve covers automatic prompt optimization, fine-tuning into smaller models, synthetic data generation, and an AI assistant you can chat with to run experiments. Collaborate covers datasets versioned in your Git repo and feedback from the wider team. The site says it works with more than 200 models, in the cloud or locally, and offers a model library of tested capabilities. Product managers, QA staff and subject experts can contribute ratings, feedback and data without coding.
Capabilities
- Evals with LLM-as-judge, golden datasets and human ratings
- Automatic optimization of prompts against your evals
- Fine-tuning into smaller models and synthetic data generation
- RAG, tools and MCP, skills and sub-agents
- Datasets versioned through Git, with team feedback and ratings
- MIT-licensed Python library for running tasks in production
When to use it
- You want to measure quality with evals before changing a prompt or model
- You need to fine-tune a model or generate synthetic data without writing much code
- Non-engineers on your team need to rate outputs and give feedback
- You want to run your tasks from your own Python code
