This tutorial assumes you can already load Muse Glimmer 30B locally. If you have not done that yet, complete Run Muse Glimmer 30B Locally with llama.cpp first, then return here to build an agent scaffold around the model.
Meta positions Muse Glimmer as an open-weight multimodal model for always-on local agents: tool use, long-horizon tasks, coding, and failure recovery under Apache 2.0. Local inference improves data control; it does not make tool access automatically safe.
1. Separate “model up” from “agent allowed”
Define three stages and do not skip ahead:
- Smoke test — text-only completion with a small context (see the run-locally tutorial).
- Tool-ready model — stable chat template, reasoning-effort setting, and reproducible decoding params.
- Agent loop — planner → tool calls → observation → recovery, with permissions.
If stage 1 fails, do not wire filesystem or shell tools.
2. Freeze a minimal agent contract
Write down:
- Goal class (for example: summarize a local folder of markdown notes; answer questions about a chart screenshot).
- Allowed tools (read-only file list, calculator, web disabled by default).
- Forbidden tools (shell, production APIs, messaging, credential stores).
- Stop conditions (max steps, max tokens, human approval gates).
Example policy sketch (application-level—not model weights):
tools.allow = [list_local_docs, read_local_doc, describe_image]
tools.deny = [shell, http_fetch, send_email, write_anywhere]
require_human_approval = [delete, external_network, secrets]
max_steps = 8
context_budget_tokens = 8192 # raise only after measuring KV-cache memory
3. Set Muse prompting defaults deliberately
Meta’s model card recommends temperature 1.0, top-p 0.95, top-k 64, and reasoning strength via system prompt (low / medium / high / xhigh). For agent loops:
- Start at medium reasoning.
- Use high/xhigh only when multi-step recovery quality improves enough to justify latency.
- Keep tool JSON schemas strict; reject unknown tool names before execution.
4. Add multimodal as a second milestone
Vision requires the perception projector (mmproj) path documented in the GGUF tutorial. Treat image understanding as a separate release gate:
- Test screenshots, charts, and scanned docs.
- Measure visual token cost and peak memory.
- Never grant write tools just because vision works.
5. Test failure recovery on purpose
Agent quality is recovery quality. Build a small eval set that forces:
- Invalid tool arguments
- Missing files
- Conflicting instructions in retrieved text (prompt-injection style)
- Step budgets exceeded
Score: did the agent stop safely, ask for clarification, or retry without duplicating side effects?
6. Optional acceleration: DFlash / speculative decoding
Meta publishes an optional DFlash drafter GGUF. Enable it only after:
- Your runtime documents Muse Glimmer + DFlash support
- You reproduce speed and task success on your hardware
Do not treat vendor speedups as portable across devices and quantizations.
7. Ship checklist
- Text-only smoke test passes at conservative context
- Tool allowlist is deny-by-default
- Human approval on irreversible actions
- Prompt-injection cases in the eval set
- Quantization quality vs full-precision spot-checked on your tasks
- Logging of tool calls without leaking secrets into central logs