Research / Evaluation and verification
CIS: A Deterministic, Offline-by-Default Compounding Intelligence Sandbox with Verifier-Gated Self-Improvement
Presents CIS, an offline-by-default, determinism-first sandbox for verifier-grounded self-improvement with immutable run records and promotion/rollback gating; in a local run, core_logic PASS@1 on visible tasks rises from 0.5000 to 0.8077.
- Published
- 13 January 2026
- Author
- Ali Uyar
Abstract
Claims of “model self-improvement” are frequently undermined by non-deterministic evaluation, informal ground truth, porous safety boundaries (e.g., network leakage, tool escapes), or mutable logs that prevent forensic auditing. We present CIS (Compounding Intelligence Sandbox), an end-to-end system for verifier-grounded capability growth on verifiable tasks, designed to be offline-by-default, determinism-first, and auditable via immutable run records.
CIS integrates:
- A canonical CLI surface and a workspace artifact system with schema-validated, immutable run records.
- Suite snapshots with a hidden subset protected by sealed evaluator mode and explicit unlock; default evaluation excludes hidden tasks.
- A sandbox layer for tool execution with default offline enforcement and incident recording.
- Frontier governance and difficulty indexing (DI) calibration with DI50 tracking.
- A self-improvement training loop combining verified distillation and reward learning from verifiers (RLVR) with PEFT adapters (LoRA/LoRA+/DoRA) and promotion/rollback gating.
We report early local results in a real workspace using qwen2_5_coder_7b_instruct with a DoRA adapter (champion: adapter-000075): verifier PASS@1 on visible tasks improves from iter-000021 to iter-000149 across multiple suites (e.g., core_logic from 0.5000 to 0.8077), while rollbacks reject regressions (e.g., iter-000150, iter-000151). We also document and validate a deterministic, training-only exploration rule for a SAT/UNSAT family (FAMILY-004) that addresses label collapse (policy outputs only sat) by restoring verified learning signal without altering evaluation behavior.
Why it matters
Self-improvement claims are hard to audit when evaluation is non-deterministic or logs are mutable. In CIS each promotion or rollback is a gated decision over deterministic verifiers, kept in an immutable run record.
Cite
@misc{uyar2026cis,
title = {CIS: A Deterministic, Offline-by-Default Compounding Intelligence Sandbox with Verifier-Gated Self-Improvement},
author = {Uyar, Ali},
year = {2026},
doi = {10.5281/zenodo.18234775},
publisher = {Zenodo},
url = {https://doi.org/10.5281/zenodo.18234775}
}