Skip to main content
The six courses are executable teaching projects, not reproductions of product internals or paper benchmark scores. A successful tool call is not a successful research conclusion. The lab overview explains the framework/application boundary; the repository execution ledger preserves dated implementation and validation evidence.

​
What the installed consumers prove

Python 3.10 and 3.12 outside-checkout consumers passed. The horizontal consumer made 12 requests, 10 plan revisions and 9 distinct compaction receipts; notebook records remained after cleanup. Scripted models prove mechanisms, not intelligence. The final live matrix is tracked separately below, including non-passing attempts.

​
Framework and course defects found by the projects

  • Custom agent_factory now enters the resource-owning composition root.
  • Persistent skill revisions replace duplicated course stores; selected skill instructions are not silently truncated. Memdir deletion is explicit.
  • Artifact references are evidence, not privileged developer instructions.
  • Child workspace restoration cannot overwrite the authorized conversation.
  • Same-owner pause continuation traverses restoring and persists its final head.
  • Encoded JSON scalar inspection distinguishes escaped program text from real private paths. Session contract failures do not silently advance the loop.
  • Native preflight blocks close their tool batches, including blocked/executed siblings, without duplicate compatibility results.
  • Course fixes cover packaged history identifiers, report/search instructions and explicit child budget allocation. Those remain application policy, not defaults imposed on every agent.

​
Real Qwen experiment contract

Three tasks per project, three repetitions, plus static-plan/no-memory/no-skills controls. Hermes and Voyager perform learning before a new-process recall run. Controls start from the same post-learning store bytes, before default recall writes another episode. A failed seed is not evidence of a causal benefit. The bounded qualification configuration uses 24 steps/requests and 600 seconds per task, 10,240 output tokens, and at most four concurrent project/repetition groups. Public project defaults remain configurable at 80/80/3600. There is no total experiment request or cost quota. Provider/model configuration is explicit; TLS verification is not disabled and another model is not silently substituted. The matrix completed 99 invocations: 54 default task trials, 18 preceding learning runs and 27 controls. No whole-lab live qualification claim is made. These are small, bounded teaching experiments, not rankings or evidence that dynamic planning, memory or skills generally improve task success. One Voyager learning attempt rejected snapshot content and produced no final report; it is counted as non-passing. Controller timeouts and manual interventions in this final matrix were zero; declared Session budget stops still count as failures. All 18 groups preserve installed-file digests. Relative to the final source, the live candidate differs only in _action_runtime.py (the subsequent mixed blocked/executed compatibility-result deduplication) and the Hermes course agent (subsequent literal-search/report-shape clarification). Those changes passed permanent deterministic and installed checks; the 99-run matrix was not rerun on them. Therefore it does not qualify the final source’s complete live behavior. The repository ledger includes a counts-only aggregate and installed identity document digests. Earlier source groups and every failure remain separately in private storage; none is replaced by a best-of retry. No provider payload or sensitive path is published.

​
How to inspect and learn from a failure

Each run writes report.json, a Session identifier and a canonical trajectory in the chosen private run directory. Use the project’s inspect command, then follow model request, tool terminal, verification, snapshot and final events. Compare the independent checker with the last completed operation, not the final prose. Distinguish these observed outcomes:
  1. Framework defect: an orphan batch or wrong child input required a permanent counterexample and framework fix; prompting around it was not accepted.
  2. Transport failure: provider_connection_failed can occur after code checks pass. Without a final outcome, the end-to-end attempt still fails.
  3. Task failure: final prose with wrong numbers, missing citations, unsuccessful memory recall or unverified code is not success.
  4. Budget stop: a loop reaching its declared guard remains non-passing; no silent budget extension or hidden successful continuation is credited.
Raw trajectories may contain model output, source code and private information. Keep them outside Git. scripts/report_agent_design_lab.py emits allowlisted counts only; it does not authorize publication of the underlying data. Remaining boundaries include bounded synthetic tasks, cooperative cancellation, local Docker rather than VM isolation, no external-effect exactly-once guarantee, non-transactional multi-file Memdir forgetting and no automatic signed dependency resolution for skills. These projects are research starting points, not a claim that every possible agent application is qualified.