Strategies, Not Model Ifs

Thesis. When two models share one recursive agent loop, a single default policy will punish at least one of them. The fix is not if "glm" in model: inside the engine. The fix is named, data-driven execution strategies (how the loop runs), optionally bound from profiles (who runs), and always overridable per task.

This article records the architecture that came out of RLM dogfooding on 2026-07-17: live GLM multi-cell burn, DeepSeek-style disciplined convergence, broken HITL extends when tool budgets were not on the ledger, and the decision to make approaches first-class YAML.

Context: one loop, two behaviors#

Operators asked: maybe different approaches are needed for different models — do we need that flexibility?

Yes — with a constraint: flexibility of approaches, not special cases hardwired to vendor strings.

Empirical companions:

• Small cells A/B — prompting changes cell shape; horizon still decides completion.
• Token / history note — history caps belong on the same strategy object as cell/tool limits.

Separation of axes#

Resolution (highest wins per key):

task.metadata[key]
  >  task.metadata.rlm_config_id  (named preset)
  >  profile.rlm_config_id        (convenience default)
  >  HARNESS_RLM_CONFIG / "default"

Figure: rank only — task keys always beat profile-bound presets.

Why not model ifs#

1. New models appear weekly. Engine forks rot.
2. Same model, different jobs. A short probe wants multi-block batching; a long review wants hybrid discipline (A/B note).
3. Tests and evals need pure strategy diffs without swapping weights.
4. Operators can add presets without shipping Python.

Named presets (shipped shape)#

From rlm_config.yaml (abridged intent):

Figure: max_cells/turn of 0 means no hard cap (null in YAML) for default.

Illustrative YAML:

# rlm_config.yaml
configs:
  default:
    max_tool_calls: 200
    max_model_calls: 40
    max_code_blocks_per_turn: null
    prompt_style: multi_block
    history_keep_recent_turns: 6
    history_max_total_chars: 80000

  disciplined:
    max_tool_calls: 200
    max_code_blocks_per_turn: 1
    prompt_style: single_block
    history_max_total_chars: 60000

  exploratory:
    max_tool_calls: 1000
    max_code_blocks_per_turn: 12
    hybrid_switch_after_turns: 3
    late_max_code_blocks_per_turn: 4
    prompt_style: multi_block_budgeted
    system_prompt_addon: >-
      Prefer batched reads; write findings early under /workspace/reviews/;
      after orientation, fewer smaller cells and FINAL soon.

# profiles.yaml (bindings)
deepseek-v4-flash:
  provider: openrouter
  model: deepseek/deepseek-v4-flash
  rlm_config_id: disciplined

glm-5.2:
  provider: openrouter
  model: z-ai/glm-5.2
  rlm_config_id: exploratory

Task authors still win:

agent:
  profile: glm-5.2
metadata:
  rlm_config_id: disciplined   # force different approach
  max_tool_calls: 500          # one-off key override

Strategy knobs that earned their place#

Ledger honesty (ops requirement)#

A recurring failure mode: engine stops on max_tool_calls, HITL “extend budget” raises root steps or unset ledger rows, warm-continue changes nothing useful.

Strategy work only becomes operationally real when:

1. Limits from the resolved strategy are seeded on the budget ledger (not None).
2. Actual tool/model counts are settled each invocation.
3. Warm-continue grants remaining against the raised ceiling (same pattern as iteration grants).
4. Gates surface exhaustion_reason so operators know which knob tripped.

Without (1–4), “exploratory with 1000 tool calls” is a paper policy.

Worked example#

Scenario A — short GLM probe. Binding exploratory may be wrong: high cell freedom + short steps can still finish (see multi-block A arm), but if you want comparable cell discipline for measurement, set metadata.rlm_config_id: disciplined or prompt-only single_block as in the A/B — knowing completion risk rises.

Scenario B — long GLM review. Profile default exploratory gives tool headroom + hybrid late nudge + history caps. Operator sees exhaustion_reason=max_tool_calls → extend tool budget, not steps alone.

Scenario C — DeepSeek implement. Profile disciplined keeps one-cell cadence; task can still raise max_iterations without rewriting engine code.

Counterfactuals#

1. If one global default were enough, GLM multi-cell burn and DeepSeek one-cell success would both be well-served by max_tool_calls≈100 and multi-block prompts. Dogfood showed they are not.

2. Alternative design: auto-tune from telemetry. Out of scope for this change set; strategies remain operator-declared. Auto-tune would still need a strategy representation like this YAML.

3. Alternative: per-provider adapter forks. Multiplies code paths; fails the “same harness, many models” goal.

4. Falsifier for the architecture: a required behavior that cannot be expressed as strategy keys without reading model in the engine. None appeared in this design pass; cell caps, prompts, history, and budgets covered the dogfood gaps.

Limitations#

• Preset numbers (1000 tools, 12 cells, hybrid after 3) are starting points from one dogfood week, not optimized frontiers.
• Profile → strategy binding is a convenience; mis-bound profiles can silently change cost envelopes — document defaults in run config snapshots.
• Hard cell caps can surprise models that planned later fences; deferred observation text must stay explicit (turn-discipline spec extension).
• This article does not claim the exploratory hybrid fixed deliverable writes; compaction + hybrid improved token/cell behavior while write thrash remained (token note).

Takeaways#

1. Split who (profile) from how (rlm_config strategy).
2. Encode approaches in YAML; resolve with a clear override stack; never branch the engine on model id strings.
3. Put every enforced limit on the ledger or HITL will lie.
4. Include history and cell policy in the same strategy object — they interact with tool budgets.
5. Use task metadata when the job’s horizon disagrees with the model’s default personality (small cells).

Regenerate figures#

cd $HOME/work/authoring
uv run --project publish/framework python \
  publish/framework/scripts/generate_rlm_article_figures.py