NMMO: three recurrent brains, one action

Richard here. I work on the co-gas agents. Our agent helped draft this from the NMMO source, the active controller, and completed episode evidence; I reviewed it before posting.

The active co-gas NMMO entries use the same neural consensus controller on version 0.1.4, variant default. Each player controls one hero through a 5,000-tick life-and-progression game. The score rewards balance: for each life, the useful level is min(combat level, profession level). Completed lives are banked, the current life counts only if the hero is alive at the end, and the total is divided by deaths + 1.

In compact form:

score = (banked min(combat, profession) + current-life min(combat, profession))
        / (deaths + 1)

If the hero is dead at the terminal tick, the current-life term is omitted. This makes a lopsided build and a late reckless death expensive: combat 12 with profession 3 contributes only 3 for that life.

How consensus chooses an action

We run three independent copies of the same pretrained MMONet. Each copy has 4,430,976 weights, four recurrent MinGRU layers, and a different random seed: 1, 17, and 29. On every tick, all three receive the same 1,707-byte hero observation and each samples one of 26 discrete actions.

The action rule is deliberately small:

votes = [brain_seed_1(obs), brain_seed_17(obs), brain_seed_29(obs)]
if any action has 2 or 3 votes:
    take that majority action
else:
    take seed 1's action
if the game marks a life reset:
    reset all three recurrent states before the next forward pass

That last reset is essential. The networks carry memory across ticks; keeping the hidden state from a dead life would mix two unrelated trajectories. We use the protocol's actual per-hero reset flag rather than guessing from health or position.

The three-way fallback also matters. We tested selecting the lowest numbered action whenever all three disagreed. On a complete seed-23 episode it averaged 3.75 against 5.75 for the original controller. Small action ids have no general claim to safety, so the active version preserves the canonical seed-1 vote instead.

Technical gotchas

The neural controller uses the observation bytes it was trained on. NMMO 0.1.4 added an optional clean entity view, but this policy does not request it; opting in would change the network input distribution. Our separate scripted experiments use the clean view, which is why their entity reasoning should not be attributed to the active neural controller.

Sampling is deterministic only when the seed and call order stay fixed. The WASM module uses its own libc random stream. A native build linked against a different libc can sample different actions even when logits match exactly, so we compare full WASM traces rather than assuming seed equality implies cross-runtime action equality.

Finally, consensus is not a confidence score. A 2–1 vote can reflect a narrow sampling difference, while a three-way split tells us only that no action repeated. Hand-written overrides can look semantically sensible and still fight the learned state. In held-out testing, broad model-endorsed tie overrides scored 6.333 versus 7.667 for consensus; narrower variants also regressed when they admitted minority retreats, new combat, or interface movement.

What completed games support

We moved the lower owned entry from a fully scripted controller to this same consensus family after eight completed, seat-rotated 0.1.4 episodes. Consensus averaged 6.698 across 16 appearances versus 5.634 for the scripted comparison, an 18.9% lift. The other active consensus entry averaged 6.960, while the strongest public comparison averaged 7.537. All eight replays matched their results, all 64 scores matched the executable score equation, and transport no-ops were zero.

A later horizon-aware scripted controller improved its predecessor, but in eight completed episodes it averaged 5.593 versus 7.073 for the neural comparison. It also reduced combat resets while increasing stagnation resets. We kept the consensus family.

The next useful experiment is a logged, narrow three-way-tie resolver. It should first measure which observation contexts produce splits, then admit only a source-backed action class whose exact vote and subsequent transition can be audited. The default must remain seed 1 until complete episodes show a real gain.

Which observation features best explain a three-way split between identical networks with different seeds? How do you balance combat and profession progress without sacrificing too many lives? What reset signal would you audit first when recurrent behavior suddenly changes after death? Which action classes, if any, are safe enough to resolve only true three-way disagreements?