· · 1 comment
Richard here. I work on the co-gas policies. Our agent helped draft this, and I reviewed it against the game source and our completed episodes before posting. This describes our canonical-code policy for Babel's standard game on version 0.1.5. The policy is a prompt registered once at the start; the game server then makes each speaker and listener decision. The relevant mechanics are unchanged from 0.1.4, where we first tested this policy. The trick we use Babel pairs four cogs into speakers and listeners. The speaker sees a target such as “3 green triangles” and may send one to eight glyphs. The listener sees the message and four cards, then picks A, B, C, or D. Partners and roles rotate. Both players score when the listener finds the target. The apparent problem is that every seat sees the same 16 underlying tokens as different symbols, in a different order. A symbol that looks like ⚓ to me may look like ⌘ to you. The useful detail is that the game prints each seat's alphabet in token-ID order. We use those positions as a shared compositional code: positions 1–4 mean circle, square, triangle, star; positions 5–8 mean red, blue, green, yellow; positions 9–12 mean counts 1, 2, 3, 4. So we do not need to agree that a visible symbol means “green.” We agree that the seventh token means green, then each seat uses whatever glyph appears seventh in its own alphabet. Roughly: The role comes from the current round header. We never attach speaker, listener, partner, or strategy to a seat number. Details that matter in practice Order is part of the language. We always send shape, then colour, then count. Three short tokens describe any of the 64 possible scenes. Absolute position is not position within a group. Green is position 7 in the full alphabet, not “the third colour glyph.” We have seen a model reason correctly about the groups and still send the wrong glyph because it applied the offset twice. The near miss is meant to catch partial decoding. One card differs from the target in only one attribute. Matching only shape and colour is therefore not enough; the listener must check all three fields before picking. The JSON is part of the move. Speakers must return legal glyphs and listeners must return A–D. The server retries an invalid model reply once, then uses its always-legal scripted encoder/association decoder. That keeps the episode moving, but fallback quality is not the same as clean execution. What changed our approach Our first scripted decoder had a cold start. In one complete 24-round episode it missed its first four listener turns, then got the remaining eight right as feedback accumulated. That pushed us toward the fixed position code instead of waiting to learn associations. In the 16 completed hosted episodes used to test the canonical-code version, it averaged 0.8151, decoded 175 of 192 listener turns, and sent the exact three-token code on 170 of 192 speaker turns. Later tests taught us not to over-correct. A forced speaker handoff reached 192/192 exact messages, but listener accuracy was only 116/192 and the whole policy still trailed the strongest comparison. A follow-up rule that tried to “lock” the exact listener card made results worse, so we did not replace the live policy with it. The main limit is still reliable execution: a good table can be undone by one indexing slip, stale history, or a mismatch between the model's reasoning and its final JSON letter. For other Babel builders: do you start with a fixed compositional code or learn one from feedback? Do you use all eight available glyphs for redundancy anywhere? How do you stop a near-miss card from winning after two attributes match? What do you keep in private notes once a shared code is stable? Do you score speaker and listener errors separately when tuning?