rngbench-static-eval

rngbench-static-eval

All 2 declared native panels are available as separately frozen evaluation leagues. Each submitted policy runs one complete selected panel per episode. Panel scores retain their native aggregation; this release defines no score across different panels. The fleet.json index records every panel, population, metric, and fixed reference roster. Qualification and coverage must be established separately for every selected panel.

RNG-Bench text Matching Pairs

The complete declared panel is 8x10 and 10x10 boards, each at seeds 0,1,2: six games. Sizes, seeds and five responses per pair follow the released evaluation sweep; observation modality is native text, matching the catalog. Every board has unique paired symbols. Response caps are200/250 per game, including retries. Native single_normal prompts, CoT/action format, parser, two retries, round forfeits, environment and scoring execute unchanged. The entire accumulated dialogue is retained without a history window or truncation. Text content parts are joined with newlines for the Coworld text-message transport. Greedy512-token decoding is an explicit adaptation from model-specific upstream presets; no paper-score reproduction is claimed. No vision maze, duel, no-action ablation or oracle-memory aid is used.

Primary matched_pair_fraction equals native score/total_pairs. Response count, parse failures and invalid actions remain separate. Responses per matched pair is undefined at zero matches and omitted from per-task metrics, never replaced with zero. All six tasks remain in the primary denominator. Source seeds, board hashes, every raw action/Generated response, native parsing/environment events and complete prompts remain in replay. Native event prompt hashes point to task-local interaction indices. Hidden boards remain game-side. The client transport is replaced with Session.generate; native logging is buffered until game completion to avoid rewriting quadratic histories every turn. Native errors fail the episode rather than producing partial scores.

Source: https://github.com/InternLM/RNGBench/tree/4b33c24cdb660c3d417bb2ae90f31a1e5462b430. Attribution: Ding et al., Beyond the Current Observation (2026). Source README declares MIT but its linked LICENSE file is absent from the pinned repository. README attribution is retained; qualification images remain private.