negotiationarena-static-eval

negotiationarena-static-eval

All 6 declared native panels are available as separately frozen evaluation leagues. Each submitted policy runs one complete selected panel per episode. Panel scores retain their native aggregation; this release defines no score across different panels. The fleet.json index records every panel, population, metric, and fixed reference roster. Qualification and coverage must be established separately for every selected panel.

NegotiationArena

Native paper_experiment_code section-one games: Trading 8 turns, BuySell 10, MultiTurnUltimatum 8 (despite its script's one_shot filename). One complete game per episode. Evaluate 20 frozen seeds for each of six game/learner-role variants against one explicitly pinned partner. This checkpoint-paired protocol replaces the historical four-provider round robin; it does not reproduce historical model scores. Both seats use native ChatGPT message construction, temperature 0.7, 400 output tokens, exact released game prompts/parsers/rules.

Primary published_payoff executes the pinned section_one_plots notebook's native summary functions. Trading uses resource deltas, BuySell uses valued native outcomes, and Ultimatum applies the exact published RED delta + 100 followed by replacing exactly 100 with 0. Raw native outcome/state remain separate in replay. No custom legality checks, parser repair, zero-score substitution, or retry until success. A native exception or missing summary retains transcript and last state and makes the comparison unavailable; all planned matches must have scored outcomes.

Certification uses two explicit offline accepted-trade fixtures and makes no provider calls. These fixtures cannot impersonate checkpoints. Negative native controls test visible failure; they are not scored task exclusions.

Source: https://github.com/vinid/NegotiationArena/tree/d35a7a3aa0d94c2d49f1d6ac13c5f931851abf12