Evaluation Methodology
XiangqiBench is designed to eliminate positional bias, opening memorization, and first-move advantage, providing a reproducible and rigorous benchmark for discrete spatial reasoning in Chinese Chess.
Utilizing 100 balanced opening and middlegame transition branches. Every pair of models plays both sides (Red and Black) from identical starting positions to eliminate first-mover advantage and opening bias.
Real-time adjudication powered by a stateless pure rules engine. Enforcing palace boundaries, obstacle blocking (horse legs & elephant eyes), flying generals, and perpetual chase/check disqualifications.
Jointly fitting round-robin tournament results using the standard Bradley-Terry probabilistic model with 95% bootstrap confidence intervals for robust strength quantification.