• Systematic errors in gamblegammon bots

    From MK@murat@compuplus.net to rec.games.backgammon on Sat Jan 6 17:57:10 2024
    From Newsgroup: rec.games.backgammon

    In another thread "Surprised by XG", I had written:

    "I have always argued that cube skill formulas to
    "extrapolate cubeful equities and match tables to
    "extrapolate "matchful" equities injected systematic
    "errors of unknown magnitute, each by itself or both
    "in combination. See:

    https://groups.google.com/g/rec.games.backgammon/c/GbjfpaiCnUc/m/Hs4LnaF2AQAJ

    Reading about plies in rollouts, I came across only
    one reference to systematic errors which made me
    realise that not only the concept is misunderstood
    here in general but my arguments may have been
    even more misunderstood or not understood at all
    because of my not having expressed myself clearly
    enough.

    Noo-BG manual v1.00.0, section 8.2 says:

    "1. Truncation: instead of rolling out all the way to
    "the end of the game, it can stop and pretend its
    "evaluation after a few plies is perfect. This may
    "obviously introduce some amount of systematic
    "error, but in practice this may not matter because:

    "3. different positions will be reached in different
    "trials, so the correlation between errors in each
    "trial weakens and the errors cancel out to some
    "extent;

    The errors introduced by truncation are "random",
    not "systematic". Random errors may "cancel out"
    but systematic errors don't. Just because an action,
    (i.e. truncation), performed is "consistent", (which is
    a quality of systematic error), it doesn't necessarily
    produce consistent errors. In fact, the manual does
    explicitly acknowledges this fact by saying above:
    "different positions will be reached in different trials",
    (i.e. the errors introduced will be radomly different).

    About extrapolated cubeful equities, by systematic
    errors I mean, for example, that the formula fails to
    consider "how much play is still left in the game". I
    had talked about this in the past, when explaining
    how I was making my cube decisions not based on
    fancifully calculated equities and winning chances.
    My next mutant cube skill experiment I had recently
    mentioned will be based on this concept, (i.e. the
    various stages of the game with different amounts
    of play still left in the game).

    About extrapolated "matchful" equities, (I coined it
    to mean "match play" and sound like "cubeful"), by
    systematic errors I mean, for example, that match
    equity tables are "circular", that is they are created
    based on equities from games played a certain way,
    such as like the bots, then future matchful equities
    are calculated using MET's, and then MET's are
    further refined using those equities, and so on...

    It's simple to know that these systemetic errors do
    exists. What's difficult to know if the magnitude of
    these errors. Unfortunately, since the past 30+ years
    that equity extrapolating formulas and tables have
    been used and talked about, these errors have never
    been acknowledged, nor has there been any efforts
    to quantify their magnitudes, neither has there been
    any discussions about mitigating them. :(

    Ignorance is bliss. Fallacy is blisser. Dogma is blissest.

    MK
    --- Synchronet 3.21b-Linux NewsLink 1.2