• Re: ZFS ARC tuning (was: "failed to reclaim memory" with much free physmem)

    From Rick Macklem@rick.macklem@gmail.com to muc.lists.freebsd.stable on Sat Sep 13 17:16:20 2025
    From Newsgroup: muc.lists.freebsd.stable

    On Fri, Sep 12, 2025 at 8:02rC>PM Graham Perrin <grahamperrin@gmail.com> wrote:

    On 13/09/2025 03:25, Rick Macklem wrote:

    rCa
    - setting vfs.zfs.arc_max to a much smaller value than 99.9% and rCa


    vfs.zfs.arc_max is recognised, but legacy.
    At least I can see this is used to set zfs_arc_max. See below..
    More modern:

    vfs.zfs.arc.max
    I'm blind. I can't even figure out where/what this sets.
    (If it also sets zfs_arc_max, I cannot find where in the code it does so?
    Or does it set nothing at all?)


    Also, might one of the following be a better alternative?

    vfs.zfs.arc_free_target
    Lets take this one as an example:
    It appears to set the value of zfs_arc_free_target and then
    this valriable gets used in a function called arc_available_target().
    Now, here's the first lines of that function. (If you can guess what
    this is supposed to be doing, you are better than I).
    int64_t
    arc_available_memory(void)
    {
    int64_t lowest = INT64_MAX;
    int64_t n __unused;
    /*
    * Cooperate with pagedaemon when it's time for it to scan
    * and reclaim some pages.
    */
    n = PAGESIZE * ((int64_t)freemem - zfs_arc_free_target);
    if (n < lowest) {
    lowest = n;
    }
    A few observations...
    - int64_t --> so they can be negative
    - n __unused, but then it is
    - compares n < lowest when lowest is set to the largest positive
    value possible. (Wow, that seems useful??)
    In general, it just sets lowest to "PAGESIZE * ((int64_t)freemem - zfs_arc_free_target)"
    without wondering what happens if freemem < zfs_arc_free_target, so n
    is negative?
    This is followed by some #ifdef'd code that I no idea if it is executed
    or not.
    So, at least I think I know what vfs.zfs.arc_max does and it seems to me
    that it should be less than 99.9% of the machine's ram?
    rick
    ps: Like I said, there is not a lot in sys/contrib/openzfs/module/os/freebsd/zfs/arc_os.c
    and what there is looks kinda sketchy to me.

    vfs.zfs.arc.sys_free


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Garrett Wollman@wollman@bimajority.org to muc.lists.freebsd.stable on Sat Sep 13 21:16:19 2025
    From Newsgroup: muc.lists.freebsd.stable

    <<On Sat, 13 Sep 2025 04:02:01 +0100, Graham Perrin <grahamperrin@gmail.com> said:

    Also, might one of the following be a better alternative?

    vfs.zfs.arc_free_target

    vfs.zfs.arc.sys_free

    The former *sounds* like it might be a reasonable lever, although I
    still feel like there's some sort of accounting issue here. I note on
    my systems the latter is zero, but the former has a positive value
    (which seems to be related to memory size).

    -GAWollman



    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Rick Macklem@rick.macklem@gmail.com to muc.lists.freebsd.stable on Tue Oct 21 20:43:09 2025
    From Newsgroup: muc.lists.freebsd.stable

    On Sat, Sep 13, 2025 at 6:17rC>PM Garrett Wollman <wollman@bimajority.org> wrote:

    <<On Sat, 13 Sep 2025 04:02:01 +0100, Graham Perrin <grahamperrin@gmail.com> said:

    Also, might one of the following be a better alternative?

    vfs.zfs.arc_free_target
    I still won't claim to understand the arc sizing stuff, but it looks like
    this one triggers reaping when it is greater than freemem. (It appears
    to be in pages, not bytes.)
    Once arc_available_memory() calculates "n" via
    PAGESIZE * ((int64_t)freemem - zfs_arc_free_target) and returns it
    as a negative value, arc_reap_cb_check() looks like it decides this
    is the time to reap, so I think the code calls arc_reap_cb(), which
    calls arc_kmem_reap_soon(), which calls kmem_cache_reap_now(),
    which becomes a call to FreeBSD's uma_zone_reclaim(.., UMA_RECLAIM_DRAIN).
    So, it looks like it will try fairly hard to maintain at least vfs.zfs.arc_free_target
    pages in freemem.

    vfs.zfs.arc.sys_free
    This does seem to be useful too, but I haven't figured out what it does. Something to do with eviction, but I haven't figured it out yet.
    I would like to here if fiddling with these helps w.r.t. Garrett's problem, rick

    The former *sounds* like it might be a reasonable lever, although I
    still feel like there's some sort of accounting issue here. I note on
    my systems the latter is zero, but the former has a positive value
    (which seems to be related to memory size).

    -GAWollman


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21a-Linux NewsLink 1.2