• Message bases over 2gb

    From Rob Swindell@1:103/705 to GitLab note in main/sbbs on Tue Jun 23 15:48:52 2026
    https://gitlab.synchro.net/main/sbbs/-/issues/12#note_9434

    ### Interim option: lift the per-data-file ceiling from 2 GB to 4 GB − 1

    The multi-`.sdt` design above is the real long-term fix, but a much smaller change roughly **doubles** single-data-file capacity in the meantime. The `.sdt` data offset is already stored as a `uint32_t` (`msghdr_t.offset`), so the field can address up to 4 GB − 1. The current 2 GB ceiling is **artificial** — three `(int)offset` casts in `src/smblib/smballoc.c` truncate the 64-bit `off_t` and make any offset ≥ 2³¹ look negative:

    ```
    smballoc.c smb_allocdat() : while(... (int)offset >= 0) and if ((int)offset < 0)
    smballoc.c smb_fallocdat() : if ((int)offset < 0)
    ```

    `off_t` is `int64_t` everywhere and `fseeko`/`ftello` map to the `_*i64` calls on Win32 (`xpdev/filewrap.h`), so the fix is: replace the truncating casts with an unsigned range check (`offset < 0 || offset > UINT32_MAX`), and use `fseeko`/`ftello`/`smb_fseek` instead of `fseek`/`ftell` on the `.sdt` (matters on 32-bit/Win32). Diff (3 files, +15/−9):

    ```diff
    src/smblib/smballoc.c | 20 +++++++++++++------- (the (int)offset casts + fseeko/ftello + a UINT32_MAX cap in smb_hallocdat)
    src/smblib/smbtxt.c | 2 +- (read path: smb_fseek + (off_t) cast)
    src/sbbs3/smbutil.c | 2 +- (pack data-copy: fseeko + (off_t) cast)
    ```

    ### Tested (Linux x64, gcc)

    Built `smblib` + `smbutil`/`chksmb`/`fixsmb` and ran an A/B against the unpatched HEAD build, growing a non-hyperalloc base past 2³¹ (16 MB messages):

    - **Unpatched HEAD** — adding the message whose *start offset* = 2³¹ fails:
    `!smb_addmsg returned -120: smb_allocdat invalid data offset: 2147483648` (exactly the error seen in the recent `mail`-base incident).
    - **Patched** — same add **succeeds**; base grows to `.sdt` ≈ 2.02 GiB; message at `data_offset 0x80818600` (> 2 GB) reads back correctly; `chksmb` reports **OK**; `smbutil -a pack` across the 2 GB boundary completes and `chksmb` is still **OK** afterward.

    Not yet validated: 32-bit/Win32 builds (the `fseeko`/`ftello` changes target those), and the full 0→4 GB range (the 2 GB-crossing exercises the same code path; the new ceiling is `UINT32_MAX`).

    ### Caveat

    Whatever the ceiling, hitting it must fail the message-add **cleanly** — the recent incident showed a header being committed with the `-120` error code stored as its data offset (`0xFFFFFF88`), which then broke `pack`. Tracked separately: #1170 (header committed on allocation failure) and #1171 (pack not error-safe); see also #1172 (base-lock semantics).

    Happy to open this as an MR if the interim 4 GB-1 approach is wanted ahead of the multi-file design.

    — *Authored by Claude (Claude Code), on behalf of @rswindell*
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)