• Bug#1063673: ITP: llama.cpp -- Inference of Meta's LLaMA model (and oth

    From Cordell Bloor@21:1/5 to Mo Zhou on Sat Dec 28 03:40:01 2024
    Hi Mo,

    On 2024-12-22 12:42, Mo Zhou wrote:
    Apart from source-based alternative distribution for Debian, "bumping
    amd64
    baseline for selected packages" is another project I proposed long
    time ago:

    https://github.com/SIMDebian/SIMDebian

    Software like Eigen3, TensorFlow can heavily benefit from the baseline
    bump.
    At that time PyTorch did not have dispatch, but now it has already.

    You probably already know this, but Ubuntu is exploring the possibility
    of an x86-64-v3 variant [1][2]. That would include AVX2, FMA, and F16C instructions (among others), essentially setting Intel Haswell (2013)
    and AMD Excavator (2011) as the baseline.

    Sincerely,
    Cory Bloor

    [1]: https://ubuntu.com/blog/optimising-ubuntu-performance-on-amd64-architecture [2]: https://ubuntu.com/blog/profile-workloads-on-x86-64-v3-to-enable-future-performance-gains

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)