• Re: Interview with Power's chief designer

    From MitchAlsup1@21:1/5 to Thomas Koenig on Sun Dec 29 01:58:52 2024
    On Fri, 27 Dec 2024 13:29:22 +0000, Thomas Koenig wrote:

    Not sure how many of you read Chips and Cheese, but in case you're interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/

    There is a lot of talk on OMI (he really doesn't like DDR, and gives
    reasons, especially the amount of memory and reliability), plus some
    detail on POWER11, which apparently will be a microarchitectural
    evolution, but no new ISA parts, and the philosophy behind the
    chiplet design they are about to do for the next generation after
    that.

    He makes a compelling point that DDR is using too many pins and
    still does not provide the desired BW available for that number
    pf pins. And that a SEREDS interface to DRAMs provide easier to
    achieve signaling and larger memories at the same time--similar
    to what CXL:memory is attempting.

    Apparently, nobody knows what its name will be, but it might
    be something like "Power 11 plus one".

    Let me take a guess::
    {I have to state that I have heard and read nothing}

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael S@21:1/5 to Thomas Koenig on Sun Dec 29 14:29:19 2024
    On Fri, 27 Dec 2024 13:29:22 -0000 (UTC)
    Thomas Koenig <tkoenig@netcologne.de> wrote:

    Not sure how many of you read Chips and Cheese, but in case you're interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/


    Why no editing?
    Why do I have to see 100 repetitions of "you know" ?
    And, BTW, Danish blue is a poor substitute for Roquefort. Even Pecorino
    blue is better.

    There is a lot of talk on OMI (he really doesn't like DDR, and gives
    reasons, especially the amount of memory and reliability),

    I didn't understand this part. My understanding of Power10 and supposed
    of Power11 memory architecture is that ideologically it is the same as
    Intel'a Beckton and Westmere-EX of early 2010s. Of course, everything
    is beefier than it was then - there are more links and each link runs
    4x to 5x faster than on Beckton. But basic principle is the same, what
    Intel called Buffers-on-Board (BoB). I.e. fast link runs from CPU
    through either PCB or possibly cable to distance of 1-3 cm from where
    they place memory. At this point the is a buffer chip that translates
    fast protocol to several industry-standard DDR buses. In Intel's case
    there were 2 buses. In POWER10 cases probably 3 or 4. Buffers used by
    Power10 supposedly supported both DDR4 and DDR5. I would expect that
    DDR5 is the only remaining option for POWER11.
    But the point is that behind the buffer chips they have the same DIMMs
    as everybody else with the same poor edge connectors that cause the same
    signal integrity and reliability problems.


    plus some
    detail on POWER11, which apparently will be a microarchitectural
    evolution, but no new ISA parts, and the philosophy behind the
    chiplet design they are about to do for the next generation after
    that.


    I read it differently. Like POWER11 is already made out of one IO
    chiplet and several compute chiplets.


    Apparently, nobody knows what its name will be, but it might
    be something like "Power 11 plus one".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael S@21:1/5 to mitchalsup@aol.com on Sun Dec 29 14:48:46 2024
    On Sun, 29 Dec 2024 01:58:52 +0000
    mitchalsup@aol.com (MitchAlsup1) wrote:

    On Fri, 27 Dec 2024 13:29:22 +0000, Thomas Koenig wrote:

    Not sure how many of you read Chips and Cheese, but in case you're interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/

    There is a lot of talk on OMI (he really doesn't like DDR, and gives reasons, especially the amount of memory and reliability), plus some
    detail on POWER11, which apparently will be a microarchitectural
    evolution, but no new ISA parts, and the philosophy behind the
    chiplet design they are about to do for the next generation after
    that.

    He makes a compelling point that DDR is using too many pins and
    still does not provide the desired BW available for that number
    pf pins. And that a SEREDS interface to DRAMs provide easier to
    achieve signaling and larger memories at the same time--similar
    to what CXL:memory is attempting.


    Unlike CXL:memory, OMI is not layered on top PCIe gen5 phy.
    They claim the same bandwidth with lower latency and lower power.

    I don't know where to looks for details of physical layer of OMI, but
    would suspect that it is more like HyperTransport or Intel QPI/UPI than
    like PCIe. I.e. timing, including phase, is not recovered independently
    from every data lane, but provided as a dedicated signal. Likely one
    timing signal per group of 4 or 5 data signals.
    All above are my speculations not based on knowledge.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Koenig@21:1/5 to Michael S on Mon Dec 30 14:27:17 2024
    Michael S <already5chosen@yahoo.com> schrieb:
    On Sun, 29 Dec 2024 01:58:52 +0000
    mitchalsup@aol.com (MitchAlsup1) wrote:

    On Fri, 27 Dec 2024 13:29:22 +0000, Thomas Koenig wrote:

    Not sure how many of you read Chips and Cheese, but in case you're
    interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/

    There is a lot of talk on OMI (he really doesn't like DDR, and gives
    reasons, especially the amount of memory and reliability), plus some
    detail on POWER11, which apparently will be a microarchitectural
    evolution, but no new ISA parts, and the philosophy behind the
    chiplet design they are about to do for the next generation after
    that.

    He makes a compelling point that DDR is using too many pins and
    still does not provide the desired BW available for that number
    pf pins. And that a SEREDS interface to DRAMs provide easier to
    achieve signaling and larger memories at the same time--similar
    to what CXL:memory is attempting.


    Unlike CXL:memory, OMI is not layered on top PCIe gen5 phy.
    They claim the same bandwidth with lower latency and lower power.

    Interestingly enogh (I only found this when looking)
    OMI appears to have been absorbed by CXL.

    You can find the OpenCAPI specs, which OMI is based on, at https://computeexpresslink.org/resource/opencapi-specification-archive/

    I don't know where to looks for details of physical layer of OMI, but
    would suspect that it is more like HyperTransport or Intel QPI/UPI than
    like PCIe. I.e. timing, including phase, is not recovered independently
    from every data lane, but provided as a dedicated signal. Likely one
    timing signal per group of 4 or 5 data signals.
    All above are my speculations not based on knowledge.

    If you're so inclined, you can wade through the specs, but it's
    likely a non-trivial amount of work...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Koenig@21:1/5 to Michael S on Mon Dec 30 19:21:01 2024
    Michael S <already5chosen@yahoo.com> schrieb:
    On Fri, 27 Dec 2024 13:29:22 -0000 (UTC)
    Thomas Koenig <tkoenig@netcologne.de> wrote:

    Not sure how many of you read Chips and Cheese, but in case you're
    interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/


    Why no editing?
    Why do I have to see 100 repetitions of "you know" ?

    It probably took you longer to write that complaint than to read
    over all of these :-)

    And, BTW, Danish blue is a poor substitute for Roquefort. Even Pecorino
    blue is better.

    There is a lot of talk on OMI (he really doesn't like DDR, and gives
    reasons, especially the amount of memory and reliability),

    I didn't understand this part. My understanding of Power10 and supposed
    of Power11 memory architecture is that ideologically it is the same as Intel'a Beckton and Westmere-EX of early 2010s.

    BTW, some more searching turned up https://opencapi.github.io/omi-doc/ .

    Of course, everything
    is beefier than it was then - there are more links and each link runs
    4x to 5x faster than on Beckton. But basic principle is the same, what
    Intel called Buffers-on-Board (BoB). I.e. fast link runs from CPU
    through either PCB or possibly cable to distance of 1-3 cm from where
    they place memory. At this point the is a buffer chip that translates
    fast protocol to several industry-standard DDR buses. In Intel's case
    there were 2 buses. In POWER10 cases probably 3 or 4.

    Power10 has 16 OMI links per SCM (which is the smallest unit, containing
    10, 12 or 15 cores), driven by eight on-chip memory controllers.
    IBM claims a maximum bandwidth of 409 GB/s per SCM.

    You need fewer pins and lower power on your main CPU with that
    approach, and can get thus access more DIMMs in parallel.

    Buffers used by
    Power10 supposedly supported both DDR4 and DDR5. I would expect that
    DDR5 is the only remaining option for POWER11.

    The CPU doesn't care what is on the other side, that is the beauty
    of it.

    But the point is that behind the buffer chips they have the same DIMMs
    as everybody else with the same poor edge connectors that cause the same signal integrity and reliability problems.

    One important point appears to be that the DDR protocol needs
    to be run over shorter wires. Also, it seems they ECC checks,
    error correction and even isolation of flaky units for high
    reliability.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Koenig@21:1/5 to All on Fri Dec 27 13:29:22 2024
    Not sure how many of you read Chips and Cheese, but in case you're
    interested: Here is an inteview with IBM Power's chief designer,
    Bill Starke:

    https://old.chipsandcheese.com/2024/12/26/ibm-power-whats-next/

    There is a lot of talk on OMI (he really doesn't like DDR, and gives
    reasons, especially the amount of memory and reliability), plus some
    detail on POWER11, which apparently will be a microarchitectural
    evolution, but no new ISA parts, and the philosophy behind the
    chiplet design they are about to do for the next generation after
    that.

    Apparently, nobody knows what its name will be, but it might
    be something like "Power 11 plus one".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)