1. Forum
  2. USENET
  3. comp.lang.apl

  • Nars floor of a big float

    From Rosario19@Ros@invalid.invalid to comp.lang.apl on Thu Feb 22 11:34:06 2024
    From Newsgroup: comp.lang.apl

    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    but here result 12

    (4) -> floor( log10 ( 9999999999999.9))
    (4) 12.0

    --- Synchronet 3.21d-Linux NewsLink 1.2
  • From James Cloos@cloos@jhcloos.com to comp.lang.apl on Thu Feb 22 10:44:18 2024
    From Newsgroup: comp.lang.apl

    "R" == Rosario19 <Ros@invalid.invalid> writes:

    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    I don't know nars, you are probably being hit by floating point
    precision issues.

    with arbitrary precision, the log10 of 9999999999999.9 is:

    +12.999999999999995657055180967460008764615648097801793308999084...

    Using ieee 64bit floats, you are likely to get 12.999999999999995,
    but with 32bit floats you'll get 13.0.

    Even with 64bit floats, rounding issues along the way could lead to a
    result of 13 rather than 12.99999999...

    Hense the difference in floor()'s output.

    -JimC
    --
    James Cloos <cloos@jhcloos.com>
    OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc
    --- Synchronet 3.21d-Linux NewsLink 1.2
  • From Rosario19@Ros@invalid.invalid to comp.lang.apl on Thu Feb 22 22:41:34 2024
    From Newsgroup: comp.lang.apl

    On Thu, 22 Feb 2024 10:44:18 -0500, James Cloos wrote:

    "R" == Rosario19 <Ros@invalid.invalid> writes:

    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    I don't know nars, you are probably being hit by floating point
    precision issues.

    with arbitrary precision, the log10 of 9999999999999.9 is:

    +12.999999999999995657055180967460008764615648097801793308999084...

    Using ieee 64bit floats, you are likely to get 12.999999999999995,
    but with 32bit floats you'll get 13.0.

    Even with 64bit floats, rounding issues along the way could lead to a
    result of 13 rather than 12.99999999...

    Hense the difference in floor()'s output.

    -JimC

    ok pheraps you are right for 64 and 32 bit numbers, but
    9999999999999.9v with fpc of 128 it should mean it is one float of 128
    bits
    --- Synchronet 3.21d-Linux NewsLink 1.2
  • From Bob Smith@bsmith@sudleydeplacespam.com to comp.lang.apl on Thu Feb 22 19:41:28 2024
    From Newsgroup: comp.lang.apl

    On 2/22/2024 5:34 AM, Rosario19 wrote:
    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    but here result 12

    (4) -> floor( log10 ( 9999999999999.9))
    (4) 12.0


    The answer depends upon your setting for rAoCT. If you are using the
    default value of 3E->15, then yes the floor of that multi-precision
    number is 13. That's because the algorithm first looks for an integer
    within Comparison Tolerance of the number. If it finds one, then that's
    the answer, as in

    araE10 rif 9999999999999.9v
    a+a|urAoCT
    13.0000000000000347
    --
    _________________________________________
    Bob Smith -- bsmith@sudleydeplacespam.com
    http://www.sudleyplace.com - http://www.nars2000.org

    To reply to me directly, delete "despam".

    --- Synchronet 3.21d-Linux NewsLink 1.2