• Bug#1105051: imagemagick breaks xplanet autopkgtest: test_compare_image

    From Alexander Reichle-Schmehl@21:1/5 to All on Fri May 16 14:10:01 2025
    Hi!

    Thanks for checking and all the updates.

    There would also be option c) dropping the compare tests.
    I'm honestly not sure if the compare tests are worth the effort.

    And xplanet would still have the file type tests.


    Best regards,
    Alexander

    Jochen Sprickerhof <jspricke@debian.org> hat am 16.05.2025 13:31 CEST geschrieben:


    Hi,

    * Paul Gevers <elbrus@debian.org> [2025-05-10 14:24]:
    With a recent upload of imagemagick the autopkgtest of xplanet fails
    in testing when that autopkgtest is run with the binary packages of >imagemagick from unstable. It passes when run with only packages from >testing. In tabular form:

    pass fail
    imagemagick from testing 8:7.1.1.47+dfsg1-1
    xplanet from testing 1.3.1-3
    all others from testing from testing

    I copied some of the output at the bottom of this report.

    Currently this regression is blocking the migration of imagemagick to >testing [1]. Due to the nature of this issue, I filed this bug report >against both packages. Can you please investigate the situation and >reassign the bug to the right package?

    There are multiple problems:

    1. imagemagick changed the output of compare in 7.1.1-44 (d85a7583f)
    from some number <= 1 with 1 meaning the images are the same two two
    numbers where the first is the distortion and the second one is the
    old percentage in parenthesis. This breaks the test code in
    https://sources.debian.org/src/xplanet/1.3.1-3/debian/tests/compare-create-image/#L32
    I assume that the new format will be kept so xplanet should adopt the
    logic. Maybe something like:

    result_jpg="$( compare -metric ncc $AUTOPKGTEST_TMP/expected/$filename.jpeg $AUTOPKGTEST_TMP/$filename.jpeg $AUTOPKGTEST_TMP/diff.jpg 2>&1 | sed 's/.*(\(.*\))/\1/' )"

    2. imagemagick in stable returns values below 0 for similar but
    different images. I did not check if this was different in older
    versions but I would expect so, given that the autopkgtest code
    checks the value for >0.99. This has been corrected by multiple
    patches in imagemagick git main.

    3. imagemagick in unstable has a bug resulting in the same picture
    getting a percentage of 0 and different images a value of > 1. This
    has also been corrected in imagemagick git main (912b6a838).

    4. The results of earth_markers.* in the compare-create-image
    autopkgtest improved (not sure due to what) so the files in
    debian/tests/expected/earth_markers.* need to be replaced.

    I think we have to accept that the output of compare is not stable and
    adopt xplanet. I see two options what to do:

    a. Adopt the xplanet to cope with any output of compare. This could be
    either by just using the return value, hoping that the output on
    different architectures is stable nowadays (I did not test this) or
    by adding logic that only compares the number in case the return
    value is not 0 (perfect match) and then try to interpret the
    number(s).

    b. Apply the imagemagick patches from git main and use the call from 1.
    above.

    Cheers Jochen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jochen Sprickerhof@21:1/5 to All on Fri May 16 13:40:01 2025
    Hi,

    * Paul Gevers <elbrus@debian.org> [2025-05-10 14:24]:
    With a recent upload of imagemagick the autopkgtest of xplanet fails
    in testing when that autopkgtest is run with the binary packages of >imagemagick from unstable. It passes when run with only packages from >testing. In tabular form:

    pass fail
    imagemagick from testing 8:7.1.1.47+dfsg1-1
    xplanet from testing 1.3.1-3
    all others from testing from testing

    I copied some of the output at the bottom of this report.

    Currently this regression is blocking the migration of imagemagick to >testing [1]. Due to the nature of this issue, I filed this bug report >against both packages. Can you please investigate the situation and
    reassign the bug to the right package?

    There are multiple problems:

    1. imagemagick changed the output of compare in 7.1.1-44 (d85a7583f)
    from some number <= 1 with 1 meaning the images are the same two two
    numbers where the first is the distortion and the second one is the
    old percentage in parenthesis. This breaks the test code in
    https://sources.debian.org/src/xplanet/1.3.1-3/debian/tests/compare-create-image/#L32
    I assume that the new format will be kept so xplanet should adopt the
    logic. Maybe something like:

    result_jpg="$( compare -metric ncc $AUTOPKGTEST_TMP/expected/$filename.jpeg $AUTOPKGTEST_TMP/$filename.jpeg $AUTOPKGTEST_TMP/diff.jpg 2>&1 | sed 's/.*(\(.*\))/\1/' )"

    2. imagemagick in stable returns values below 0 for similar but
    different images. I did not check if this was different in older
    versions but I would expect so, given that the autopkgtest code
    checks the value for >0.99. This has been corrected by multiple
    patches in imagemagick git main.

    3. imagemagick in unstable has a bug resulting in the same picture
    getting a percentage of 0 and different images a value of > 1. This
    has also been corrected in imagemagick git main (912b6a838).

    4. The results of earth_markers.* in the compare-create-image
    autopkgtest improved (not sure due to what) so the files in
    debian/tests/expected/earth_markers.* need to be replaced.

    I think we have to accept that the output of compare is not stable and
    adopt xplanet. I see two options what to do:

    a. Adopt the xplanet to cope with any output of compare. This could be
    either by just using the return value, hoping that the output on
    different architectures is stable nowadays (I did not test this) or
    by adding logic that only compares the number in case the return
    value is not 0 (perfect match) and then try to interpret the
    number(s).

    b. Apply the imagemagick patches from git main and use the call from 1.
    above.

    Cheers Jochen

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEc7KZy9TurdzAF+h6W//cwljmlDMFAmgnIigACgkQW//cwljm lDNJFA//dddi/7smhhYrRywAD9YW9lnQNgCE1AEvMHeaX+arZQwBTDHH+OsfAQaF 6TZQK+nDW2uYJFdLOYlon8PN8MuL7325ApLKASwex86Up0KYuylSdNxbE0r0DY4c QZLR5H3OSu2j0tVIrFK2mxutfwoyw/fi3ZXQYJSLUAaoEWRAaG8gESd+y8NQIHwH TaCJiDFq7fAxDvmlW28uc0d+STdjKv34EyXmg14TxztSAGAd0JM9Yh4iD9oXEvLu WKf2kOhEHWJRNesgpYxPPOUJtfky1vrYrZQaKfCuIXlfCkUUPARMIxngT0KqUvpw Vcs/e9yNDke8u7yNShYCs7bEIR97XCaf5T1NWxVH29/dpHeK3vVcORlGh8vUcAw4 Xs/Z5qbb6W5QC5md9XPrMz+xJIka5pTxgInQf6VZDXfbRxdUC739Q+CHtJF5kWYx M2AmehyyWobjDUYKFDneNm4zwRFB97WBVVdVCL/pJLxEndSM2lqDuPFmzOlFr1It 19ragFFLVA+Vtmim1U3LJ7OfJ5RWhdCacsU0n6v6u4MNmO/7GCCGWVFH+uAltmiu Q1IuWjMOfNIle5FakbhLN8EJpfN5GU2v+l8gBULH6lH9tNyc47/8Th9nMEK0VKm6 QRWlknq8uotosnw2cScm6eN1WEsUufp88LSKyvIqGepuO0QE/8I=
    =3o4m
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bastien Roucaries@21:1/5 to All on Fri May 16 14:53:23 2025
    Copy: elbrus@debian.org (Paul Gevers)

    Le vendredi 16 mai 2025, 13:31:55 heure d’été d’Europe centrale Jochen Sprickerhof a écrit :
    Hi,

    * Paul Gevers <elbrus@debian.org> [2025-05-10 14:24]:
    With a recent upload of imagemagick the autopkgtest of xplanet fails
    in testing when that autopkgtest is run with the binary packages of >imagemagick from unstable. It passes when run with only packages from

    testing. In tabular form:
    pass fail

    imagemagick from testing 8:7.1.1.47+dfsg1-1
    xplanet from testing 1.3.1-3
    all others from testing from testing

    I copied some of the output at the bottom of this report.

    Currently this regression is blocking the migration of imagemagick to >testing [1]. Due to the nature of this issue, I filed this bug report >against both packages. Can you please investigate the situation and >reassign the bug to the right package?

    There are multiple problems:

    1. imagemagick changed the output of compare in 7.1.1-44 (d85a7583f)
    from some number <= 1 with 1 meaning the images are the same two two
    numbers where the first is the distortion and the second one is the
    old percentage in parenthesis. This breaks the test code in

    https://sources.debian.org/src/xplanet/1.3.1-3/debian/tests/compare-create-> image/#L32 I assume that the new format will be kept so xplanet should adopt
    the logic. Maybe something like:

    result_jpg="$( compare -metric ncc $AUTOPKGTEST_TMP/expected/$filename.jpeg $AUTOPKGTEST_TMP/$filename.jpeg $AUTOPKGTEST_TMP/diff.jpg 2>&1 | sed 's/.*(\(.*\))/\1/' )"

    2. imagemagick in stable returns values below 0 for similar but
    different images. I did not check if this was different in older
    versions but I would expect so, given that the autopkgtest code
    checks the value for >0.99. This has been corrected by multiple
    patches in imagemagick git main.

    3. imagemagick in unstable has a bug resulting in the same picture
    getting a percentage of 0 and different images a value of > 1. This
    has also been corrected in imagemagick git main (912b6a838).

    4. The results of earth_markers.* in the compare-create-image
    autopkgtest improved (not sure due to what) so the files in
    debian/tests/expected/earth_markers.* need to be replaced.

    I think we have to accept that the output of compare is not stable and
    adopt xplanet. I see two options what to do:

    a. Adopt the xplanet to cope with any output of compare. This could be
    either by just using the return value, hoping that the output on
    different architectures is stable nowadays (I did not test this) or
    by adding logic that only compares the number in case the return
    value is not 0 (perfect match) and then try to interpret the
    number(s).

    b. Apply the imagemagick patches from git main and use the call from 1.
    above.4

    go b before hard freeze. Can I have git commit ?


    Cheers Jochen


    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEXQGHuUCiRbrXsPVqADoaLapBCF8FAmgnNUMACgkQADoaLapB CF+6uA/+JgCrxJ5Pt8cuDVF4arHQ70nmL1LhfI/CtK2dcygvFiyVLEmzsrEufdx4 P+oeAiqwXjSKSYSiUPsNB2L6ibQlxSYDQjucI/H9P/ABfx0gzsMc+W8FTvIS5kXL 5ONCC9DnrFbD5yCgvxzJCXX0qOOmwOEdhf9+fsQJfNwUBC0Oa1Y9TgvF9/X6HKhl ZD1L1tbsFe/bm7L02TASKnX6c6yBFxhUjA9HonfM4LfH4dKHYSP7A+1xjekkSxZz 7BfZjkTRaOUEBJDOR2OSsW+d/nFZVKE8US4pAqdS3ecGRwVcvtprgS1Xa1nLyxZ+ er7GdG2AKHkwGa2f4qIBd0WlN1Vmrl0YbrTfZv4ZL9DCEXiQ2twv1/gveBTg1K72 PWXvJ2iRsbzN/NlwjRZJ1mIo19sKH74N0MOVGa2Oz9+diUMz9JKAG3Dru8SmdERi Ua82cX7qW2qjSUzQDLh5vGTMZ8EoAnKuNqnu1zg4ceyKXKrUJ0uEY/ZYcqzR3Oxk I5soZJsmVo0mJvilyxNzR6n/VutfDi+DcHn4L9w0za4eM/D9WEM4zecKaMJ59Sik QYFwDe4+N9cANCb3MjA7verLJxCTqqA1ozRk/O/2evX2whmTIuWFtxo92f6epUd+ Ar1Huz0ZLuT2IN+gPAEuH/IX1n7vl2rWh0k/gK+kwUSfJP13osQ=
    =RuD3
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jochen Sprickerhof@21:1/5 to All on Fri May 16 15:10:01 2025
    Hi Bastien,

    * Bastien Roucaries <rouca@debian.org> [2025-05-16 14:53]:
    Le vendredi 16 mai 2025, 13:31:55 heure d’été d’Europe centrale Jochen >Sprickerhof a écrit :
    Hi,

    * Paul Gevers <elbrus@debian.org> [2025-05-10 14:24]:
    With a recent upload of imagemagick the autopkgtest of xplanet fails
    in testing when that autopkgtest is run with the binary packages of
    imagemagick from unstable. It passes when run with only packages from

    testing. In tabular form:
    pass fail

    imagemagick from testing 8:7.1.1.47+dfsg1-1
    xplanet from testing 1.3.1-3
    all others from testing from testing

    I copied some of the output at the bottom of this report.

    Currently this regression is blocking the migration of imagemagick to
    testing [1]. Due to the nature of this issue, I filed this bug report
    against both packages. Can you please investigate the situation and
    reassign the bug to the right package?

    There are multiple problems:

    1. imagemagick changed the output of compare in 7.1.1-44 (d85a7583f)
    from some number <= 1 with 1 meaning the images are the same two two
    numbers where the first is the distortion and the second one is the
    old percentage in parenthesis. This breaks the test code in

    https://sources.debian.org/src/xplanet/1.3.1-3/debian/tests/compare-create-> image/#L32 I assume that the new format will be kept so xplanet should adopt
    the logic. Maybe something like:

    result_jpg="$( compare -metric ncc
    $AUTOPKGTEST_TMP/expected/$filename.jpeg $AUTOPKGTEST_TMP/$filename.jpeg
    $AUTOPKGTEST_TMP/diff.jpg 2>&1 | sed 's/.*(\(.*\))/\1/' )"

    2. imagemagick in stable returns values below 0 for similar but
    different images. I did not check if this was different in older
    versions but I would expect so, given that the autopkgtest code
    checks the value for >0.99. This has been corrected by multiple
    patches in imagemagick git main.

    3. imagemagick in unstable has a bug resulting in the same picture
    getting a percentage of 0 and different images a value of > 1. This
    has also been corrected in imagemagick git main (912b6a838).

    4. The results of earth_markers.* in the compare-create-image
    autopkgtest improved (not sure due to what) so the files in
    debian/tests/expected/earth_markers.* need to be replaced.

    I think we have to accept that the output of compare is not stable and
    adopt xplanet. I see two options what to do:

    a. Adopt the xplanet to cope with any output of compare. This could be
    either by just using the return value, hoping that the output on
    different architectures is stable nowadays (I did not test this) or
    by adding logic that only compares the number in case the return
    value is not 0 (perfect match) and then try to interpret the
    number(s).

    b. Apply the imagemagick patches from git main and use the call from 1.
    above.4

    go b before hard freeze. Can I have git commit ?

    See all those patches for metrics here:

    https://github.com/ImageMagick/ImageMagick/commits/main/MagickCore/compare.c

    Not sure which ones to select (except 912b6a838).

    Cheers Jochen

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEc7KZy9TurdzAF+h6W//cwljmlDMFAmgnNkcACgkQW//cwljm lDNmehAAoE6MiU9W40zFTnK3xNKzHb8wiiIttln1VTuxs8oIoUywy/dwTnme3aY3 kAbfYPbdrnIJ2Vo1/p3x9rpuEoamtFhGRAmTjAu8JKM16pvS16iIUF0kSo4cY4xZ RpKghTEIp6jRVhZ8ofZWV7Z2PRvzqQWy8+Mt8/imyC0CpHB2ExmaatYkNR6a9Wt5 1ecRAHeKFZzFsMvvEF8aP1JnYhBRQW9HJxJYvWARkPKiY4unWGwbP9BsD3vgS3d4 e5oQFP8koVAhQ56hXQ01pDTLyvL7vEGMOdGPxq4cXeb/VtZ06YmAlgJpcPMjtZi4 xVTj+VtWiVTfOpS8zA2Y0B31g7R1XiL2O/f4MXpfdEWE5hPHcip0rraWIY9IKvai 1cuN3R3JkP+6KvO8IBkBkRatHK4UiVvfj6CMZT5SZryoD90lGVzSctMND+tsKFeM kIXZZ2enWmEw8mbTRz9V2d59+flGCXZ5+L8a9C880GSTdhGBnKOo5mi4PUq4EzaU zk174jmwhGra5vbGkSg3eqFy0ndPPQ1We6cf2XW9GwIM4a7OAvnrMKhgVQsR2ZfQ Jr0Dl1mC6n8NYW9UPPG88IVChxQdBSjl1mvAlKsi4nU4E0tEeya0oDvOkMsoY/eX C9nL1MSlj4MfekF9C325w8B4dGVgbihprxne/UWTGl3jNT4m6T4=
    =5lzm
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)