From Newsgroup: comp.lang.tcl

I have some binary data in a large string. I get the binary data by
selecting a sub-string of the large one with the correct start and end indexes. Everything looks fine.

Then I save it to a file in binary mode. It is opened in"wb" mode, fconfigured, it is written with nonewlines, etc. However, the new file
ends up corrupted. It has the same number of bytes as the original, and
I can confirm they are also identical content-wise by opening them in an editor.

Is there something else I can check?

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

Am 16.02.2026 um 19:16 schrieb saito:

I have some binary data in a large string. I get the binary data by selecting a sub-string of the large one with the correct start and end indexes. Everything looks fine.

Then I save it to a file in binary mode. It is opened in"wb" mode, fconfigured, it is written with nonewlines, etc. However, the new file
ends up corrupted. It has the same number of bytes as the original, and
I can confirm they are also identical content-wise by opening them in an editor.

Is there something else I can check?

Sounds strange...

TCL version ?

What I can imagine is, that there are unicode codepoints not in range
0-255...

Harald
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/16/2026 1:19 PM, Harald Oehlmann wrote:

Am 16.02.2026 um 19:16 schrieb saito:

Is there something else I can check?

Sounds strange...

TCL version ?

What I can imagine is, that there are unicode codepoints not in range 0-255...

I suspect some cr or crnl characters somewhere in there. I trim for them
but it doesn't seem to help.




--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/16/2026 11:11 AM, saito wrote:

On 2/16/2026 1:19 PM, Harald Oehlmann wrote:

Am 16.02.2026 um 19:16 schrieb saito:

Is there something else I can check?

Sounds strange...

TCL version ?

What I can imagine is, that there are unicode codepoints not in range 0-255...

I suspect some cr or crnl characters somewhere in there. I trim for them but it doesn't seem to help.

How did you get the original binary string? Did you construct it or read it in? This sounds like an encoding problem, likely on reading a file without using -encoding binary on the read.

A quick check might be:

puts "String length: [string length $data]"
puts "Byte length: [string length [encoding convertto binary $data]]"


If these differ, your data is already corrupted before writing.

-et


--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/16/2026 3:44 PM, et99 wrote:

A quick check might be:

puts "String length: [string length $data]"
puts "Byte length: [string length [encoding convertto binary $data]]"

I tried it but it says "unknown encoding "binary"". This is on version 8.6.

Here is the data. I can include a few characters before and after it if
you want:

set data [base64::decode { iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5 gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]


--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/16/2026 3:31 PM, saito wrote:

On 2/16/2026 3:44 PM, et99 wrote:

A quick check might be:

puts "String length: [string length $data]"
puts "Byte length: [string length [encoding convertto binary $data]]"

I tried it but it says "unknown encoding "binary"". This is on version 8.6.

Here is the data. I can include a few characters before and after it if you want:

set data [base64::decode { iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5 gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

Ok, I needed some help on this. If you use a range of 0 to end, it will produce a 256 byte valid png picture file, a small square with a gradient at the bottom.

With a range of 1 to end-1, it chops off the first and last bytes of the file, so I think that means it's working.

The encoding iso8859-1 is the key.



package require base64

# Decode your binary data
set data [base64::decode { iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb701aS/CcEwS2SQ5 gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

# Extract bytes using iso8859-1 encoding
set bytes [encoding convertto iso8859-1 $data]

# Extract a range
set start 1
set end end-1
set subset [string range $bytes $start $end]

# Write to file with iso8859-1 encoding
set f [open "image.png" wb]
fconfigure $f -translation binary -encoding iso8859-1
puts -nonewline $f $subset
close $f

puts "Wrote [string length $subset] bytes - should be 254 bytes"

-et




--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/16/2026 3:44 PM, et99 wrote:

A quick check might be:

puts "String length: [string length $data]"
puts "Byte length: [string length [encoding convertto binary $data]]"

I tried it but it says "unknown encoding "binary"". This is on version 8.6.

Here is the data. I can include a few characters before and after it if
you want:

set data [base64::decode { iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5 gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==

$ base64 --decode b64 > png

$ sha256sum png
043a502edc956c6cc750a0182a6309b5e9c2a02d372f16990b9d9b5d414b494d png


$ rlwrap tclsh
% set tcl_patchLevel
8.6.12
% package require base64
2.5
% set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}] ; list
% package require sha256
1.0.4
% sha2::sha256 -hex -- $data
043a502edc956c6cc750a0182a6309b5e9c2a02d372f16990b9d9b5d414b494d
% set middle [string range $data 1 end-1] ; list
% sha2::sha256 -hex -- $middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760
% set fd [open middle {WRONLY CREAT TRUNC BINARY}]
file3
% puts -nonewline $fd $middle
% close $fd
% exit

$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle

With 8.6.12 everything works as expected.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/16/2026 9:07 PM, Rich wrote:

saito <saitology9@gmail.com> wrote:

On 2/16/2026 3:44 PM, et99 wrote:

A quick check might be:

puts "String length: [string length $data]"
puts "Byte length: [string length [encoding convertto binary $data]]"

I tried it but it says "unknown encoding "binary"". This is on version 8.6. >>
Here is the data. I can include a few characters before and after it if
you want:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA >> sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE >> wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5 >> gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz >> iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==

$ base64 --decode b64 > png

$ sha256sum png
043a502edc956c6cc750a0182a6309b5e9c2a02d372f16990b9d9b5d414b494d png

$ rlwrap tclsh
% set tcl_patchLevel
8.6.12
% package require base64
2.5
% set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}] ; list
% package require sha256
1.0.4
% sha2::sha256 -hex -- $data
043a502edc956c6cc750a0182a6309b5e9c2a02d372f16990b9d9b5d414b494d
% set middle [string range $data 1 end-1] ; list
% sha2::sha256 -hex -- $middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760
% set fd [open middle {WRONLY CREAT TRUNC BINARY}]
file3
% puts -nonewline $fd $middle
% close $fd
% exit

$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle

With 8.6.12 everything works as expected.

Thanks for checking this Rich. What I still am unsure of is how the binary data string is created. If done with a base64::decode, then there can't be any codepoints > 0xFF but if the string is read in and the file is not read as binary, then the data could be corrupted there.

In my code, I used -encoding iso8859-1, but after thinking some more, I think this could be -encoding binary. The only advantage of iso8859-1 would be that it should error if there are any codepoints >= 0xFF. I don't know if -encoding binary does that too.

Also, there's [string bytelength $data] that can be used, which if > [string length $data] then there are some bytes that are >= 0x80 and need 2 bytes to encode them.

So, I'm still not sure why Saito was getting any errors on taking a slice with string range.

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

Hi,
please allow me to give my two cents..

Thanks for checking this Rich. What I still am unsure of is how the
binary data string is created. If done with a base64::decode, then there can't be any codepoints > 0xFF but if the string is read in and the file
is not read as binary, then the data could be corrupted there.

Correct

In my code, I used -encoding iso8859-1, but after thinking some more, I think this could be -encoding binary. The only advantage of iso8859-1
would be that it should error if there are any codepoints >= 0xFF. I
don't know if -encoding binary does that too.

"encoding binary" and "-encoding iso-8859-1" are identical.
The more important parameters are -translation and -eofchar

The following two open sequences:
- set all ok for binary operation
- are interoperable for 8.6 and 9

- open $file wb
- open $file wb
- set h [open $file]; fconfigure $h -translation binary

https://core.tcl-lang.org/tcl/wiki?name=Migrating+scripts+to+Tcl+9&p

Also, there's [string bytelength $data] that can be used, which if >
[string length $data] then there are some bytes that are >= 0x80 and
need 2 bytes to encode them.

"string bytelength" will not do what you want. It does not return the
binary length. It returns the current memory consumption of the string,
which may be an arbitrary value. As strings are encoded in modified
UTF-8, most binary bytes consume more than one "string bytelength" value.
Try: string bytelength \0
"string bytelength" is removed in tcl9

So, I'm still not sure why Saito was getting any errors on taking a
slice with string range.

Anything may happen, if a binary string is read as Unicode.
The resulting sequence may be illegal and any string command may error
out. In contrast, any well formed binary data may be handled with any
string command.

Also remark, that a unicode value above \u00ff is xor 0xff on 8.6 and an
error in 9.0:

8.6:
% binary scan \u0100 c v
1
% set v
0

9.0:
% binary scan \u0100 c v
expected byte sequence but character 0 was '-C' (U+000100)

Harald
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

* et99 <et99@rocketship1.me>
| In my code, I used -encoding iso8859-1, but after thinking some more,
| I think this could be -encoding binary. The only advantage of
| iso8859-1 would be that it should error if there are any codepoints >=
| 0xFF. I don't know if -encoding binary does that too.

In my understanding of the TCL source code (generic/tclIO.c), binary
encoding *is* iso8859-1 (and vice versa). Since iso8859-1 treats the
input as single bytes, there cannot be any codepoints > 0xff by
definition.

R'
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

Harald Oehlmann <wortkarg3@yahoo.com> schrieb:

"encoding binary" and "-encoding iso-8859-1" are identical.
The more important parameters are -translation and -eofchar

When I had similar aches, I found that using the readFile/writeFile
convenience functions from the Tcl library[1] is the happy path.

[1] man 3tcl library

Best regards,
Tilmann.
--
Cow-Kasus
kuhf|nlliger Name eines Gebirgszugs in Rinderindien
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 12:07 AM, Rich wrote:

saito <saitology9@gmail.com> wrote:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA >> sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE >> wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5 >> gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz >> iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==

$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle

With 8.6.12 everything works as expected.

The issue is that the final image, after the data is written to a file,
is corrupted. Can you open that image file and see it? For me, if I try
to open from a File Epxlorer, it can't. Even the small preview image is
messed up. Some image programs may open it but after complaining and
with a lot of distortions.


--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 5:37 AM, et99 wrote:

On 2/16/2026 9:07 PM, Rich wrote:

So, I'm still not sure why Saito was getting any errors on taking a
slice with string range.

...

However, the new file ends up corrupted. It has the same number of bytes as the original, and I can confirm they are also identical content-wise by opening them in an editor.

The string manipulation and saving it seems OK. However, the new file is corrupted. In Windows, File Explorer can open it, and its small preview
icon is messed up.

I asked Claude AI and it said the new image file is corrupted beyond
repair. It said it has ascii characters inside it. Weird.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 8:37 AM, saito wrote:

On 2/17/2026 12:07 AM, Rich wrote:

saito <saitology9@gmail.com> wrote:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

-a-a $ cat b64
-a-a iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
-a-a sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
-a-a wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
-a-a gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
-a-a iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==

-a-a $ sha256sum middle
-a-a 1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760-a middle

With 8.6.12 everything works as expected.

The issue is that the final image, after the data is written to a file, is corrupted. Can you open that image file and see it? For me, if I try to open from a File Epxlorer, it can't. Even the small preview image is messed up. Some image programs may open it but after complaining and with a lot of distortions.

Are you saying that after taking a slice of that 256 bytes and writing that out you can't display the picture any longer?

That file is a PNG with a 4 byte header 0x89 followed by "PNG", and any slice of it is unlikely to display. When I chopped the first byte off and tried to display it using irfanview, it said unknown picture format or corrupted file, which is just what I would expect. It's not 256 raw pixel bytes.

Is this what you meant by corruption?

If, however, after the base64::decode, which produced the 256 bytes, when all was written to the file as binary, it did display a very small picture - square or rectangular mostly light blue with a gradient at the bottom. I had to zoom in to see that though.

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 3:10 PM, et99 wrote:

Are you saying that after taking a slice of that 256 bytes and writing
that out you can't display the picture any longer?

Yes, it is corrupted. I chose a rather small picture so its b64 version wouldn't be overwhelming for posting. So some programs do display it
with some distortions, and others flat out refuse it. For larger files,
it always gets refused as being corrupted. The gradient lines you see
are distortions.

I made a page on this. Hopefully someone can shed some light on it and
help solve it.

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/17/2026 12:07 AM, Rich wrote:

saito <saitology9@gmail.com> wrote:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==

$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle >>
With 8.6.12 everything works as expected.

The issue is that the final image, after the data is written to a file,
is corrupted. Can you open that image file and see it? For me, if I try
to open from a File Epxlorer, it can't. Even the small preview image is messed up. Some image programs may open it but after complaining and
with a lot of distortions.

The original base64 data was a tiny png image. It opened in an image
viewer just fine.

My "slice" was just taking everything minus the very first and very
last byte. That is no longer a valid png image. However what
sha256sum found as the hash while still in Tcl's binary string matched
what was written to the file.

Is there a reason you are trying to "clip" pieces out of a binary
string? Do you have a series of concatenated png's and you are trying
to separate them? Then perhaps your indexes are not exactly where they
need to be to extract all the bytes necessary to produce a valid
output?
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/17/2026 3:10 PM, et99 wrote:

Are you saying that after taking a slice of that 256 bytes and writing
that out you can't display the picture any longer?

Yes, it is corrupted.

That would have been helpful to have mentioned in the article where you
posted the base64 encoding.

feh (which using imlib2) does not complain.

pngtopam does complain:

$ pngtopam img.png > img.pam
pngtopam: fatal libpng error: IDAT: invalid distance too far back
pngtopam: setjmp returns error condition

That error message makes me think there's a back-match length in the
gzip data that is saying to look before the start of the gzip data.

I made a page on this. Hopefully someone can shed some light on it and
help solve it.

We can't read your mind. Where's "the page"?

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 8:11 PM, Rich wrote:

saito <saitology9@gmail.com> wrote:

On 2/17/2026 12:07 AM, Rich wrote:

saito <saitology9@gmail.com> wrote:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==



$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle

With 8.6.12 everything works as expected.

The issue is that the final image, after the data is written to a file,
is corrupted. Can you open that image file and see it? For me, if I try
to open from a File Epxlorer, it can't. Even the small preview image is
messed up. Some image programs may open it but after complaining and
with a lot of distortions.

The original base64 data was a tiny png image. It opened in an image
viewer just fine.

My "slice" was just taking everything minus the very first and very
last byte. That is no longer a valid png image. However what
sha256sum found as the hash while still in Tcl's binary string matched
what was written to the file.

Is there a reason you are trying to "clip" pieces out of a binary
string? Do you have a series of concatenated png's and you are trying
to separate them? Then perhaps your indexes are not exactly where they
need to be to extract all the bytes necessary to produce a valid
output?

This is what has confused me. If Saito is trying to extract portions of a png file, to somehow display them, this won't work. If some picture viewers are actually able to display something, it's likely they are treating the data as raw.

The old photoshop used to do this when it only had a raw string of bytes. But if any of the png data is compressed, there's no way treating it as raw pixels should display anything meaningful. And even when it doesn't compress, it still has filters and even uncompressed blocks have header bytes.

Bottom line: He needs to decode the PNG first, then extract pixels and create a new image to output, not extract PNG file bytes.




--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 8:41 PM, et99 wrote:

Is there a reason you are trying to "clip" pieces out of a binary
string?-a Do you have a series of concatenated png's and you are trying
to separate them?-a Then perhaps your indexes are not exactly where they
need to be to extract all the bytes necessary to produce a valid
output?

Ah, this could be it. But with png files, one needs to skip over each chunk until the end of each png, or at least scan for the iend chunk, which is probably safely unique, but not guaranteed. And then there could be junk data between the pngs, so you'd need to find the next signature for the start of the next one.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

et99 <et99@rocketship1.me> wrote:

On 2/17/2026 8:11 PM, Rich wrote:

saito <saitology9@gmail.com> wrote:

On 2/17/2026 12:07 AM, Rich wrote:

saito <saitology9@gmail.com> wrote:

set data [base64::decode {
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==}]

$ cat b64
iVBORw0KGgoAAAANSUhEUgAAABEAAAATCAIAAAD5x3GmAAAACXBIWXMAAAsSAAALEgHS3X78AAAA
sklEQVQokZ1SQQ7CMAyL3fz/lfwAicNgNYeMqEBTMXzYYY1ruzEu15skmwHA9L9v28OsF5yW15EE
wMB0+htBltR7d0nk3EPYS6mk+cJ3DOeppKA5yUpHwnh9fr21BszfzQypE0kOb601aS/CcEwS2SQ5
gGIg9X72/V6dxd5e7yDSSEr4dT+j+AlO2mFtbEIAAMD/0PGxHevRxCpPluUEp4L33uuO6sP50Tcz
iyItaPae6gkuMnIItElP1QAAAABJRU5ErkJggg==



$ sha256sum middle
1b886148980fddaa2f333c7e6ecad3f31d1385b831b17431ed93d2f9133cb760 middle

With 8.6.12 everything works as expected.

The issue is that the final image, after the data is written to a file,
is corrupted. Can you open that image file and see it? For me, if I try
to open from a File Epxlorer, it can't. Even the small preview image is
messed up. Some image programs may open it but after complaining and
with a lot of distortions.

The original base64 data was a tiny png image. It opened in an image
viewer just fine.

My "slice" was just taking everything minus the very first and very
last byte. That is no longer a valid png image. However what
sha256sum found as the hash while still in Tcl's binary string matched
what was written to the file.

Is there a reason you are trying to "clip" pieces out of a binary
string? Do you have a series of concatenated png's and you are trying
to separate them? Then perhaps your indexes are not exactly where they
need to be to extract all the bytes necessary to produce a valid
output?

This is what has confused me.

Not just you. Whatever Saito is doing, it has been massively
underspecified here to the point that it is hard to help.

If Saito is trying to extract portions of a png file, to somehow
display them, this won't work.

In general, yes. One cannot simply [string range ...] on a PNG and get
back a valid image.

What can work is if one is parsing the PNG chunk structure and
extracting particular chunks to work with (i.e. searching for and
extracting PNG text comment chunks) -- but Saito has not
indicated that this is what is being attempted

If some picture viewers are actually able to display something, it's
likely they are treating the data as raw.

Different viewers do different things when presented with data they
can't identify. pngtopam simply gave up and produced no output (not an unreasonable result, esp. w/ a CLI tool). feh (and in fact imlib2)
made a best effort to interpret the 256 bytes as a tiny image (also not
an unreasonable response from a GUI tool).

The old photoshop used to do this when it only had a raw string of
bytes.

Never used Photoshop (old or new) so I did not know this about it.

But if any of the png data is compressed, there's no way
treating it as raw pixels should display anything meaningful.

Well, it would display something. But the resulting image will likely
look more like randomly generated pixel data than anything
recognizable.

And even when it doesn't compress, it still has filters and even uncompressed blocks have header bytes.

If you read the PNG spec, there are no "uncompressed" PNG's. As in the
pixel data is always the output from a gzip compression pass. The
underlying gzip algorithm may output an 'uncompressed' stream, but
"PNG" always has "gzip streams" as pixel data inside.

Bottom line: He needs to decode the PNG first, then extract pixels
and create a new image to output, not extract PNG file bytes.

If he is trying to display/extract pixel data from PNG's, yes, he has
lots more work to do than just [string range ...] and [open ...
{binary}]. But he's not said that is what he is doing. He could (made
up example) have some game data file that is actually a set of
concatenated PNG images and he may be trying to find the start/end of
each and write each to an individual image file. Or he could be doing
a nearly infinite set of other things, none of which he's told us any
info about.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

et99 <et99@rocketship1.me> wrote:

On 2/17/2026 8:41 PM, et99 wrote:

Is there a reason you are trying to "clip" pieces out of a binary
string?-a Do you have a series of concatenated png's and you are trying
to separate them?-a Then perhaps your indexes are not exactly where they
need to be to extract all the bytes necessary to produce a valid
output?

Ah, this could be it. But with png files, one needs to skip over
each chunk until the end of each png, or at least scan for the iend
chunk, which is probably safely unique, but not guaranteed. And then
there could be junk data between the pngs, so you'd need to find the
next signature for the start of the next one.

And, if the creator of the concatenated PNG's was devious, there could
be ztxt chunks mixed in that contain false PNG header or false PNG
ending signatures to confuse someone trying to find the start/end
indexes of the actual PNG's in the concatenated file.

If one knows beforehand where each PNG begins, then the 'false header'
and 'false trailer' signatures are skipped over by following the chunk
links. If one is scanning the raw binary bytes looking for
header/footer signatures, these would very much confuse things.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/18/2026 9:15 AM, Rich wrote:

et99 <et99@rocketship1.me> wrote:

This is what has confused me.

Not just you. Whatever Saito is doing, it has been massively
underspecified here to the point that it is hard to help.

Sorry to hear that. I believe I was pretty clear. In my first post, I
said this:

"I have some binary data in a large string. I get the binary data by
selecting a sub-string of the large one with the correct start and end indexes."

In general, yes. One cannot simply [string range ...] on a PNG and get
back a valid image.

I never said anything about parsing png's or clipping png's or anything
like that. I see that et99 guessed that this *might* be the case. It was
just a guess, no need to treat it as fact or as if I said it.


--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/17/2026 11:27 PM, Rich wrote:

saito <saitology9@gmail.com> wrote:

On 2/17/2026 3:10 PM, et99 wrote:

Are you saying that after taking a slice of that 256 bytes and writing
that out you can't display the picture any longer?

Yes, it is corrupted.

That would have been helpful to have mentioned in the article where you posted the base64 encoding.

By "it", I mean the png file. In the context of the question above, "you
can't display the picture any longer?"

I made a page on this. Hopefully someone can shed some light on it and
help solve it.

We can't read your mind. Where's "the page"?

Fair point. I meant a wiki page. There aren't that many places to get
help on Tcl.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

* saito <saitology9@gmail.com>
| Sorry to hear that. I believe I was pretty clear. In my first post, I
| said this:

| "I have some binary data in a large string. I get the binary data by
| selecting a sub-string of the large one with the correct start and end
| indexes."

Rich has presented in Message-ID: <10n0t3a$1fqd7$1@dont-email.me>
a means to check the data for validity, both in memory and when written
on disk - calculate the checksums of the data. Could you repeat that
check and post the code you used to perform it? Or maybe start with the
code Rich has posted and check whether that works on your side, too.

R'
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/17/2026 11:27 PM, Rich wrote:

saito <saitology9@gmail.com> wrote:

On 2/17/2026 3:10 PM, et99 wrote:

Are you saying that after taking a slice of that 256 bytes and writing >>>> that out you can't display the picture any longer?

Yes, it is corrupted.

That would have been helpful to have mentioned in the article where you
posted the base64 encoding.

By "it", I mean the png file.

Well, that, but the fact that the 256 bytes of base64 encoded binary
was a png that was "corrupted". Since feh did not complain about it, I
did not even realize it was "corrupted" until a later message here
indicated that fact.

In the context of the question above, "you can't display the picture
any longer?"

The 256 byte png will display with feh, but the CLI tool pngtopam
complains about it and does not generate any output at all. Two
different tools, two opposite extremes in handling.

I made a page on this. Hopefully someone can shed some light on it and
help solve it.

We can't read your mind. Where's "the page"?

Fair point. I meant a wiki page. There aren't that many places to get
help on Tcl.

Sadly, the wiki is thousands of pages. A URL of the actual page is
what would be helpful to us here who can't read your mind to know which
page you created.
--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/18/2026 9:15 AM, Rich wrote:

et99 <et99@rocketship1.me> wrote:

This is what has confused me.

Not just you. Whatever Saito is doing, it has been massively
underspecified here to the point that it is hard to help.

Sorry to hear that. I believe I was pretty clear. In my first post, I
said this:

"I have some binary data in a large string. I get the binary data by selecting a sub-string of the large one with the correct start and end indexes."

Which still provides too little context for us here who don't know why
you have a "large string" nor why you are trying to snip pieces out,
nor why the example 256 byte file you posted was a png image (and, was
that 256 bytes snipped from the larger string?).

In general, yes. One cannot simply [string range ...] on a PNG and get
back a valid image.

I never said anything about parsing png's or clipping png's or
anything like that.

True, but because you never said anything beyond "large binary string"
and "snipping pieces using indexes from large binary string" et99 was
left to guess at what you might really be doing, and the fact that the
example turned out to be a png file, and having zero other context,
making a guess that you might be extracting pngs from a large binary
string, or extracting pieces from individual pngs, at least fit the
lack of context we have so far.

I see that et99 guessed that this *might* be the case. It was just a
guess, no need to treat it as fact or as if I said it.

--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

On 2/18/2026 2:59 PM, Rich wrote:

Sadly, the wiki is thousands of pages. A URL of the actual page is
what would be helpful to us here who can't read your mind to know which
page you created.

It is listed on the main page. It is a new page so it shows up under
Recently changed. It is titled "Help Tclhttpd".

I don't have the tools you have nor the familiarity with them. So I
really hope that you could identify what the issue is.

I trust the string because it is the query string. Tcl httpd works from
this same string and is able to create a proper png file from the binary
part of it. However, that method doesn't work for me for a few reasons,
one being not providing the type of the file.

If we can fix the proc on that page as described, that will be the ideal solution. I see that it does a lot of regsubs and reverses them later.

Based on your statement that the string is indeed corrupted, I suspect
that the reverse regsub's have not been applied yet. But I am not sure
where or how it happens so I can fix it.





--- Synchronet 3.21b-Linux NewsLink 1.2

From Newsgroup: comp.lang.tcl

saito <saitology9@gmail.com> wrote:

On 2/18/2026 2:59 PM, Rich wrote:

Sadly, the wiki is thousands of pages. A URL of the actual page is
what would be helpful to us here who can't read your mind to know which
page you created.

It is listed on the main page. It is a new page so it shows up under Recently changed. It is titled "Help Tclhttpd".

Yes, but until this message here, you have not told us anything here on
Usenet about this being within Tclhttpd. Had I gone to look at the
recent changes list from your prior message here, I would never have considered that "Help Tclhttpd" would have ever been the page that you
had created.

I don't have the tools you have nor the familiarity with them. So I
really hope that you could identify what the issue is.

For us to help, we have to understand what's going on. And you've left
out lots of not just useful, but necessary, details here on Usenet.
The wiki page is 100x more detailed (and it still omits some useful
details, like which http attribute options you've set on your "file
input" tag. They may matter, because it is possible your "corruption"
may be occurring during the upload rather than anywhere in the Tclhttpd
side.

I trust the string because it is the query string.

When dealing with something like this, trust nothing. Verify that
every step along the way is doing exactly what you expect it should be
doing. Any part you implicitly trust is a part where the issue might
lie, and that trust will mean you miss the actual problem.

Tcl httpd works from this same string and is able to create a proper
png file from the binary part of it. However, that method doesn't
work for me for a few reasons, one being not providing the type of
the file.

If Tcl httpd can create a proper png (which I interpret to mean "not corrupted") then why do you care one iota about "providing the type of
the file". The filename is usually included with the form upload that includes the data, just save it out under the supplied name (or save
the supplied name somewhere to use for download later).

If we can fix the proc on that page as described, that will be the ideal solution. I see that it does a lot of regsubs and reverses them later.

On the help Tclhttpd the only code present does zero regexes. The code
is a very simple server side 'receiver' that simply returns a basic
html page showing what form variables were in the upload. But there
are zero regexes on that page.

Are you referring to an as yet unnamed other page?

Based on your statement that the string is indeed corrupted, I suspect
that the reverse regsub's have not been applied yet. But I am not sure
where or how it happens so I can fix it.

Again, zero regexes on the Help Tclhttpd page.

WE CANNOT READ YOUR MIND OVER USENET.
--- Synchronet 3.21b-Linux NewsLink 1.2