From Newsgroup: comp.unix.shell

Is there a (standard) tool or simple way to remove ANSI escapes from
a file or stream of data without enumerating all possible sequences?

My first thought was to use sed to remove $'\E[[][^A-Za-z]*[A-Za-z]'
but then I noticed there's, e.g., also other patterns like $'\E[78]'
to consider.

Janis
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.unix.shell

On 09/09/2025 10:26, Janis Papanagnou wrote:

Is there a (standard) tool or simple way to remove ANSI escapes from
a file or stream of data without enumerating all possible sequences?

I don't think so.

My first thought was to use sed to remove $'\E[[][^A-Za-z]*[A-Za-z]'
but then I noticed there's, e.g., also other patterns like $'\E[78]'
to consider.

<https://en.wikipedia.org/wiki/ANSI_escape_code> says:

"""
For Control Sequence Introducer, or CSI, commands, the ESC [ (written as
\e[, \x1b[ or \033[ in several programming languages) is followed by any number (including none) of "parameter bytes" in the range 0x30rCo0x3F
(ASCII 0rCo9:;<=>?), then by any number of "intermediate bytes" in the
range 0x20rCo0x2F (ASCII space and !"#$%&'()*+,-./), then finally by a
single "final byte" in the range 0x40rCo0x7E (ASCII @ArCoZ[\]^_`arCoz{|}~).
"""


I have this (although I only use it to strip off ansi colour sequences,
never tried it with anything else).

#include <stdio.h>

int main(void)
{
int c, n;

while ( (c = fgetc(stdin)) != EOF )
{
if ( c == 0x1b )
{
if ( (n = fgetc(stdin)) != '[' )
ungetc(n, stdin);
else
{
while ( (c = fgetc(stdin)) != EOF )
{
if ( c >= 0x40 && c <= 0x7e )
break;
}

continue;
}
}

fputc(c, stdout);
}

return 0;
}



--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.unix.shell

Richard Harnden <richard.nospam@gmail.invalid> wrote or quoted:

On 09/09/2025 10:26, Janis Papanagnou wrote:

Is there a (standard) tool or simple way to remove ANSI escapes from
a file or stream of data without enumerating all possible sequences?

I don't think so.

There are ansifilter and astrp.

(It seems like the hot thing in the Python world right now
is making every terminal output all flashy and colorful.
The problem is, I can barely read it anymore since the contrast
against the background ends up way too low . . . )


--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.unix.shell

Richard Harnden <richard.nospam@gmail.invalid> wrote or quoted:

I have this (although I only use it to strip off ansi colour sequences, >never tried it with anything else).

Thanks for the interesting source code!

I took some inspiration from it and tried writing a Python
script. Below you can see the demo output [1] and the script [2].
- That said, I never actually tested "print_help" or "main"!

[1] Demo Output

Before strip_c0_controls: 'Hello\x07World'
After: 'HelloWorld'

Before strip_c1_controls: 'Test\x1bOTest'
After: 'TestTest'

Before strip_csi_sequences: 'Blue\x1b[1;34mText\x1b[0m'
After: 'BlueText'

Before strip_string_terminated_sequences: 'Test\x1bPExample DCS\x1b\\Test' After: 'TestTest'

Before strip_ansi_sequences: 'Hello\x07World\x1bOBold \x1b[1;34mBlue\x1b[0m Normal \x1bPThis is a DCS message\x1b\\ Text\x1b]2;Title\x1b\\ More\x1b^Privacy Msg\x1b\\ End\x1b_UniqueCmd\x1b\\ Finish.'
After: 'HelloWorldBold Blue Normal Text More End Finish.'

[2] ansi-strip.py

# Copyright 2025 Stefan Ram
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This thing was just thrown together pretty quickly, it hasn't really
# been used in practice yet and barely got tested. - Stefan Ram on 2025-09-09

import argparse
import re
import sys

# ANSI escape sequence components based on the grammar of ECMA-48:
#
# ESC = "\x1B" (escape character, starts most control sequences)
# CSI = ESC + "[" (Control Sequence Introducer)
# ST = ESC + "\" # String Terminator (ends string-terminated sequences)
# DIGIT = "0" | "1" | ... | "9"
# SEMICOLON = ";"
# parameters = zero or more DIGIT sequences separated by SEMICOLON, may include '?'
# command_char = ASCII chars in range 0x40 to 0x7E (final byte of CSI sequences)
#
# C1 control sequences = ESC followed by one byte in range 0x40 to 0x5F (@ to _)
# String-terminated sequences (DCS, OSC, PM, APC) start with ESC + [P, ], ^, _] # and end with the String Terminator (ESC \)
# C0 controls are single bytes in 0x00rCo0x1F excluding ESC (0x1B), plus DEL (0x7F)

def strip_c1_controls(text):
"""Strip C1 control sequences: ESC followed by one char in range @-_ (0x40-0x5F)."""
pattern = re.compile(r'\x1B[@-_]')
return pattern.sub('', text)

def strip_csi_sequences(text):
"""Strip CSI sequences: ESC [ parameters command_char."""
pattern = re.compile(r'\x1B\[[0-9;?]*[\x40-\x7E]')
return pattern.sub('', text)

def strip_string_terminated_sequences(text):
"""Strip DCS, OSC, PM, APC sequences starting ESC P, ESC ], ESC ^, ESC _
terminated by ESC \."""
pattern = re.compile(r'\x1B[P\]\^_].*?\x1B\\', re.DOTALL)
return pattern.sub('', text)

def strip_c0_controls(text):
"""Strip C0 control characters except ESC (0x1B) which is 27 decimal."""
# 0x00-0x1F except 0x1B (ESC), plus 0x7F (DEL)
pattern = re.compile(r'[\x00-\x1A\x1C-\x1F\x7F]')
return pattern.sub('', text)

def strip_ansi_sequences(text):
prev = None
while prev != text:
prev = text
# Remove ESC-based sequences first, so ESC bytes remain intact
text = strip_string_terminated_sequences(text)
text = strip_csi_sequences(text)
text = strip_c1_controls(text)
# Finally remove remaining C0 controls excluding ESC
text = strip_c0_controls(text)
return text

def demo():
# Test parts individually
c0_str = "Hello\x07World" # C0
c1_str = "Test\x1BOTest" # C1
csi_str = "Blue\x1B[1;34mText\x1B[0m" # CSI
sts_str = "Test\x1BPExample DCS\x1B\\Test" # String terminated sequences

print("Before strip_c0_controls:", repr(c0_str))
print("After:", repr(strip_c0_controls(c0_str)), "\n")

print("Before strip_c1_controls:", repr(c1_str))
print("After:", repr(strip_c1_controls(c1_str)), "\n")

print("Before strip_csi_sequences:", repr(csi_str))
print("After:", repr(strip_csi_sequences(csi_str)), "\n")

print("Before strip_string_terminated_sequences:", repr(sts_str))
print("After:", repr(strip_string_terminated_sequences(sts_str)), "\n")

# Combined test
combined_str = (
"Hello\x07World"
+ "\x1BO"
+ "Bold \x1B[1;34mBlue\x1B[0m Normal "
+ "\x1BPThis is a DCS message\x1B\\"
+ " Text\x1B]2;Title\x1B\\"
+ " More\x1B^Privacy Msg\x1B\\"
+ " End\x1B_UniqueCmd\x1B\\"
+ " Finish."
)
print("Before strip_ansi_sequences:", repr(combined_str))
print("After:", repr(strip_ansi_sequences(combined_str)))

def print_help():
# WARNING: was never tested!
help_text = """
ansi-strip - Strip ANSI control sequences from text input

Usage:
ansi-strip [OPTIONS] [FILE]

Options:
--help Show this help message and exit

If FILE is not provided, reads from standard input.
Stripped output is written to standard output.

This utility removes ANSI escape sequences including:
- C0 and C1 control characters
- CSI sequences (Control Sequence Introducer)
- String-terminated sequences (DCS, OSC, PM, APC)

Example:
python3 ansi-strip.py file.txt > clean.txt
cat file.txt | ansi-strip
"""
print(help_text.strip())

def main():
# WARNING: was never tested!
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('file', nargs='?', help='Input file (default: stdin)')
parser.add_argument('--help', action='store_true', help='Show help message and exit')

args = parser.parse_args()

if args.help:
print_help()
sys.exit(0)

# Read input
if args.file:
try:
with open(args.file, 'r', encoding='utf-8') as f:
content = f.read()
except Exception as e:
print(f"Error reading file {args.file}: {e}", file=sys.stderr)
sys.exit(1)
else:
content = sys.stdin.read()

# Strip ANSI sequences
output = strip_ansi_sequences(content)

# Write output
try:
sys.stdout.write(output)
except BrokenPipeError:
# Handle broken pipe for Unix pipe usage (e.g. head)
pass

if __name__ == "__main__":
# demo()
main()


--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.unix.shell

ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:

Usage:
ansi-strip [OPTIONS] [FILE]

. . .

python3 ansi-strip.py file.txt > clean.txt
cat file.txt | ansi-strip

If you don't set up the script the right way, you can just run
it with "python3 ansi-strip.py". But you can also set it up
so you just call it with "ansi-strip" instead, like by adding
a shebang, using chmod, and renaming it.

Here's another demo:

Output:

reset: output: '][', input: ']\x1b[0m['
bold: output: '][', input: ']\x1b[1m['
dim: output: '][', input: ']\x1b[2m['
italic: output: '][', input: ']\x1b[3m['
underline: output: '][', input: ']\x1b[4m['
blink: output: '][', input: ']\x1b[5m['
reverse: output: '][', input: ']\x1b[7m['
hidden: output: '][', input: ']\x1b[8m['
strikethrough: output: '][', input: ']\x1b[9m['
fg_black: output: '][', input: ']\x1b[30m['
fg_red: output: '][', input: ']\x1b[31m['
fg_green: output: '][', input: ']\x1b[32m['
fg_yellow: output: '][', input: ']\x1b[33m['
fg_blue: output: '][', input: ']\x1b[34m['
fg_magenta: output: '][', input: ']\x1b[35m['
fg_cyan: output: '][', input: ']\x1b[36m['
fg_white: output: '][', input: ']\x1b[37m['
fg_default: output: '][', input: ']\x1b[39m['
bg_black: output: '][', input: ']\x1b[40m['
bg_red: output: '][', input: ']\x1b[41m['
bg_green: output: '][', input: ']\x1b[42m['
bg_yellow: output: '][', input: ']\x1b[43m['
bg_blue: output: '][', input: ']\x1b[44m['
bg_magenta: output: '][', input: ']\x1b[45m['
bg_cyan: output: '][', input: ']\x1b[46m['
bg_white: output: '][', input: ']\x1b[47m['
bg_default: output: '][', input: ']\x1b[49m['
fg_bright_black: output: '][', input: ']\x1b[90m['
fg_bright_red: output: '][', input: ']\x1b[91m['
fg_bright_green: output: '][', input: ']\x1b[92m['
fg_bright_yellow: output: '][', input: ']\x1b[93m['
fg_bright_blue: output: '][', input: ']\x1b[94m['
fg_bright_magenta: output: '][', input: ']\x1b[95m['
fg_bright_cyan: output: '][', input: ']\x1b[96m['
fg_bright_white: output: '][', input: ']\x1b[97m['
bg_bright_black: output: '][', input: ']\x1b[100m['
bg_bright_red: output: '][', input: ']\x1b[101m['
bg_bright_green: output: '][', input: ']\x1b[102m['
bg_bright_yellow: output: '][', input: ']\x1b[103m['
bg_bright_blue: output: '][', input: ']\x1b[104m['
bg_bright_magenta: output: '][', input: ']\x1b[105m['
bg_bright_cyan: output: '][', input: ']\x1b[106m['
bg_bright_white: output: '][', input: ']\x1b[107m['
cursor_up: output: '][', input: ']\x1b[1A['
cursor_down: output: '][', input: ']\x1b[1B['
cursor_forward: output: '][', input: ']\x1b[1C['
cursor_back: output: '][', input: ']\x1b[1D['
cursor_home: output: '][', input: ']\x1b[H['
fg_256_1: output: '][', input: ']\x1b[38;5;34m['
fg_256_2: output: '][', input: ']\x1b[38;5;198m['
fg_256_3: output: '][', input: ']\x1b[38;5;117m['
fg_256_4: output: '][', input: ']\x1b[38;5;202m['
fg_256_5: output: '][', input: ']\x1b[38;5;9m['
fg_256_6: output: '][', input: ']\x1b[38;5;226m['
fg_256_7: output: '][', input: ']\x1b[38;5;45m['
fg_256_8: output: '][', input: ']\x1b[38;5;160m['
fg_256_9: output: '][', input: ']\x1b[38;5;240m['
fg_256_10: output: '][', input: ']\x1b[38;5;75m['
bg_256_1: output: '][', input: ']\x1b[48;5;33m['
bg_256_2: output: '][', input: ']\x1b[48;5;129m['
bg_256_3: output: '][', input: ']\x1b[48;5;221m['
bg_256_4: output: '][', input: ']\x1b[48;5;196m['
bg_256_5: output: '][', input: ']\x1b[48;5;15m['
bg_256_6: output: '][', input: ']\x1b[48;5;100m['
bg_256_7: output: '][', input: ']\x1b[48;5;5m['
bg_256_8: output: '][', input: ']\x1b[48;5;240m['
bg_256_9: output: '][', input: ']\x1b[48;5;28m['
bg_256_10: output: '][', input: ']\x1b[48;5;93m['
fg_rgb_1: output: '][', input: ']\x1b[38;2;12;200;150m[' fg_rgb_2: output: '][', input: ']\x1b[38;2;255;180;60m[' fg_rgb_3: output: '][', input: ']\x1b[38;2;90;20;240m[' fg_rgb_4: output: '][', input: ']\x1b[38;2;10;175;75m[' fg_rgb_5: output: '][', input: ']\x1b[38;2;200;40;180m[' fg_rgb_6: output: '][', input: ']\x1b[38;2;70;255;10m[' fg_rgb_7: output: '][', input: ']\x1b[38;2;250;120;120m[' fg_rgb_8: output: '][', input: ']\x1b[38;2;30;200;255m[' fg_rgb_9: output: '][', input: ']\x1b[38;2;220;220;90m[' fg_rgb_10: output: '][', input: ']\x1b[38;2;180;80;20m[' bg_rgb_1: output: '][', input: ']\x1b[48;2;220;40;70m[' bg_rgb_2: output: '][', input: ']\x1b[48;2;15;120;255m[' bg_rgb_3: output: '][', input: ']\x1b[48;2;80;200;120m[' bg_rgb_4: output: '][', input: ']\x1b[48;2;160;90;240m[' bg_rgb_5: output: '][', input: ']\x1b[48;2;255;220;180m[' bg_rgb_6: output: '][', input: ']\x1b[48;2;100;40;20m[' bg_rgb_7: output: '][', input: ']\x1b[48;2;250;150;10m[' bg_rgb_8: output: '][', input: ']\x1b[48;2;30;255;170m[' bg_rgb_9: output: '][', input: ']\x1b[48;2;170;170;210m[' bg_rgb_10: output: '][', input: ']\x1b[48;2;200;200;30m[' cursor_pos_1: output: '][', input: ']\x1b[3;5H['
cursor_pos_2: output: '][', input: ']\x1b[10;15H['
cursor_pos_3: output: '][', input: ']\x1b[40;20H['
cursor_pos_4: output: '][', input: ']\x1b[25;60H['
cursor_pos_5: output: '][', input: ']\x1b[5;30H['
cursor_pos_6: output: '][', input: ']\x1b[12;12H['
cursor_pos_7: output: '][', input: ']\x1b[70;10H['
cursor_pos_8: output: '][', input: ']\x1b[15;45H['
cursor_pos_9: output: '][', input: ']\x1b[1;80H['
cursor_pos_10: output: '][', input: ']\x1b[50;25H['
erase_line: output: '][', input: ']\x1b[2K['
erase_to_end: output: '][', input: ']\x1b[K['
erase_screen: output: '][', input: ']\x1b[2J[' erase_screen_from_cursor: output: '][', input: ']\x1b[J['
save_cursor: output: '][', input: ']\x1b[s['
restore_cursor: output: '][', input: ']\x1b[u['
alt_screen_on: output: '][', input: ']\x1b[?1049h['
alt_screen_off: output: '][', input: ']\x1b[?1049l['

Source code (to be called in the context of the
previous script):

def demo1():
# ANSI Escape Sequences Demonstration in Python

# Reset
reset = "\033[0m"

# Text style
bold = "\033[1m"
dim = "\033[2m"
italic = "\033[3m"
underline = "\033[4m"
blink = "\033[5m"
reverse = "\033[7m"
hidden = "\033[8m"
strikethrough = "\033[9m"

# Foreground colors (basic)
fg_black = "\033[30m"
fg_red = "\033[31m"
fg_green = "\033[32m"
fg_yellow = "\033[33m"
fg_blue = "\033[34m"
fg_magenta = "\033[35m"
fg_cyan = "\033[36m"
fg_white = "\033[37m"
fg_default = "\033[39m"

# Background colors (basic)
bg_black = "\033[40m"
bg_red = "\033[41m"
bg_green = "\033[42m"
bg_yellow = "\033[43m"
bg_blue = "\033[44m"
bg_magenta = "\033[45m"
bg_cyan = "\033[46m"
bg_white = "\033[47m"
bg_default = "\033[49m"

# Bright foreground colors
fg_bright_black = "\033[90m"
fg_bright_red = "\033[91m"
fg_bright_green = "\033[92m"
fg_bright_yellow = "\033[93m"
fg_bright_blue = "\033[94m"
fg_bright_magenta = "\033[95m"
fg_bright_cyan = "\033[96m"
fg_bright_white = "\033[97m"

# Bright backgrounds
bg_bright_black = "\033[100m"
bg_bright_red = "\033[101m"
bg_bright_green = "\033[102m"
bg_bright_yellow = "\033[103m"
bg_bright_blue = "\033[104m"
bg_bright_magenta = "\033[105m"
bg_bright_cyan = "\033[106m"
bg_bright_white = "\033[107m"

# Cursor movement
cursor_up = "\033[1A"
cursor_down = "\033[1B"
cursor_forward = "\033[1C"
cursor_back = "\033[1D"
cursor_home = "\033[H"

# 256-color (foreground and background)
fg_256_1 = "\033[38;5;34m"
fg_256_2 = "\033[38;5;198m"
fg_256_3 = "\033[38;5;117m"
fg_256_4 = "\033[38;5;202m"
fg_256_5 = "\033[38;5;9m"
fg_256_6 = "\033[38;5;226m"
fg_256_7 = "\033[38;5;45m"
fg_256_8 = "\033[38;5;160m"
fg_256_9 = "\033[38;5;240m"
fg_256_10 = "\033[38;5;75m"

bg_256_1 = "\033[48;5;33m"
bg_256_2 = "\033[48;5;129m"
bg_256_3 = "\033[48;5;221m"
bg_256_4 = "\033[48;5;196m"
bg_256_5 = "\033[48;5;15m"
bg_256_6 = "\033[48;5;100m"
bg_256_7 = "\033[48;5;5m"
bg_256_8 = "\033[48;5;240m"
bg_256_9 = "\033[48;5;28m"
bg_256_10 = "\033[48;5;93m"

# True color (RGB foreground and background)
fg_rgb_1 = "\033[38;2;12;200;150m"
fg_rgb_2 = "\033[38;2;255;180;60m"
fg_rgb_3 = "\033[38;2;90;20;240m"
fg_rgb_4 = "\033[38;2;10;175;75m"
fg_rgb_5 = "\033[38;2;200;40;180m"
fg_rgb_6 = "\033[38;2;70;255;10m"
fg_rgb_7 = "\033[38;2;250;120;120m"
fg_rgb_8 = "\033[38;2;30;200;255m"
fg_rgb_9 = "\033[38;2;220;220;90m"
fg_rgb_10 = "\033[38;2;180;80;20m"

bg_rgb_1 = "\033[48;2;220;40;70m"
bg_rgb_2 = "\033[48;2;15;120;255m"
bg_rgb_3 = "\033[48;2;80;200;120m"
bg_rgb_4 = "\033[48;2;160;90;240m"
bg_rgb_5 = "\033[48;2;255;220;180m"
bg_rgb_6 = "\033[48;2;100;40;20m"
bg_rgb_7 = "\033[48;2;250;150;10m"
bg_rgb_8 = "\033[48;2;30;255;170m"
bg_rgb_9 = "\033[48;2;170;170;210m"
bg_rgb_10 = "\033[48;2;200;200;30m"

# Cursor position samples (x, y values)
cursor_pos_1 = "\033[3;5H"
cursor_pos_2 = "\033[10;15H"
cursor_pos_3 = "\033[40;20H"
cursor_pos_4 = "\033[25;60H"
cursor_pos_5 = "\033[5;30H"
cursor_pos_6 = "\033[12;12H"
cursor_pos_7 = "\033[70;10H"
cursor_pos_8 = "\033[15;45H"
cursor_pos_9 = "\033[1;80H"
cursor_pos_10 = "\033[50;25H"

# Erasing
erase_line = "\033[2K"
erase_to_end = "\033[K"
erase_screen = "\033[2J"
erase_screen_from_cursor = "\033[J"

# Saving and restoring cursor
save_cursor = "\033[s"
restore_cursor = "\033[u"

# Alternative screen buffer
alt_screen_on = "\033[?1049h"
alt_screen_off = "\033[?1049l"

for name in locals():
val = "]"+locals()[ name ]+"["
out = strip_ansi_sequences( val )
print( f"{name+":":<25} output: {out!r}, input: {val!r}" )


--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.unix.shell

On 9 Sep 2025 12:08:44 GMT, Stefan Ram wrote:

def strip_c1_controls(text):
"""Strip C1 control sequences: ESC followed by one char in range @-_ (0x40-0x5F)."""
pattern = re.compile(r'\x1B[@-_]')
return pattern.sub('', text)

C1 controls are actually in the range 0x80-ox9F.
--- Synchronet 3.21a-Linux NewsLink 1.2