Skip to content

Fix: Replace non-ASCII and non-printable characters with dots in logger #1218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fix: Replace non-ASCII and non-printable characters with dots in logger #1218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea10c7f
Fix: Replace non-ASCII and non-printable characters with dots in log …
pietern May 9, 2025
e23bee8
Fix: Preserve tab characters in log output
pietern May 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion logger/httplog/body_logger.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"fmt"
"sort"
"strings"
"unicode"
)

type bodyLogger struct {
Expand Down Expand Up @@ -122,7 +123,15 @@ func (b bodyLogger) redactedDump(prefix string, body []byte) string {
sb.WriteString("\n")
}
line = strings.Trim(line, "\r")
sb.WriteString(fmt.Sprintf("%s%s", prefix, line))
sb.WriteString(prefix)
Copy link

@denik denik May 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's maybe out of scope for this PR but I'd rather it decompressed the response (based on content-encoding header) and printed it.

If the reason I use debug log is to see requests & responses then that's what I'd be interested in.

for _, c := range line {
// Convert non-ASCII and non-printable characters to '.'
// But preserve tabs and spaces for indentation
if c > unicode.MaxASCII || (!unicode.IsPrint(c) && c != '\t') {
c = '.'
}
sb.WriteRune(c)
}
}
return sb.String()
}
Expand Down
5 changes: 5 additions & 0 deletions logger/httplog/body_logger_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ func TestBodyLoggerNotJson(t *testing.T) {
assert.Equal(t, dump, "<html>")
}

func TestBodyLoggerNotAscii(t *testing.T) {
dump := bodyLogger{debugTruncateBytes: 20}.redactedDump("", []byte("\x01 🚀 🚀"))
assert.Equal(t, dump, ". . .")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, why 🚀 is filtered out? It's printable.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it helps to see a lot of dots if request body is a large gzipped file? Compressing them all into single <binary> would be more useful for logs.

There could also be a case where users actually need to see the response (for debugging). In that case applying lossy transformation makes that impossible and thus worse than having binary in the output (which you could use if you really wanted to).

If decompressing is not possible / out of scope then printing escaped string could be next best option, e.g. we can adopt Python's approach:

>>> open('cli', 'rb').read(100)
b'\xcf\xfa\xed\xfe\x0c\x00\x00\x01\x00\x00\x00\x00\x02\x00\x00\x00\x12\x00\x00\x00\xd8\n\x00\x00\x04\x00 \x00\x00\x00\x00\x00\x19\x00\x00\x00H\x00\x00\x00__PAGEZERO\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

}

func TestBodyLoggerNonJSONNewline(t *testing.T) {
dump := bodyLogger{debugTruncateBytes: 100}.redactedDump("< ", []byte(`<html>
<body>
Expand Down
Loading