com.google.gson.stream.MalformedJsonException: Invalid escape sequence at line

[ProgramCareer]

2021-09-28 09:58:28,087 - org.apache.skywalking.oap.server.analyzer.agent.kafka.provider.handler.LogHandler - 83 [KafkaConsumer-1-thread-2] ERROR [] - com.google.gson.stream.MalformedJsonException: Invalid escape sequence at line 1 column 685 path $.body.text.text
com.google.protobuf.InvalidProtocolBufferException: com.google.gson.stream.MalformedJsonException: Invalid escape sequence at line 1 column 685 path $.body.text.text
at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:1347) ~[protobuf-java-util-3.12.4.jar:?]
at com.google.protobuf.util.JsonFormat$Parser.merge(JsonFormat.java:477) ~[protobuf-java-util-3.12.4.jar:?]
at org.apache.skywalking.oap.server.library.util.ProtoBufJsonUtils.fromJSON(ProtoBufJsonUtils.java:54) ~[library-util-8.6.0.jar:8.6.0]

Source code

private static String toString(byte[] b, int off, int len) {
StringBuilder result = new StringBuilder();
if (b == null) {
return result.toString();
} else if (off >= b.length) {
return result.toString();
} else {
if (off + len > b.length) {
len = b.length - off;
}
for(int i = off; i < off + len; ++i) {
int ch = b[i] & 255;
if (ch >= 32 && ch <= 126 && ch != 92) {
result.append((char)ch);
} else {
//The following processing results in an error**
result.append("\x");
result.append(HEX_CHARS_UPPER[ch / 16]);
result.append(HEX_CHARS_UPPER[ch % 16]);
}
}
return result.toString();
}
}

[wu-sheng]

You are using newer agent to send data old backend? Read our compatibility document.

[ProgramCareer]

It is a problem in the encoding conversion of a class named Bytes under the Kafka package. It converts Chinese into single byte hexadecimal (for example: \ x), resulting in protobuf directly reporting com.google.gson.stream.malformedjsonexception: invalid escape sequence at line when it encounters a backslash in parsing JSON string

[wu-sheng]

You could take a look at Kafka fetcher at OAP side. Ad try to provide a fix according to your data.

[ProgramCareer]

Will this coding bug be fixed in a recent version?

[wu-sheng]

As we mentioned, most core contributors are working for US company, we don't have time or experience to fix Chinese char encoding issue.
Community needs everyone to contribute their own fix.

[ProgramCareer]

OK,Then I'll try to solve it，Thank you for your reply