Can skywalking work with logs collected from other log system rather than from agent?

[kyle-oceanbase]

In my application system, I have already deployed a log storage which can be accessed with sql compatible with mysql or postgresql, all my logs are collected to such a storage with fluentbit. Now I'm trying to use skywalking as a opentracing system, but I do not want to collect redundant logs to skywalking. I want to know if skywalking can work with logs from other log system or storage

[wu-sheng]

It depends on what you want to do.

SkyWalking could accept logs from Fluentbit via

1. OTEL log format. https://skywalking.apache.org/docs/main/latest/en/setup/backend/log-otlp/
2. Logs collected by Filebeat, Fluentd and FluentBit via SkyWalking native format. https://skywalking.apache.org/docs/main/latest/en/setup/backend/filelog-native/

Both of ways require extra Log analysis scripts written in Groovy engine to format logs.

• https://skywalking.apache.org/docs/main/latest/en/setup/backend/log-analyzer/
• LAL script, https://skywalking.apache.org/docs/main/latest/en/concepts-and-designs/lal/

If you want SkyWalking OAP server reads logs from your own build system, you have to change the source codes. We can't access private system or use private APIs

[wu-sheng]

There is no magic we could read data from another place. SQL is just a skeleton; it doesn't mean it could work automatically.

We supported MySQL and PG for a long time, but your level compatibility is unknown to me.