Skip to content

Feature: Generate function and extension logs via Telemetry API receiver #1347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Sep 4, 2024
Merged

Feature: Generate function and extension logs via Telemetry API receiver #1347

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
bd5dd05
Added WithLogs and its handling
jerrytfleung May 24, 2024
63558a8
nits
jerrytfleung May 25, 2024
1ae4fdd
Added extensions
jerrytfleung May 25, 2024
0f74cbe
Fixed unit tests
jerrytfleung May 28, 2024
0c54292
Added unit test cases
jerrytfleung May 28, 2024
dd2e317
Added config (#26)
jerrytfleung Jun 20, 2024
13a99af
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Jun 20, 2024
4ff5876
Added severityTextToNumber function
jerrytfleung Jun 20, 2024
730cfc9
Corrected README.md
jerrytfleung Jun 27, 2024
999a7ce
Handled empty types array
jerrytfleung Jun 27, 2024
1190cb5
Added CRITICAL & ALL
jerrytfleung Jun 27, 2024
165cdda
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Jul 15, 2024
26a3c68
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Jul 17, 2024
16f92d1
Removed invalid test case
jerrytfleung Jul 17, 2024
2575871
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Aug 22, 2024
a297035
Fixed code after rebase
jerrytfleung Aug 22, 2024
2d33076
Updated README.md
jerrytfleung Aug 22, 2024
cdea000
Used time.RFC3339 format
jerrytfleung Aug 22, 2024
166628a
Applied review comments
jerrytfleung Aug 27, 2024
2e03ffe
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Aug 28, 2024
3b7cda5
Added WARNING, Updated test cases, Added String.ToUpper
jerrytfleung Aug 28, 2024
1bc05fb
Merge branch 'main' into feature/telemetryapireceiver-configuration
jerrytfleung Sep 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion collector/receiver/telemetryapireceiver/receiver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ func (r *telemetryAPIReceiver) createLogs(slice []event) (plog.Logs, error) {
}
}
if level, ok := record["level"].(string); ok {
logRecord.SetSeverityNumber(severityTextToNumber(level))
logRecord.SetSeverityNumber(severityTextToNumber(strings.ToUpper(level)))
logRecord.SetSeverityText(logRecord.SeverityNumber().String())
}
if requestId, ok := record["requestId"].(string); ok {
Expand Down Expand Up @@ -260,6 +260,7 @@ func severityTextToNumber(severityText string) plog.SeverityNumber {
"FATAL4": plog.SeverityNumberFatal4,
"CRITICAL": plog.SeverityNumberFatal,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding CRITICAL here. It is because Python logging library can log CRITICAL (50) and it is mapped to Fatal in opentelmetry-python https://opentelemetry-python.readthedocs.io/en/latest/_modules/opentelemetry/_logs/severity.html#std_to_otel

"ALL": plog.SeverityNumberTrace,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding ALL here. It is because Java log4j can log ALL and it is mapped to Trace in opentelemetry-java https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/a87245b842a03bc42af6b388bb837fe8884aadae/instrumentation/log4j/log4j-appender-2.17/library/src/main/java/io/opentelemetry/instrumentation/log4j/appender/v2_17/internal/LogEventMapper.java#L201

"WARNING": plog.SeverityNumberWarn,
Copy link
Contributor Author

@jerrytfleung jerrytfleung Aug 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding WARNING here. It is because Python logger can log WARNING level when the function record is in JSON format.

Example

{
    "timestamp": "2024-07-12T20:06:53Z",
    "level": "WARNING",
    "message": "30 warning {'filters': [], 'name': 'root', 'level': 0, 'parent': None, 'propagate': True, 'handlers': [<LambdaLoggerHandlerWithFrameType (NOTSET)>], 'disabled': False, '_cache': {10: True, 20: True, 30: True}}",
    "logger": "root",
    "requestId": "dcbdc55b-25bb-49dd-8273-64a96181ca2a",
    "warning": [
        "from-extra",
        30
    ]
}

}
if ans, ok := mapping[strings.ToUpper(severityText)]; ok {
return ans
Expand Down
144 changes: 34 additions & 110 deletions collector/receiver/telemetryapireceiver/receiver_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,117 +354,41 @@ func TestCreateLogs(t *testing.T) {
func TestSeverityTextToNumber(t *testing.T) {
t.Parallel()

testCases := []struct {
level string
number plog.SeverityNumber
}{
{
level: "TRACE",
number: plog.SeverityNumberTrace,
},
{
level: "TRACE2",
number: plog.SeverityNumberTrace2,
},
{
level: "TRACE3",
number: plog.SeverityNumberTrace3,
},
{
level: "TRACE4",
number: plog.SeverityNumberTrace4,
},
{
level: "DEBUG2",
number: plog.SeverityNumberDebug2,
},
{
level: "DEBUG3",
number: plog.SeverityNumberDebug3,
},
{
level: "DEBUG4",
number: plog.SeverityNumberDebug4,
},
{
level: "INFO",
number: plog.SeverityNumberInfo,
},
{
level: "INFO2",
number: plog.SeverityNumberInfo2,
},
{
level: "INFO3",
number: plog.SeverityNumberInfo3,
},
{
level: "INFO4",
number: plog.SeverityNumberInfo4,
},
{
level: "WARN",
number: plog.SeverityNumberWarn,
},
{
level: "WARN2",
number: plog.SeverityNumberWarn2,
},
{
level: "WARN3",
number: plog.SeverityNumberWarn3,
},
{
level: "WARN4",
number: plog.SeverityNumberWarn4,
},
{
level: "ERROR",
number: plog.SeverityNumberError,
},
{
level: "ERROR2",
number: plog.SeverityNumberError2,
},
{
level: "ERROR3",
number: plog.SeverityNumberError3,
},
{
level: "ERROR4",
number: plog.SeverityNumberError4,
},
{
level: "FATAL",
number: plog.SeverityNumberFatal,
},
{
level: "FATAL2",
number: plog.SeverityNumberFatal2,
},
{
level: "FATAL3",
number: plog.SeverityNumberFatal3,
},
{
level: "FATAL4",
number: plog.SeverityNumberFatal4,
},
{
level: "CRITICAL",
number: plog.SeverityNumberFatal,
},
{
level: "ALL",
number: plog.SeverityNumberTrace,
},
{
level: "UNKNOWN",
number: plog.SeverityNumberUnspecified,
},
goldenMapping := map[string]plog.SeverityNumber{
"TRACE": plog.SeverityNumberTrace,
"TRACE2": plog.SeverityNumberTrace2,
"TRACE3": plog.SeverityNumberTrace3,
"TRACE4": plog.SeverityNumberTrace4,
"DEBUG": plog.SeverityNumberDebug,
"DEBUG2": plog.SeverityNumberDebug2,
"DEBUG3": plog.SeverityNumberDebug3,
"DEBUG4": plog.SeverityNumberDebug4,
"INFO": plog.SeverityNumberInfo,
"INFO2": plog.SeverityNumberInfo2,
"INFO3": plog.SeverityNumberInfo3,
"INFO4": plog.SeverityNumberInfo4,
"WARN": plog.SeverityNumberWarn,
"WARN2": plog.SeverityNumberWarn2,
"WARN3": plog.SeverityNumberWarn3,
"WARN4": plog.SeverityNumberWarn4,
"ERROR": plog.SeverityNumberError,
"ERROR2": plog.SeverityNumberError2,
"ERROR3": plog.SeverityNumberError3,
"ERROR4": plog.SeverityNumberError4,
"FATAL": plog.SeverityNumberFatal,
"FATAL2": plog.SeverityNumberFatal2,
"FATAL3": plog.SeverityNumberFatal3,
"FATAL4": plog.SeverityNumberFatal4,
"CRITICAL": plog.SeverityNumberFatal,
"ALL": plog.SeverityNumberTrace,
"WARNING": plog.SeverityNumberWarn,
}
for level, number := range goldenMapping {
require.Equal(t, number, severityTextToNumber(level))
}
for _, tc := range testCases {
require.Equal(t, tc.number, severityTextToNumber(tc.level))

others := []string{"", "UNKNOWN", "other", "anything"}
for _, level := range others {
require.Equal(t, plog.SeverityNumberUnspecified, severityTextToNumber(level))
}
}
Loading