[POC] netdata collector of thin-edge measurements

[didier-wenzek]

Proposed changes

Exploring the idea of a Netdata collector for tedge measurements.

Netdata provides out-of-the box many collector for device-level metrics (cpu, mem, network, disk, ...). Thin-edge can add application-specific insight by injecting netdata the measurements published by thin-edge devices.

Run it

A stream of thin-edge measurements:

$ tedge mqtt sub 'te/+/+/+/+/m/#`'
[te/device/main///m/environment] { "temperature": 1 , "time": 1709315843.322205791, "pressure": 11 }
[te/device/main///m/environment] { "temperature": 2 , "time": 1709315844.330121504, "pressure": 22 }
[te/device/main///m/environment] { "temperature": 3 , "time": 1709315845.336501517, "pressure": 33 }

is transformed into a stream of netdata CHART | DIMENSION | SET events consumed by Netdata plugin.d:

$ target/debug/netdata-collector
CHART "tedge.main/environment" "tedge.main/environment" "tedge.main/environment" "units"
DIMENSION "temperature" "temperature"
DIMENSION "pressure" "pressure"
BEGIN "tedge.main/environment"
SET "temperature" = 1
SET "pressure" = 11
END
BEGIN "tedge.main/environment" 1012796
SET "temperature" = 2
SET "pressure" = 22
END
BEGIN "tedge.main/environment" 1011501
SET "temperature" = 3
SET "pressure" = 33
END

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s)
• I ran cargo fmt as mentioned in CODING_GUIDELINES
• I used cargo clippy as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
604	1	3	605	99.83	1h49m25.993514s

Failed Tests

Name	Message	⏱️ Duration	Suite
Updating entities from a child device	'' does not contain '"name":"Child 01"'	1.847 s	Tedge Http

[reubenmiller]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

[didier-wenzek]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

Yes, this won't impact build times as the netdata interface is really thin. However, I wonder if the proper place for this plugin shouldn't be under https://github.com/thin-edge/thin-edge.io_examples. And yes, more testing is required.

[reubenmiller]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

Yes, this won't impact build times as the netdata interface is really thin. However, I wonder if the proper place for this plugin shouldn't be under https://github.com/thin-edge/thin-edge.io_examples. And yes, more testing is required.

I'm reluctant to add it to thin-edge.io_examples as it makes it harder to improve. Maybe this could be the first rust based plugin which re-uses some of the thin-edge.io crates, if so then we can create a separate repository for this (under the thin-edge gh organization).