Provide configurable memory limit for Conn tracker buffer usage #2196
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Provide configurable memory limit for Conn tracker buffer usage
Some memory conscious users want to prevent PEM's from having unbounded memory use. The socket tracer's conn tracker component allows unbounded memory growth due to storing an uncapped amount of socket data. From profiling the PEM's heap use, this memory growth is a significant contributor to the PEM's memory footprint.
By leveraging the existing table store limit and this new configuration option, these memory conscious users run Pixie with a smaller and more predictable steady state memory usage.
Relevant Issues: N/A
Type of change: /kind feature
Test Plan: Ran perf_tool and k9s based load tests to verify the following:
Changelog Message: PEM's can now limit the total memory used by Pixie's socket tracer buffers through the
--total_conn_tracker_mem_usage
command line flag or thePX_TOTAL_CONN_TRACKER_MEM_USAGE
env var. This allows memory conscious users to have more predictable memory use when paired withPL_TABLE_STORE_TABLE_SIZE_LIMIT
.