Memory Leak Causing Pod Shutdowns #115
Description
Error description:
Since introducing the session serialization, we are experiencing multiple pod kills in our environment.
The Vaadin application is running in production and the pods on which it runs are regularly reaching the memor limit of 3gb, causing the pods to be shut down. Sometimes they reach the limit in as little as 4 hours.
This only happens since we use the session serialization with the kubernetes kit. Before that, the memory usage would go down during the afternoon and evening, when we have less traffic.
It looks as though there is a memory leak somewhere, as the memory is not cleared properly.
We tried using a different garbage collector. This helped a bit, but did not solve the problem.
Expected behaviour:
The memory usage should not be permanently affected by the session serialization.
While it is clear that there will be more memory usage during the serialization itself, it should not cause lasting memory leaks.
Errors during serialization or deserialization should not cause memory leaks.
Details:
A comparison of the memory usage of our pods before and after the introduction of the session serialization (session serialization was introduced on the 3rd of March):
It is visible from the logs, that the frequency with which pods are killed has increased drastically:
The memory leaks seem to happen in "jumps".
This pod is soon going to be killed after one more memory leak event. Logically there are more memory leak events occuring during times of higher usage (in our case, during working hours).
The new garbage collector does not solve the problem:
