Massive memory leak since 8.5.5 / leaking the entire stack frame and all local variables everywhere

[luk2302]

In #187 a change was implemented that included the current stack frame in an internal cache, therefore causing all local variables of the entire call stack to never be freed. Demo:

from dataclasses import dataclass
import marshmallow_dataclass
from marshmallow.schema import BaseSchema
from marshmallow_dataclass import _internal_class_schema


class Outer:
    def __del__(self):
        print("deleting outer...")


@dataclass(frozen=True)
class Sample:
    a: str
    def __del__(self):
        print("deleting sample...")


def inner_demo():
    schema = marshmallow_dataclass.class_schema(Sample, base_schema=BaseSchema)()
    data = schema.load({"a": "a string"})
    return data


def demo():
    d = inner_demo()
    o = Outer()


for i in range(10):
    demo()
    print(_internal_class_schema.cache_info())

print("done with demo, no object deleted so far")
_internal_class_schema.cache_clear()
print("cleared cache")

No __del__ is called on either Sample or Outer during the method invocations, only after the cache_clear can the local variables be garbage collected. If you run the demo with 8.4.4 or just replace _internal_class_schema(clazz, base_schema, clazz_frame) with _internal_class_schema(clazz, base_schema, None) in marshmallow_dataclass.class_schema. Additionally this also demonstrates that the caching does no longer work as there no cache hits at all anymore.

[LostInDarkMath]

Hi,
I came across this issue because our software has a memory leak in production. After days of troubleshooting and narrowing down the problem, I came across this ticket.

Are there any plans to fix this in the near future?
And how can it be that such a critical bug remains unfixed for 2 years?