-
-
Notifications
You must be signed in to change notification settings - Fork 675
feat: Optimize metadata-storage build with HashMap caching for O(1) lookups #1779
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
@MichalLytek Thanks for taking a look at this, tests should be fixed. The root issue was a lack of resetting the new cache state in the global metadata-storage instance, and I also ran into some weirdness of needing to reset the state in |
@MichalLytek Bumping this, would love to get it merged 😄 |
I promise I will try to find some time to review it 😉 |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1779 +/- ##
==========================================
+ Coverage 95.50% 95.59% +0.08%
==========================================
Files 113 114 +1
Lines 1847 1929 +82
Branches 364 366 +2
==========================================
+ Hits 1764 1844 +80
- Misses 83 85 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Hey @MichalLytek, bumping this again! Let me know if you get a chance to take a look. Thanks in advance. |
Performance Optimization: Map-based Caches for Metadata Class Building
Problem
Our schema generation process was becoming exponentially slower as our schema grew, taking over a minute to complete in some cases. With our large schema containing ClassMetadata arrays ranging from 3,000 to 15,000 objects, the repeated
.find()
and.filter()
operations within forEach loops were creating significant performance bottlenecks.Solution
Implemented map-based "caches" during the metadata class build process to replace inefficient array operations. These caches are added as new variables on the MetadataStorage class, while preserving list-based attributes where required by existing components.
Impact
This optimization has dramatically improved performance as demonstrated by the following benchmarks:
before (using O(n) lookups)
after (using O(1) lookups)
Implementation Notes
.find()
and.filter()
operations within loop contexts