Skip to content

docker: Speed up startup of the materialized container #33100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 22, 2025

Conversation

def-
Copy link
Contributor

@def- def- commented Jul 21, 2025

Local run with bin/mzcompose --find testdrive down && bin/mzcompose --find testdrive run default. Before:

 ✔ Container testdrive-materialized-1          Healthy    61.3s

With this PR

 ✔ Container testdrive-materialized-1          Healthy    10.7s

Another example: bin/mzcompose --find balancerd down && bin/mzcompose --find balancerd run default. Before:

✔ Container balancerd-materialized-1   Healthy  39.6s

After:

✔ Container balancerd-materialized-1   Healthy   5.6s

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

@def- def- force-pushed the pr-speedup-materialized-container branch 10 times, most recently from fb4b0d6 to 4808827 Compare July 21, 2025 22:05
@def- def- requested a review from antiguru July 21, 2025 22:19
@def- def- marked this pull request as ready for review July 21, 2025 22:19
@def- def- requested review from ggevay and ptravers July 21, 2025 22:20
@antiguru antiguru requested a review from Copilot July 22, 2025 05:04
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes the startup time of the materialized Docker container by implementing a pre-warmed data archive approach and streamlining PostgreSQL initialization. The changes reduce container startup time from 61.3s to 10.7s by creating a pre-built data archive during image build and modifying the runtime initialization to use background processes.

  • Creates a pre-warmed data archive during Docker build to avoid cold startup overhead
  • Switches from pg_ctlcluster to direct postgres binary execution with background processes
  • Consolidates PostgreSQL schema creation into a single command

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/materialized/ci/Dockerfile Adds build-time container warm-up and data archiving
src/materialized/ci/entrypoint.sh Optimizes runtime startup with background processes and archive extraction
misc/images/materialized-base/Dockerfile Adds required dependencies and removes redundant PostgreSQL initialization

@def- def- force-pushed the pr-speedup-materialized-container branch 6 times, most recently from f7ae78c to b6809a8 Compare July 22, 2025 11:59
@def- def- force-pushed the pr-speedup-materialized-container branch from b6809a8 to 29b5e7a Compare July 22, 2025 11:59
@def- def- enabled auto-merge July 22, 2025 11:59
@def- def- merged commit cee85b6 into MaterializeInc:main Jul 22, 2025
116 checks passed
@def- def- deleted the pr-speedup-materialized-container branch July 22, 2025 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants