[feature] Use an ECR Pullthrough cache for Spark examples #823
Description
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
What is the outcome that you are trying to reach?
The Spark examples are pulling the same spark image but when running a large number of fresh instances I have seen some rate limiting and image pull issues. Using an ECR pull through cache seems like it would help reduce the issues while allowing the flexibility to use upstream images.
Describe the solution you would like
The spark operator blueprint could add an optional ECR pullthrough cache and rules to the vpc.tf file which then gives the details for the cache . We could then have commands in our docs to do the replacement of the images using like we do for the Bucket names.