-
Notifications
You must be signed in to change notification settings - Fork 94
Description
We want to add initial support for the client/server architecture split that the remote execution has as part of the Astro CLI's local project setup.
So, as part of the astro dev init
command, we would want to introduce a --remote-execution-enabled
boolean flag (default set to false
).
When that flag is passed, the users would be prompted to enter the private registry hosting the client images for their Astro deployment. The private registry would be stored as part of the configuration in the local project. Note, we already have a notion of storing config for a local project, which looks like .astro/config.yaml
in the filesystem, and the interaction is handled by the config
package in the codebase. Storing it in config should allow users to modify the private registry endpoint down the line via the astro config set <config-name> <config-value>
command.
$ astro dev init --remote-execution-enabled
Enter the remote Docker repository for the client image (leave blank if not known
but you will not be able to use the Astro CLI to deploy the image until configured)
Remote client image repository endpoint (e.g. quay.io/acme/my-deployment-image): quay.io/astronomer/ian-remote-eg
If you want to modify the remote repository down the line, use the `astro config set <config-name> <config-value>` command while being inside the local project
Note that we should expect the users to enter the image endpoint for the remote repository and not just the repository hostname, as not all registries support nested structure, and so it would not work if the CLI tries to add the image name on its own, which cannot be controlled by the user.
The dev init command with the additional flag should generate the following file structure for the local project. Note the additional client files
.astro/
config.yaml
airflow_settings.yaml
dags/
Dockerfile
**Dockerfile.client**
include/
packages.txt
**packages-client.txt**
plugins/
README.md
requirements.txt
**requirements-client.txt**
tests/
The client Dockerfile should be initialized with the latest base runtime version of the client image or the runtime version passed in the input flag. We would have to think about a way to query available client images in the CLI. We could think about adding them to updates.astronomer.io
FROM images.astronomer.cloud/baseimages/astro-remote-execution-agent:3.0-10-python....-base