Skip to content

Merge feature branch feature/easier-pool-join #6305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

gangj
Copy link
Contributor

@gangj gangj commented Feb 14, 2025

No description provided.

gangj and others added 27 commits October 23, 2024 10:18
…xapi-project#6049)

- add a new type of origin: "remote_pool"
- add a new API: "introduce_remote_pool" to init a remote_pool repository
- add a new field: "certificate" for a remote_pool repository
- for a remote_pool repository, binary_url will be reused to hold the
  base URL of binary packages in the local repository of the remote pool in
  https://<coordinator-ip>/repository format

Signed-off-by: Gang Ji <gang.ji@cloud.com>
The HTTP /repository handler is guarded by a mutex `exposing_pool_repo_mutex` currently.
Since now HTTP /repository is protected by `session_id` cookie, we can remove the mutex
from this handler and keep the handler enabled all the time.
Also, rename the mutex `exposing_pool_repo_mutex` to `pool_update_ops_mutex`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
The HTTP /repository handler is guarded by a mutex
`exposing_pool_repo_mutex` currently. Since now HTTP /repository is
protected by `session_id` cookie, we can remove the mutex and keep the
handler enabled all the time. Also, rename the mutex
`exposing_pool_repo_mutex` to `pool_update_ops_mutex`.
merge master to feature/easier-pool-join
Now xapi supports setting up rpc to hosts in the pool and appliances,
while for syncing updates from remote_pool type repository, we need to
set up rpc to remote coordinator with its certificate verfieid.

Add util Helpers.make_external_host_verified_rpc, which will set up a
secure connection to the external host(host outside the pool) with its
host certificate verified.

Signed-off-by: Gang Ji <gang.ji@cloud.com>
Now xapi supports setting up rpc to hosts in the pool and appliances,
while for syncing updates from remote_pool type repository, we need to
set up rpc to remote coordinator with its certificate verfieid.

Add util Helpers.make_external_host_verified_rpc, which will set up a
secure connection to the external host(host outside the pool) with its
host certificate verified.
Add an assertion to restrict `binary_url` of remote_pool repository to be in the
format of `https://<coordinator-ip>/repository/enabled`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
…roject#6089)

1. Add an assertion to restrict `binary_url` of remote_pool repository
to be in the format of `https://<coordinator-ip>/repository/enabled`.
2. Add UT for restrict/check `binary_url` of remote_pool repository.
Add handler for `/repository/enabled`. Replase `/enabled` with
the current enabled repository.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Adding process logic for `/repository/enabled`. If there is `/enabled`
after `/repository` in URI, then to find the current enabled repository.
Otherwise, keep the existing process logic.
1. `remote_pool` repo doesn't support periodic sync updates.
2. Periodic sync updates should be auto-disabled when calling `set_repositories`
   and `add_repository` for `remote_pool` repo.
3. If `remote_pool` repository is enabled, it should be the single one
   enabled.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
)

1. `remote_pool` repo doesn't support periodic sync updates.
2. Periodic sync updates should be auto-disabled when calling
`set_repositories` and `add_repository` for `remote_pool` repo.
4. Update UT.
Signed-off-by: Gang Ji <gang.ji@cloud.com>
Signed-off-by: Gang Ji <gang.ji@cloud.com>
When a remote_pool type repository, which points to the enabled
repository in the remote pool coordinator, is set as the enabled
repository of the pool, updates can be synced from it with API
pool.sync_updates.

The username password of the remote pool coordinator is required as
parameters for pool.sync_updates to login the remote pool.

And the remote pool coordinator's host server certificate needs to be
configured in the remote_pool repository, it will be used to verify the
remote end when sending out username passwords and syncing updates from
it.

A new yum/dnf plugin "xapitoken" is introduced to set xapi token as HTTP
cookie: "session_id" for each HTTP request which downloads files from the
remote_pool repository.

Signed-off-by: Gang Ji <gang.ji@cloud.com>
Will re-enable repo_gpgcheck by reverting this commit after CP-51429 is done.

Signed-off-by: Gang Ji <gang.ji@cloud.com>
CP-50787 CP-51347: Support pool.sync_updates from remote_pool repo

When a remote_pool type repository, which points to the enabled
repository in the remote pool coordinator, is set as the enabled
repository of the pool, updates can be synced from it with API
pool.sync_updates.

The username password of the remote pool coordinator is required as
parameters for pool.sync_updates to login the remote pool.

And the remote pool coordinator's host server certificate needs to be
configured in the remote_pool repository, it will be used to verify the
remote end when sending out username passwords and syncing updates from
it.

A new yum/dnf plugin "xapitoken" is introduced to set xapi token as HTTP
cookie: "session_id" for each HTTP request which downloads files from the
remote_pool repository.


CP-52245: Temp disable repo_gpgcheck when syncing from remote_pool repo

Will re-enable repo_gpgcheck by reverting this commit after CP-51429 is done.
…_pool repo"

This reverts commit c710e8f.

Signed-off-by: Gang Ji <gang.ji@cloud.com>
Solve conflict:
Stunnel.with_client_proxy -> Stunnel.with_client_proxy_systemd_service
Solve conflict:
Stunnel.with_client_proxy -> Stunnel.with_client_proxy_systemd_service
When enabling pool's repositories, if enabling bundle repo and remoe_pool
repositories at the same time, it returns error message:
`If the bundle repository or remote_pool repository is enabled, it should be
the only one enabled repository of the pool.
repo_types: bundle`
The `repo_types` is confusing and tedious as only these 2 types of repository
can meet this error. So remove the parameter `repo_types`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
When enabling pool's repositories, if enabling bundle repo and
remoe_pool
repositories at the same time, it returns error message:
`If the bundle repository or remote_pool repository is enabled, it
should be
the only one enabled repository of the pool.
repo_types: bundle`
The `repo_types` is confusing and tedious as only these 2 types of
repository
can meet this error. So remove the parameter `repo_types`.
@gangj gangj force-pushed the private/gangj/easier-pool-join_master-merge branch from 803839f to 1d5cc18 Compare February 14, 2025 11:03
@gangj
Copy link
Contributor Author

gangj commented Feb 14, 2025

$ git show 1d5cc1876d6
commit 1d5cc1876d65b114444a26388d68e74199d61476 (HEAD -> private/gangj/easier-pool-join_master-merge, my_gh/private/gangj/easier-pool-join_master-merge)
Merge: f4f00b3971 753a667332
Author: Gang Ji <gang.ji@cloud.com>
Date:   Fri Feb 14 18:52:58 2025 +0800

    Merge branch 'feature/easier-pool-join' into master

    Signed-off-by: Gang Ji <gang.ji@cloud.com>

diff --cc ocaml/idl/schematest.ml
index dcb0722a8c,8f87550cc0..d17b6cf488
--- a/ocaml/idl/schematest.ml
+++ b/ocaml/idl/schematest.ml
@@@ -3,7 -3,7 +3,7 @@@ let hash x = Digest.string x |> Digest.
  (* BEWARE: if this changes, check that schema has been bumped accordingly in
     ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *)

- let last_known_schema_hash = "6f6230f87a92572b68ebd742196ffd0e"
 -let last_known_schema_hash = "ffceac5e586329de3267b9bb958524a7"
++let last_known_schema_hash = "05ac9223f9c17b07b12e328d5dc3db52"

  let current_schema_hash : string =
    let open Datamodel_types in

@minglumlu
Copy link
Member

Need change in datamodel_lifecycle.ml

param_type= String
; param_name= "password"
; param_doc= "The password of the remote pool"
; param_release= numbered_release "24.39.0-next"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be 25.6.0-next

param_type= String
; param_name= "username"
; param_doc= "The username of the remote pool"
; param_release= numbered_release "24.39.0-next"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be 25.6.0-next

Signed-off-by: Gang Ji <gang.ji@cloud.com>
@gangj gangj force-pushed the private/gangj/easier-pool-join_master-merge branch from 6be6c7e to 5edbf0a Compare February 17, 2025 14:50
@gangj gangj added this pull request to the merge queue Feb 17, 2025
Merged via the queue into xapi-project:master with commit 4338494 Feb 17, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants