Skip to content

CP 49214: Upload and reposync from the bundle #5851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

BengangY
Copy link
Contributor

@BengangY BengangY commented Jul 18, 2024

Upload a bundle file and unpack it with module Tar_ext. After that, sync the bundle repository.

@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 46b9c6b to c4d58e9 Compare July 18, 2024 08:19
@BengangY BengangY marked this pull request as ready for review July 18, 2024 08:32
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from c4d58e9 to 710d8a8 Compare July 20, 2024 13:59
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch 4 times, most recently from 4e83d4b to fd0a796 Compare July 23, 2024 13:32
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from fd0a796 to 4ef1f95 Compare July 24, 2024 02:56
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 4ef1f95 to 9c46d5c Compare July 24, 2024 13:21
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 9c46d5c to 550c3d8 Compare July 25, 2024 07:05
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 550c3d8 to be17cdd Compare July 25, 2024 13:45
@BengangY
Copy link
Contributor Author

BengangY commented Jul 29, 2024

There is an existing issue. Uploading an incomplete file will lead the TCP connection hanging. VM_import has the same issue.
In tar_ext.ml, unpack_tar_file -> Tar_unix.Archive.with_next_file will parse the file header in the tar file. If the header is incomplete, this function will hang and not go ahead.
Need to find a method to add a timeout mechanism.
I have already created a CA ticket to investigate this issue.

@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from be17cdd to 60495b9 Compare July 29, 2024 09:26
BengangY added 3 commits July 29, 2024 20:54
Upload a bundle file and unpack it with module Tar_ext. After that,
sync the bundle repository.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Add an allowed_operations for sync bundle.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 60495b9 to 2609fa9 Compare July 29, 2024 13:02
1. Replace all `Client.Pool.get_all` and `Client.Pool.get_master` with
   self-defined function `cli_util.get_pool` and `cli_util.get_master` so that
   we can reduce the use of `List.hd`.
2. Remove some unused getting pool_master code in function `host_careful_op`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>
@BengangY BengangY force-pushed the private/bengangy/CP-49214 branch from 2609fa9 to c870b26 Compare July 29, 2024 13:06
@@ -51,13 +51,16 @@ let introduce ~__context ~name_label ~name_description ~binary_url ~source_url
let introduce_bundle ~__context ~name_label ~name_description =
Copy link
Contributor

@kc284 kc284 Jul 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was probably introduced in a different PR, but I've downloaded the generated SDK from the github actions of the current PR and the release info for repository.introduce_bundle is 1.301.0, which looks like a copy-paste error from repository.introduce.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a build issue. The reason is below:
The release info is based on a field lifecycle in the code. As the code is still in review and being tested, we don't know what value should it be, so the lifecycle is kept empty.
After the code is merged into the feature branch and the testing is finished, we will then merge the code into the master branch. At this time point, we will change the lifecycle to the actual release info such as 24.23.0.
For the koji build, it will execute make build first (if the lifecycle is empty, it will auto-fill a temporary value such as 24.21.0-next into lifecycle), and then execute make sdk to generate the SDK.
But for the PR build, make build and make sdk are executed concurrently, so the lifecycle is still empty when the SDK is generated, this leads the release info to be an old previous version number.

The koji-built SDK doesn't have this issue, you can download from here which is built based on this PR this morning.
https://koji.eng.citrite.net/kojifiles/repos/pb-t_bengangy-non-cdn/231372/x86_64/toplink/packages/xapi/24.21.0.17.gc870b266b/1.2.g8796faa.xs8/noarch/xapi-sdk-24.21.0.17.gc870b266b-1.2.g8796faa.xs8.noarch.rpm

@gangj
Copy link
Contributor

gangj commented Jul 30, 2024

There is an existing issue. Uploading an incomplete file will lead the TCP connection hanging. VM_import has the same issue. In tar_ext.ml, unpack_tar_file -> Tar_unix.Archive.with_next_file will parse the file header in the tar file. If the header is incomplete, this function will hang and not go ahead. Need to find a method to add a timeout mechanism. I have already created a CA ticket to investigate this issue.

As this issue only reproduce with a bad bundle file by an authorized user, and can be resolved by cancelling the bundle syncing from xe client, and as we are targeting for feature branch, I think we can merge it for now and have more investigation and discussion later.

@gangj gangj merged commit 5c68118 into xapi-project:feature/non-cdn-update Jul 30, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants