Skip to content

Avoid re-fetching binaries when they exist #492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 11, 2024

Conversation

vacantron
Copy link
Collaborator

@vacantron vacantron commented Sep 6, 2024

Now, we fetch the SHA1 first and verify the binaries. Only if the check fails, the download begins.

Copy link
Contributor

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improve git commit messages.

If you tend to trigger tests, specify "WIP:" in the subject.

@vacantron vacantron marked this pull request as draft September 6, 2024 07:51
@ChinYikMing
Copy link
Collaborator

ChinYikMing commented Sep 6, 2024

Rather than checking for the existence of prebuilt executable filenames, I prefer calculating and comparing their hash values with pre-calculated ones. For verifying the hash of a file or directory, see the verify function in "mk/external.mk".

define verify

@jserv jserv added this to the release-2024.1 milestone Sep 7, 2024
@vacantron vacantron changed the title Update artifact.mk Avoid re-fetching binaries when they exist Sep 10, 2024
@vacantron vacantron marked this pull request as ready for review September 10, 2024 13:29
Copy link
Collaborator

@ChinYikMing ChinYikMing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enhance the git commit message by providing more details, specifically explaining how the SHA1 value is handled.

@ChinYikMing
Copy link
Collaborator

ChinYikMing commented Sep 10, 2024

Is there any relationship between the 'sha1sums.csv' files in this link and this link and this commit?

If not, they can be removed from the rv32emu-prebuilt repo to keep it clean.

@vacantron
Copy link
Collaborator Author

Is there any relationship between the 'sha1sums.csv' files in this link and this link and this commit?

If not, they can be removed from the rv32emu-prebuilt repo to keep it clean.

rv32emu-prebuilt now only releases the tarball of the prebuilt binaries, and all other files in the repository will not be updated in the future.

@ChinYikMing
Copy link
Collaborator

Is there any relationship between the 'sha1sums.csv' files in this link and this link and this commit?
If not, they can be removed from the rv32emu-prebuilt repo to keep it clean.

rv32emu-prebuilt now only releases the tarball of the prebuilt binaries, and all other files in the repository will not be updated in the future.

I’m still confused about file suffixes related to CSV. Could you please provide more details?

@vacantron
Copy link
Collaborator Author

vacantron commented Sep 11, 2024

I’m still confused about file suffixes related to CSV. Could you please provide more details?

The contents of *.csv files are copied from the output of sha1sum and are changed to comma-separated manually (the released sha1sum-* are still whitespace separated).

We fetch the SHA1 first and verify the binaries. If any verification
fails, the returned value of the "verify" macro becomes non-zero.
Only if the verification fails, the download begins.

Close sysprog21#491
@ChinYikMing
Copy link
Collaborator

I’m still confused about file suffixes related to CSV. Could you please provide more details?

The contents of *.csv files are copied from the output of sha1sum and are changed to comma-separated manually (the released sha1sum-* are still whitespace separated).

So, are the *.csv files primarily used to make comparisons easier to view compared to the space-separated sha1sum-* files?

@vacantron
Copy link
Collaborator Author

vacantron commented Sep 11, 2024

So, are the *.csv files primarily used to make comparisons easier to view compared to the space-separated sha1sum-* files?

I think both of them are the same with awk command. The only difference is the separating character needs to be specified when processing CSV files.

Or, do you care about the sha1sum-* files without suffix might be confused with the prebuilt binaries?

@ChinYikMing
Copy link
Collaborator

ChinYikMing commented Sep 11, 2024

So, are the *.csv files primarily used to make comparisons easier to view compared to the space-separated sha1sum-* files?

I think both of them are the same with awk command. The only difference is the separating character needs to be specified when processing CSV files.

Or, do you care about the sha1sum-* files without suffix might be confused with the prebuilt binaries?

I was wondering, since we have the space-separated sha1sum-* files, would the *.csv files still be necessary?

@vacantron
Copy link
Collaborator Author

vacantron commented Sep 11, 2024

I was wondering, since we have the space-separated sha1sum-* files, would the *.csv files still be necessary?

No, but any file in rv32emu-prebuilt won't be updated since it is only for the purpose of releasing prebuilt binaries. We only fetch the files from the RELEASE page.

Or, we can simply clear the repository, but not in this pull request.

@ChinYikMing
Copy link
Collaborator

I was wondering, since we have the space-separated sha1sum-* files, would the *.csv files still be necessary?

No, but any file in rv32emu-prebuilt won't be updated since it is only for the purpose of releasing prebuilt binaries. We only fetch the files from the RELEASE page.

Or, we can simply clear the repository, but not in this pull request.

Agree. From my perspective, the *.csv files can be a bit confusing in terms of maintainability, especially since the useful ones are available on the RELEASE page, as you mentioned.

Copy link
Collaborator

@ChinYikMing ChinYikMing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, LGTM.

@jserv jserv merged commit d0a6060 into sysprog21:master Sep 11, 2024
8 checks passed
vestata pushed a commit to vestata/rv32emu that referenced this pull request Jan 24, 2025
Avoid re-fetching binaries when they exist
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants