Skip to content

Add proof runtime to GITHUB_STEP_SUMMARY #221

@hanno-becker

Description

@hanno-becker

run-cbmc-proofs.py supports adding a summary table to GITHUB_STEP_SUMMARY, which is convenient for Github CI's.

Currently, this table has only two columns: Proof name and proof status.

To help identify long-running proofs, it would be useful to log also the runtime of the proofs in that table, either unconditionally or via another parameter to run-cbmc-proofs.py.

Here is a sketch that adds a column with the cumulative runtime of all jobs run for a proof:

--- a/cbmc/proofs/lib/summarize.py
+++ b/cbmc/proofs/lib/summarize.py
@@ -84,7 +84,7 @@ def _get_status_and_proof_summaries(run_dict):
     The second sub-list maps each proof to its status.
     """
     count_statuses = {}
-    proofs = [["Proof", "Status"]]
+    proofs = [["Proof", "Status", "Duration (in s)"]]
     for proof_pipeline in run_dict["pipelines"]:
         status_pretty_name = proof_pipeline["status"].title().replace("_", " ")
         try:
@@ -93,7 +93,11 @@ def _get_status_and_proof_summaries(run_dict):
             count_statuses[status_pretty_name] = 1
         if proof_pipeline["name"] == "print_tool_versions":
             continue
-        proofs.append([proof_pipeline["name"], status_pretty_name])
+        duration = 0
+        for stage in proof_pipeline["ci_stages"]:
+            for job in stage["jobs"]:
+                duration += int(job["duration"])
+        proofs.append([proof_pipeline["name"], status_pretty_name, str(duration)])
     statuses = [["Status", "Count"]]
     for status, count in count_statuses.items():
         statuses.append([status, str(count)])

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions