feat: check resource usage in full_kubeflow_integration_test and add section of resource utilization of each component in README

[milinddethe15]

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Describe the changes you have made, including any refactoring or feature additions.

with this PR I'm trying to run the kubeflow ecosytem on ubuntu-latest-16-cores runner (instead of using my local cluster) and checking the resources consumption of components by modifying the full_kubeflow_integration_test to just deploy the components (I removed the tests to get minimum usage of pods) and read the usage using metrics-server.
We aren't merging this PR.

• check resource usage in full_kubeflow_integration_test using metrics-server
• add section of requests/min. resource utilization of each component in README

📦 Dependencies

List any dependencies or related PRs (e.g., "Depends on #123").

🐛 Related Issues

Link any issues that are resolved or affected by this PR.

fixes #3083

✅ Contributor Checklist

• I have tested these changes with kustomize. See Installation Prerequisites.
• All commits are signed-off to satisfy the DCO check.

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[juliusvonkohout]

Please use always rebase in the future instead of the merge from master as you did.

[juliusvonkohout]

Probably its best to create a new fresh PR from scratch and not use merge from master in the future, because i think it messes up files and commit history

[milinddethe15]

Probably its best to create a new fresh PR from scratch and not use merge from master in the future, because i think it messes up files and commit history

I have fixed the commits in this PR itself. Now, I will add the components' resource usage table in README

[juliusvonkohout]

That looks good. Now we just need to generate the markdown table at the end with the maximum of actual usage and requests.

[milinddethe15]

@juliusvonkohout I am not sure how to structure the documentation but I gave a try.
PTAL: https://github.com/milinddethe15/kf-manifests/tree/patch-1?tab=readme-ov-file#resource-usage-by-components

[juliusvonkohout]

@juliusvonkohout I am not sure how to structure the documentation but I gave a try. PTAL: https://github.com/milinddethe15/kf-manifests/tree/patch-1?tab=readme-ov-file#resource-usage-by-components

Looks good, In the end you need to automatically first calculate maximum of actual usage and requests for cpu/memory/storage per pod. So a triplet per pod: (1.maximum(cpu usage,cpu request), 2. maximum(memory usage, memory request), 3. size in GB of mounted PVCs)

After having done this you can aggregate/summarize pods by components via the labels (pipelines, katib, istio etc) and then print this as markdown table in the GHA itself.

Component	CPU	Memory	Storage (GB)
Katib
istio
...