Trace modification and generation

[drmorr0]

Right now the only way of getting a trace file is by capturing something from an existing cluster. I have ambitions to make this much more flexible in the future, but there's a lot of ways this could look. At a high level, these are the kinds of use cases I want to support:

1. Randomized generation of events (based on what probability distribution? How do we know these "random" traces are representative of "reality"?)
2. Amplification of existing traces (i.e., we run 100 deployments today, what needs to change when we scale up to 1000, or 10000?)
3. Perturbation of existing traces (i.e., this is a real trace we saw, generate me another 50 traces that "look like" this one)
4. Comparison of workloads (i.e., I want to test/compare Volcano and Yunikorn, let me rip out the higher-level tracked objects but keep all the pod lifecycle data the same)

There's possibly some other use cases we could support here too. Open questions I have:

1. Should these all be done by the same tool or different tools?
2. Should these be built into the skctl CLI, or do we need some other method for doing this?
3. How much human involvement do we need for trace generation, versus programmatic work?
4. Is the config for trace generation "just YAML" or should you have to write some code?