[AN-557] Include VM initialization time in cost calculation for Batch jobs #7744
Conversation
| val endedRegex = ".*RUNNING to.*".r // can be SUCCEEDED or FAILED | ||
| // on Batch, when job transitions to SCHEDULED state it indicates that the VM is being initialized. Users are billed for this | ||
| // startup time. Hence, the 'vmStartTime' corresponds to when the job enters the SCHEDULED state. | ||
| val startedRegex = ".*QUEUED to SCHEDULED.*".r |
There was a problem hiding this comment.
Any chance we might be transitioning to scheduled from a state other than QUEUED?
There was a problem hiding this comment.
My understanding from conversation with Batch team was that SCHEDULED only happens after QUEUED.
It seems the only other state that can exist before QUEUED is STATE_UNSPECIFIED but I am not sure if job can go from STATE_UNSPECIFIED -> SCHEDULED 🤔
https://cloud.google.com/batch/docs/reference/rest/v1/projects.locations.jobs#State
There was a problem hiding this comment.
Do we want the start regex to be ".*to SCHEDULED.*" to be safe?
There was a problem hiding this comment.
That does seem a little safer, but I'm good either way based on your previous comment.
There was a problem hiding this comment.
I will update the regex just to be safe. The important part here is reaching the SCHEDULED state which would be achieved by ".*to SCHEDULED.*" as well.
| @@ -238,13 +251,44 @@ class BatchRequestExecutorSpec | |||
| result.status match { | |||
| case RunStatus.Failed(_, events, _) => | |||
| val eventNames = events.map(_.name) | |||
| val eventTimes = events.map(_.offsetDateTime.toString) | |||
| println(eventNames) | |||
There was a problem hiding this comment.
I'm guessing this is a debug line that should be removed?
There was a problem hiding this comment.
Yes. Maybe this is a leftover from previous PR. The test do succeed without the println.
There was a problem hiding this comment.
Oh, yes, right, I got confused and though it was being added here 🤦♀️
Description
Jira: https://broadworkbench.atlassian.net/browse/AN-557
This PR attempts to fix the underestimation of cost for Batch jobs. On Batch, users incur cost when VM is being initialized. Hence use corresponding events for
vmStartTimeandvmEndTimefor cost calculations.Events for a job run locally:
vmStartTimenow corresponds to SCHEDULED state:Release Notes Confirmation
CHANGELOG.mdCHANGELOG.mdin this PRCHANGELOG.mdbecause it doesn't impact community usersTerra Release Notes