Jobs retention

[thoven87]

• Adding job retention configuration
• Make job priority static vars instead of static func

[adam-fowler]

A couple of comments

[adam-fowler]

Sorry I'm wondering if we do this another way. Basically the retention policy is retain or not, without a timeout. Then we add a function for users to call.

func purgeRetainedJobsEarlierThan(cancelled: Date, finished: Date, failed: Date) {

Actually you could just update the cleanup function.

Then users could add their own scheduled job. We could even write the job for them

struct PurgeOldJobs: JobParameters {
    let jobQueue: JobQueue<PostgresJobQueue>
}

jobQueue.registerJob(parameters: PurgeOldJobs.self) { parameters,_ in
    parameters.jobQueue.cleanUp(
        cancelledJobs: .remove(olderThan: .now - 4*24*60*60),
        finishedJobs: .remove(olderThan: .now - 7*24*60*60)
    )
}

jobSchedule = JobSchedule(job: PurgeOldJobs.self, .onMinutes(0,10,20,30,40,50))

I know this ends up being more work for the user, but it makes it a lot more configurable

[thoven87]

Sorry I'm wondering if we do this another way. Basically the retention policy is retain or not, without a timeout. Then we add a function for users to call.

func purgeRetainedJobsEarlierThan(cancelled: Date, finished: Date, failed: Date) {

Actually you could just update the cleanup function.

Then users could add their own scheduled job. We could even write the job for them

struct PurgeOldJobs: JobParameters {
    let jobQueue: JobQueue<PostgresJobQueue>
}

jobQueue.registerJob(parameters: PurgeOldJobs.self) { parameters,_ in
    parameters.jobQueue.cleanUp(
        cancelledJobs: .remove(olderThan: .now - 4*24*60*60),
        finishedJobs: .remove(olderThan: .now - 7*24*60*60)
    )
}

jobSchedule = JobSchedule(job: PurgeOldJobs.self, .onMinutes(0,10,20,30,40,50))

I know this ends up being more work for the user, but it makes it a lot more configurable

I think this works better and also maybe introduce a job rescue (orphan jobs :))?

[adam-fowler]

Any reason you want to keep this separate from cleanup function. Also making the retain times parameters of the function instead of static data might make it flexible.

[adam-fowler]

Given we are expecting people to set up a Job to call this, why don't we provide them with one.

[thoven87]

Any reason you want to keep this separate from cleanup function. Also making the retain times parameters of the function instead of static data might make it flexible.

The notification from yesterday was because I merged the main branch into this branch. I have a few meetings today and should have the updated logic in by this weekend.

[thoven87]

Sorry I'm wondering if we do this another way. Basically the retention policy is retain or not, without a timeout. Then we add a function for users to call.

func purgeRetainedJobsEarlierThan(cancelled: Date, finished: Date, failed: Date) {

Actually you could just update the cleanup function.
Then users could add their own scheduled job. We could even write the job for them

struct PurgeOldJobs: JobParameters {
    let jobQueue: JobQueue<PostgresJobQueue>
}

jobQueue.registerJob(parameters: PurgeOldJobs.self) { parameters,_ in
    parameters.jobQueue.cleanUp(
        cancelledJobs: .remove(olderThan: .now - 4*24*60*60),
        finishedJobs: .remove(olderThan: .now - 7*24*60*60)
    )
}

jobSchedule = JobSchedule(job: PurgeOldJobs.self, .onMinutes(0,10,20,30,40,50))

I know this ends up being more work for the user, but it makes it a lot more configurable

I think this works better and also maybe introduce a job rescue (orphan jobs :))?

I have thought of this implementation for a while, implementing as suggested will require JobQueue to conform to Codable. Is it a good idea to do that?