Feedback on Background Jobs

[andrewmckaskill]

Hi all,

I had a discussion with @nul800sebastiaan at the Hackathon last November around improving the visibility and usability of the background services in Umbraco. We broadly agreed that the following would be a great set of goals to start with:

• update existing background jobs so they could be used with other engines (eg. Hangfire, Quartz.net)
• make it easier for people to add their own background jobs
• improve visibility of background job success/failure with a dashboard

I've spent some time looking at the hosted services now and come up with the following as a set of steps to start with (hopefully broken down enough that each can turn into a story by itself):

• Refactor existing jobs (DRAFT PR: Refactor hostedServices into background jobs #14291)

  • Extract a new common interface for background jobs.
  • Change existing services to use interface instead of being tied directly to IHostedService
  • Deprecate existing services

• Create simple dashboard

  • Dashboard with:
    • job name (or class),
    • last run datetime,
    • last run success / failure status,
    • last SUCCESSFUL run datetime
    • failure count
  • MVP version could just keep data in memory, but persisting to DB / some other store would be nicer. Then you could also aggregate background jobs from all servers rather than just the current one)
  • Stretch version could have a "run now" button too

• Create alternate implementation engines for Quartz and Hangfire ( these could start as packages if need be )

I'd also like to investigate whether or not some of these jobs could be farmed out to one/multiple external process as well, like a functions app or a windows service/daemon (for on-prem installations) - but that's a separate discussion in it's own right.

I'm aware that lots of our community have implemented background services of various kinds in both umbraco projects as well as standalone projects, so please fire ideas, questions, and comments at me 😄

[skttl]

Related PR #11338

[andrewmckaskill]

Cool! I think that should be easy to add into my PR as well. The advantage of splitting out the background jobs into a different interface means that the event publisher only needs to go in the "engine" class now - rather than having to change the constructors of all of the jobs.

[skttl]

Yes, I also think you are on the right track. My PR was just to enable me doing eg. a dashboard with stats.

[andrewmckaskill]

Do you still think there is benefit in publishing notifications separately from having a dashboard? Or would a dashboard be sufficient by itself?

[skttl]

A dashboard is enough for me

[c9mb]

@andrewmckaskill - Hah! - yes, indeed. I recently had to create my own background-tasks scheduler (admittedly for an older version of Umbraco) so I could launch/schedule some custom tasks for my clients, and constantly found myself wondering why this wasn't part of the standard services.

If I was writing this myself, I'd certainly be looking to provide a notification extensions, so that failure (and success) notifications could be emailed from appropriate points in the custom task.

I have one task at the moment, which needs to pull data from an external cloud-based accounting system, and push it to an MS-Dataverse table - so that employees can run business-critical PowerApps in the mornings. Both external services require a heap of OAuth gymnastics on top of their API complexities, and so there are multiple potential failure points. Knowing that the data may not be current was important and so required a bit of extra code that I would have made into a common service except for the age of the platform.

[andrewmckaskill]

This is tricky. I'm trying to work out how much to add to the basic implementation that comes out of the box vs duplicating the massive feature set provided by something like hangfire, especially if we can make a nice integration between them.

I'm not familiar enough with triggering emails based on custom notifications; is publishing the notification event like in the PR from @skttl enough that it can be hooked to easily, or does it need more work? If it just needs events publishing then I can merge the changes in easily enough.

[c9mb]

Yes, I was speaking in broad terms of what I would have needed to do, but I suspect that in my case sending an email is simply a custom response to a notification event.

[Digipex]

Love the dashboard idea. and a run now option would be handy too.

One QoL I'd add would be to set the task to run at a specific time and/or day