Webhook add a payload wrapper with more information

[royberris]

With the introduction of the Webhook feature in Umbraco 13 I am missing some context when receiving webhooks.

1. It is impossible to know if the received item is media or content before deserializing the body
2. Information about the type of event is presented in the header, I would expect it in the body
3. Knowing the content or media type before deserializing the request would be nice so we can work with generic type when for example deserializing a generic content published webhook to it's desired type.
4. The datetime of the webhook trigger is not present in the webhook body or header. This might be crucial when processing webhooks asynchronously on the receiving application (using polling, which might be 6 hours later)
   a. The content does have the content published datetime, but that is not the (original) webhook sent datetime
   b. The media does not have a date at all, which will benefit from a (original) webhook sent datetime
5. Additional information like the retry count is available in the logs, but not in the webhook itself.

While the configuration of the webhook itself will present context in most places (like the endpoint might give context on what it should receive). There is no support for a generic endpoint that will place things on a queue to be processed asynchronously on the receiving side. Which is common in a event based architecture.

I'd like to introduce a payload wrapper which will be generic for any webhook that is called. It will have generic information about the webhook instead of only the payload itself.

public class PayloadWrapper
{
  public string Event { get; set; } // The event that was triggered, e.g. Umbraco.ContentPublished. This is now in the header which is not wrong but kind of a design flaw in my example. It shows the use case for a wrapper in the first place.

  public string Category { get; set; } // Content, or Media, or Custom?

  public string? ContentType { get; set; } // The type of the payload content, which we can use to serialize in the generic types, probably optional when working with custom events?

  public int RetryCount { get; set; } // Might be important for the receiving application (for example it is double)

  public DateTime DateTime { get; set; } // When the webhook was originally sent

  public object? Payload { get; set; } // The payload as it is now
}

Above is an example and more data might need to be added, or naming should be changed. But I believe it gives an impression.

I see that the implementation of the webhook right now is based on the Content Delivery API. But when working with webhooks I believe it requires some more context for the receiving party because Umbraco is the one doing the request and not the other way around as it is with the Content Delivery API.

[kjac]

Hi @royberris,

Thanks for your input here 😄

While we agree with the principle in this feature request, we believe it would be a mistake to go about it with a wrapper object. This wouldn't solve the deserialization issue on the receiving and, only (perhaps) make it slightly smaller.

Another issue with the wrapper is that it very quickly turns into something core specific (i.e. ContentType). We want the webhooks to support any data, so we can't enforce an opinionated schema here.

Instead we suggest adding custom request headers to the webhook requests. We could then add default headers containing the triggering event, the retry count, maybe also the category and the request timestamp (as individual headers, of course). And at a later stage, each webhook implementation could supply its own custom headers too, i.e. adding the content type header for content related events.

Thoughts?

[royberris]

Hi,

Sounds good, still sending the relevant data and solving the problem. This could also manually be done but the core of my feature request was missing context in the webhook by default. Adding it in the body or headers works for me as long as it is consistent.

Thanks!

[kjac]

Thanks @royberris 😄

[kjac]

A little update is due here.

When executing a webhook we already append an Umb-Webhook-Event request header which contains the triggering event. So far, so good. We'll be looking into appending the retry count too 👍

As an example, the triggering event could be Umbraco.ContentPublish ... see more of the built-in ones here.

[ronaldbarendse]

The retry count is added in PR #15375, together with other improvements that allow you to configure the named HTTP client that's used, so you can add other default headers (although arguably not yet based on the event data itself).

The following headers are added by default:

• User-Agent - in the form of Umbraco-CMS/13.0.0, added as DefaultRequestHeaders of the named HTTP client (can be cleared in code if you want)
• Umb-Webhook-Event - the webhook event name, as Kenn mentioned above
• Umb-Webhook-RetryCount - starting at 0 and increasing for every attempt to send the payload (until the configured MaximumRetries is reached)

We could also add the date the webhook event was created, as the webhooks are sent in batches every 10 seconds and can be retried 5 times (by default). I'll add a quick (final, final @Zeegaan 😝) suggestion on the PR to add this!