Generic Payload Protocol

[thomas-watters-skydio]

The goal of the proposed message set is to support as many payloads as possible and support discovery.

[thomas-watters-skydio]

@hamishwillee @julianoes would love a good look over this one

[hamishwillee]

As per https://github.com/mavlink/rfcs/pull/28/files#r2373809642 probably a payload id to allow FC-attached payloads. Maybe remove the name and provide that as a metadata. Essentially, anything static could be metadata or in some other low rate message.

[thomas-watters-skydio]

i don't think a payload id would be needed at this level, i could be convinced to add a payload id just now sure how thats any different than a component id.

[thomas-watters-skydio]

I agree we should move the metadata/static data to another message that is on request, which is essentially everything except for the uptime_ms, error_flags, and the "extension fields"

[hamishwillee]

i don't think a payload id would be needed at this level, i could be convinced to add a payload id just now sure how thats any different than a component id.

For a MAVLink component the payload id is the same thing as a component id. So why do you need it, I hear you say?

• Firstly, this allows you to address a particular generic component in a mission. The mission item messages don't include a target component field like command protocol, so if you don't add the id then the mission executor can really only broadcast the mission item to all components, or send to anything that it can identify as the desired type.
• For FC connected components the component id will be the autopilot id. So in this case you would need to have an ID so it can be resolved. This is an irritation of the limitations of FCs - in no way good MAVLink design, but it is forced upon us in many cases.

[hamishwillee]

I would be tempted to suggest that this be split into static and dynamic info. Use the https://mavlink.io/en/services/gimbal_v2.html#discovery-of-gimbal-manager approach based on MAV_TYPE to discover the payloads.

Not entirely sure, because it mixes status and metadata. Also I'm still not sure on how functions and payloads map in the real world, so maybe this is sensible.

[thomas-watters-skydio]

i think that makes sense.

[hamishwillee]

pproach based on MAV_TYPE to discover the payloads.

Though of course this might not work because you don't know the MAV_TYPE of a generic payload necessarily. More thought required.

[hamishwillee]

@thomas-watters-skydio Thanks for this. Just had a proper read through.

1. Would it be possible to outline how this would replace "theoretically" the winch and/or gripper mechanism if you wanted to use this instead (say on a system that didn't support those). I'm having trouble understanding exactly what messages would be sent to represent these cases. (I.e. what payload, what function, what telemetry) and what a GCS could do with this compared to custom messages.
2. Do you have a flight stack/GCS you plan to implement this in? Would the integration be visible to us? How would you test it - with what payloads?
3. What kinds of timeframes are you looking at?
4. What kinds of payloads do you have in mind?

Historically we've considered that the requirements of payloads are quite different - consider a camera, illuminator, winch or a gripper - and hence that a useful integration requires custom messages.

That said their are common behaviours that we've started to identify as pain points in integrations. For example, a payload might have certain requirements on its use in a mission - such as indicating when it is ready to proceed to the next waypoint, or whether the vehicle needs to be able to land to use it.

Once I have a good understanding of how this works we'll put this before the flight stack vendors and GCS to gauge interest.

[thomas-watters-skydio]

@hamishwillee @julianoes i updated the RFC to address i think most of your concerns, i welcome your feedback

[thomas-watters-skydio]

@thomas-watters-skydio Thanks for this. Just had a proper read through.

1. Would it be possible to outline how this would replace "theoretically" the winch and/or gripper mechanism if you wanted to use this instead (say on a system that didn't support those). I'm having trouble understanding exactly what messages would be sent to represent these cases. (I.e. what payload, what function, what telemetry) and what a GCS could do with this compared to custom messages.
2. Do you have a flight stack/GCS you plan to implement this in? Would the integration be visible to us? How would you test it - with what payloads?
3. What kinds of timeframes are you looking at?
4. What kinds of payloads do you have in mind?

Historically we've considered that the requirements of payloads are quite different - consider a camera, illuminator, winch or a gripper - and hence that a useful integration requires custom messages.

That said their are common behaviours that we've started to identify as pain points in integrations. For example, a payload might have certain requirements on its use in a mission - such as indicating when it is ready to proceed to the next waypoint, or whether the vehicle needs to be able to land to use it.

Once I have a good understanding of how this works we'll put this before the flight stack vendors and GCS to gauge interest.

1. Would it be possible to outline how this would replace "theoretically" the winch and/or gripper mechanism if you wanted to use this instead (say on a system that didn't support those). I'm having trouble understanding exactly what messages would be sent to represent these cases. (I.e. what payload, what function, what telemetry) and what a GCS could do with this compared to custom messages.

I added an example of how we could replace the illuminator protocol

2. Do you have a flight stack/GCS you plan to implement this in? Would the integration be visible to us? How would you test it - with what payloads?

We will be implementing in our flight stack, we plan to do an end-to-end integration with a "anything" attachment, something with a sensor and an actuator. I could probably release the attachment side code relevant to the integration, but we will be integrating it with our first-party application

3. What kinds of timeframes are you looking at?

1-2 months

4. What kinds of payloads do you have in mind?

listed in the rfc

[hamishwillee]

This makes sense rather than a command, given the fields. But it does have two limitations. This means the feature can't be used in missions. It also has no ACK, and no target device or or component for addressing - you would need to add fields for these.

[thomas-watters-skydio]

we could technically wrap this in a CMD if we came up with a COMMAND_VAR, or something that had a variable length array. but i don't think that is needed as part of this RFC

[hamishwillee]

but i don't think that is needed as part of this RFC

I appreciate you have an internal implementation that does need this, but IMO support in missions is likely to be a dealbreaker for acceptance into common. So is having an ACK for acceptance of the command.
This maps quite easily conceptually as shown.

      <entry value="60000" name="MAV_CMD_GENERIC_PAYLOAD_FUNCTION" hasLocation="false" isDestination="false">
        <description>Command to control a generic payload function.</description>
        <param index="1" label="Payload id" minValue="1" maxValue="255">Component ID of the generic payload. Used in missions only.</param>
        <param index="2" label="Index" minValue="1">Index of this function on the generic payload.</param>
        <param index="3" label="Control mode" enum="GENERIC_PAYLOAD_CONTROL_MODE">Desired Control Mode.</param>
        <param index="4" label="Enable" enum="MAV_BOOL">1: Enable, 2: Disable.</param>
        <param index="6" label="Value_low" units = "ms" minValue="0">Lower 32 bits of value to set. For 32-bit types, this is the entire value. Little-endian.</param>
        <param index="7" label="Value_high" minValue="0">Upper 32 bits of value to set. Only used for 64-bit types. Little-endian.</param>
        <param index="8" label="Timeout" minValue="0">Timeout for momentary operation if control_mode is GENERIC_PAYLOAD_CONTROL_MODE_MOMENTARY.</param>
      </entry>

[thomas-watters-skydio]

I'm ok with adding a mav_cmd, but I think there is also a benefit to having a control message that can be streamed/sent more frequently than the command protocol allows, so I think both should be added/documented properly.

[hamishwillee]

Does this/should this match GENERIC_PAYLOAD_FUNCTION_DESCRIPTION? If so, can't we use that for this?

[thomas-watters-skydio]

i think i like them separate just for consistency, ie the FUNCTION set of messages and the TELEMETRY set.

to push them into the same message we'd have to add a field to indicate function/telem, and then reuses enable/update_rate field

[hamishwillee]

OK. This is likely to come up again. Flight stacks prefer fewer messages. Let's see.

[hamishwillee]

Thanks for the changes @thomas-watters-skydio - really helped me get my head around this.
Added a few notes, but happy to take this to the dev call.

Note

That's just the beginning bit though. We'll have to get broader buy-in from the various communities.
A flight stack might not have any interest in this, or might have strong opinions about need for support in missions etc.
For example, I know that the biggest problem with using generic actuators in missions is that you don't know what is being triggered.
Let's see.

YOu've clearly put a lot of thought into this.
Just a few "stream of conciousness" notes as I was reading through.

• Limitations

  • Can't be used in missions - because these use messages not commands.
  • Can only be directed at components. What that means is that an FC could have at most one attached "logical" generic component.
    You could of course have (say) ten components represented as functions with different names etc.
  • The control messages need to be targetted or everything might try run them. Using the target_system and target_component fields. May need an ACK. Even better if a command can be used.
  • Control messages aren't efficient "over the wire" - i.e. each action must be sent in a number of messages. That will generally be OK for commands, but might not be as favourable for streamed telemetry.
  • The UI will have to be constructed dynamically from the description messages. This would be better as component metadata json because you could have control over the UI and ordering - harder for MAVLink because messages aren't necessarily ordered as you might expect.

• The _DESCRIPTION metadata messages are a great. I don't think any of them need to be streamed - they can be requested perhaps on a protocol bit.
  You could choose also choose to do these as component metadata. I would consider adding an URL for that at least to allow easier translation. Also useful for extending the extra payload metadata without having to extend the messages.

[thomas-watters-skydio]

Thanks for the changes @thomas-watters-skydio - really helped me get my head around this. Added a few notes, but happy to take this to the dev call.

Note

That's just the beginning bit though. We'll have to get broader buy-in from the various communities. A flight stack might not have any interest in this, or might have strong opinions about need for support in missions etc. For example, I know that the biggest problem with using generic actuators in missions is that you don't know what is being triggered. Let's see.

YOu've clearly put a lot of thought into this. Just a few "stream of conciousness" notes as I was reading through.

• Limitations

  • Can't be used in missions - because these use messages not commands.
  • Can only be directed at components. What that means is that an FC could have at most one attached "logical" generic component.
    You could of course have (say) ten components represented as functions with different names etc.
  • The control messages need to be targetted or everything might try run them. Using the target_system and target_component fields. May need an ACK. Even better if a command can be used.
  • Control messages aren't efficient "over the wire" - i.e. each action must be sent in a number of messages. That will generally be OK for commands, but might not be as favourable for streamed telemetry.
  • The UI will have to be constructed dynamically from the description messages. This would be better as component metadata json because you could have control over the UI and ordering - harder for MAVLink because messages aren't necessarily ordered as you might expect.

• The _DESCRIPTION metadata messages are a great. I don't think any of them need to be streamed - they can be requested perhaps on a protocol bit.
  You could choose also choose to do these as component metadata. I would consider adding an URL for that at least to allow easier translation. Also useful for extending the extra payload metadata without having to extend the messages.

We definitely intend to incorporate this into our "flight stack" and distribute and ICD pointing to it. I do not think the goal of this RFC was to cover the mission use case, but moreso just generic control/feedback from an unknown attachment. It allows payload developers to come up with things we haven't thought of or discovered yet, along with displaying useful information in the UX/UI, although I wil reiterate this is not intended to replace a first-party integration. We just are in the position where we want payload developers to be free to develop whatever they might have in min.

• Can't be used in missions - because these use messages not commands.
  -- I'm ok with that for now
• Can only be directed at components. What that means is that an FC could have at most one attached "logical" generic component. You could of course have (say) ten components represented as functions with different names etc.
  -- The FC may have other way to address components, ie in our case we know what port its connected to and so internally we can enumerate them differently.
• The control messages need to be targetted or everything might try run them. Using the target_system and target_component fields. May need an ACK. Even better if a command can be used.
  -- Yeah when I was developing this message I was not considering that a GCS would necessarily be sending these messages. The ILLUMINATOR protocol for example is used in our system internally, we do not expose this service to the GCS. We could add a COMMAND_LONG for this protocol. MAV_CMD_FUNCTION_CONTROL, but it would be byte packing the data into the float params. I'm open to further thoughts here
• Control messages aren't efficient "over the wire" - i.e. each action must be sent in a number of messages. That will generally be OK for commands, but might not be as favourable for streamed telemetry.
  -- Indeed this is a compromise we made for telemetry. Unfortunately even if we chose to bundle telemetry messages in a BER -encoded array wrapped in MAVLink we are still significantly limited to the data that can be transmitted because of MAVLink's 256 message size limit.
• The UI will have to be constructed dynamically from the description messages. This would be better as component metadata json because you could have control over the UI and ordering - harder for MAVLink because messages aren't necessarily ordered as you might expect.
  -- I agree and i think both should be available. I think using this generic protocol will be a lower barrier to entry. With the metadata service as far as i would understand is the component and the ui layer would still have to agree on some common description of the functions/capabilities of the payloads. This would mean on-going integrations with the UI and payload developers

[hamishwillee]

@thomas-watters-skydio Thanks. So the main point in response to the above is that MAVLink is supposed to be a standard. I get that you have something that works for you, but if the broader community needs support in missions, then that needs to be considered. I can pretty much guarantee they will want feedback from control actions too - at which point commands become logical solution.

In missions when using a payload there is the need for indication that an action is finished so that the next waypoint can be triggered. Do you think the current signals do this sufficiently? I'm not certain.

We could add a COMMAND_LONG for this protocol. MAV_CMD_FUNCTION_CONTROL, but it would be byte packing the data into the float params. I'm open to further thoughts here

See #28 (comment)

Yes it would require byte packing, though I'd probably require it to be sent in a COMMAND_INT. Either way, yes special handling would be needed for types that are more than 64 bits, and the UI is going to have to know this command is different - you can't just shove numbers into the params and expect it to work.

It is perhaps important to note that even though things like params allow for 64 bits, no one actually uses anything other than 32 bits and floats (32-bit reals).

With respect to your comments about this being generic and not intended to be as solid as a custom implementation, I get that. We talked in the MAV call about this, and really were not sure how much interest there would be for flight stacks. We are pretty sure that there are plenty of people doing forks who would like this though - someone who wants to add a simple trigger with a proper UI.

Discussed this a bit in the mav call. It certainly addresses the issues we've had in the past with poor payload design - such as omitting feedback mechanisms from the gripper protocol.

[thomas-watters-skydio]

@hamishwillee Thanks for the feedback. I believe the reason I am proposing this is to come to a standardization that can be broadly adopted. Also our goal of exposing this on main fork and the whole RFC rather than tucking it away in a vendor message definition is to get the feedback from the community on what has worked / may work for the community.

With respect to your comments about this being generic and not intended to be as solid as a custom implementation, I get that. We talked in the MAV call about this, and really were not sure how much interest there would be for flight stacks. We are pretty sure that there are plenty of people doing forks who would like this though - someone who wants to add a simple trigger with a proper UI.

I think you would be surprised about how much demand there is for this. I think a point of comparison would be other vendor's Payload SDK's. We are choosing MAVLink as this interface so that payload/attachment vendors are not necessarily locked into a certain platform.

With regards to incorporating the FUNCTION_CONTROL into a MAV_CMD, i'm OK with this, though I'm not sure ACK is always necessary and that there may be some user cases for tight-loop control, which may not be possible with the command protocol, so I would like to propose keeping the FUNCTION_CONTROL but also adding the MAV_CMD.

[thomas-watters-skydio]

Oh i'll add the URI for metadata as an optional field to the GENERAL_PAYLOAD_DESCRIPTION message as well, forgot that bit.

[thomas-watters-skydio]

Oh i'll add the URI for metadata as an optional field to the GENERAL_PAYLOAD_DESCRIPTION message as well, forgot that bit.

@hamishwillee i was about to add this but is this not just what the COMPONENT_METADATA would have, i might be misunderstanding, i can add similar fields ie a crc32 and 100char uri field to the description message but why can't we just re-use the component_metadata message in that case?

[hamishwillee]

@thomas-watters-skydio

I think for now let's forget the URL - it could be added as an extension or to component metadata.

Your logic for having both a command and a message makes sense.

You're right - I might well be surprised about the demand. I think once there is an implementation in QGC and perhaps in PX4 or ArduPilot then this could well take off. It's just getting to that point.

In any case this is far enough along to get feedback - which I will do when I'm on this work again next week. Thanks!

[dakejahl]

I did read through this once but have refrained from commenting. I haven't read through it thoroughly enough to fully understand the details. But in general I agree with the motivation. We need a way to dynamically render UI in the ground station for payloads. I would love to see an example in QGC/PX4 to check it out. Ideally we could create something in SITL for it.

[hamishwillee]

How would I provide metadata to represent a bitmask or the allowed values of an enum?

[hamishwillee]

@dakejahl Thanks for the feedback.

• The design is sensible with respect to deconstructing all the functions you might want to set or discover feedback from a particular hardware. The caveat is that it is not lightweight since every function/status essentially represents one field.
• I am also not sure how it provides metadata for each bit in a bitmask.

We need a way to dynamically render UI in the ground station for payloads. I would love to see an example in QGC/PX4 to check it out. Ideally we could create something in SITL for it.

The UI could dynamically render a "generic" UI fairly simply as we have a name for the payload and each of its functions, we know how many functions there are, and their order, type, and units, and ditto for status.

Upshot is that you could relatively easily generate a UI for displaying each function alongside the setter function in a 2 column grid. [I'm not sure yet how the metadata for an enum or bitmask is provided].

My concern is much like yours. While this is tested internally by skydio, it doesn't sound like they have end to end testing integration with a GCS, missions etc. To test it fully and accept it into common we really need an implementation of both ends. ideally a MAVSDK server plugin on one side with a test implementation and on the other side a QGC UI.

@thomas-watters-skydio I guess you're not going to be able to provide either of those?