drivers: usb: stm32: usb_write size on bulk transfer problematic

[JonathanVanWin]

Describe the bug
When sending data packets (as a device to host) that are multiples of 64 bytes (endpoint max packet size), they don't get detected until the device sends another packet that isn't a multiple of 64.

• Using STM32F4
• USB full speed bulk transfers with max packet size of 64.
• Host is Linux.
• Have sniffed the usb packets, and while they do arrive to the host (I see IN data0, data1 packets), the host only detects them when sending another packet that isn't a 64 multiple, and then it receives everything in one go (while the packets are stored already on the host).
• It looks more people are speaking about the same behaviour: looks like the same bug
• When sending just one packet on a clean state (after reset) of a size that isn't a multiple of 64, the device sends it like always, but the host receives it directly as expected.
• When host requests a size SIZE (bigger then one packet of 64), and we start sending packets that are multiple of 64, the host also receives them, BUT ONLY when the amount of bytes send in TOTAL is bigger than SIZE (of course as described if after a few packets we send a non multiple of 64 packet, and the total amount is less than SIZE, we also receive on the host everything in one go.)

Notice: The packets seem to arrive to the host, but don't get detected, till one of those two options happen.

Expected behavior
Receive directly everything on the host side, even if they are a multiple of 64.

Impact
It limits the type of packets a user can send.

Environment (please complete the following information):

• OS: Linux
• Toolchain Zephyr SDK
• Tag: v2.6.0

Additional context
It looks like the usb driver from zephyr isn't problematic, but the bug probably is in the hal_stm32 repo, specifically stm32f4 hal PCD, and stm32f4 ll usb.

[erwango]

It looks like the usb driver from zephyr isn't problematic, but the bug probably is in the hal_stm32 repo, specifically stm32f4 hal PCD, and stm32f4 ll usb.

@JonathanVanWin, thanks for raising the point, but if Zephyr driver is not guilty can you please raise the point directly to https://github.com/STMicroelectronics/STM32CubeF4 ? Thanks

[JonathanVanWin]

Hi @erwango, it seems that in order to solve the bug, the hal should take care of transfers that don't finish with a short packet, send a Zero length packet.
Tested it on my stm32, and it seems that it solved the issue.
Relevant information from the usb specification, chapter 5.8.3:

An endpoint must always transmit data payloads with a data field less than or equal to the endpoint’s reported wMaxPacketSize value. When a bulk IRP involves more data than can fit in one maximum-sized data payload, all data payloads are required to be maximum size except for the last data payload, which will contain the remaining data. A bulk transfer is complete when the endpoint does one of the following:

• Has transferred exactly the amount of data expected
• Transfers a packet with a payload size less than wMaxPacketSize or transfers a zero-length packet

[aurel32]

it seems that in order to solve the bug, the hal should take care of transfers that don't finish with a short packet, send a Zero length packet.

I agree that the end of transfer is signaled from the the device side by a packet shorter than wMaxPacketSize. That said I disagree that it should be done in the HAL (or if so it should be made optional as you might want to do bulk transfers bigger than the RAM buffers). IMHO this is the responsibility of the application, which is the only one to know if the current data being sent is part of a bigger transfer or not. At least this is how this is done in Zephyr where usb_transfer() send send a ZLP by default if needed, but has the USB_TRANS_NO_ZLP flag to signal more data is to be sent later in the same transfer. See for instance the bluetooth or netusb classes.

Note that the CDC ACM case is a bit tricky as a ZLP packet is interpreted by some host OS as the end of the transmission. That's why Zephyr prefer to send a short packet followed by another short packet instead of a ZLP for this class.

• When host requests a size SIZE (bigger then one packet of 64), and we start sending packets that are multiple of 64, the host also receives them, BUT ONLY when the amount of bytes send in TOTAL is bigger than SIZE (of course as described if after a few packets we send a non multiple of 64 packet, and the total amount is less than SIZE, we also receive on the host everything in one go.)

OS behaves differently on that aspect, but basically you can't expect bulk transfers to be received continuously on the host side. From the application point of view the way the data of a bulk transfer is chunked physically doesn't matter, so the data is usually passed to the application when the transfer is complete (i.e. the request size is reached or a short packet or a ZLP is received). You should either split those transfers in smaller ones (possibly sending a short packet or a ZLP) or look at interrupt or isochronous endpoint depending on your application.

[jfischer-no]

Relevant information from the usb specification, chapter 5.8.3:
An endpoint must always transmit data payloads with a data field less than or equal to the endpoint’s reported wMaxPacketSize value. When a bulk IRP involves more data than can fit in one maximum-sized data payload, all data payloads are required to be maximum size except for the last data payload, which will contain the remaining data. A bulk transfer is complete when the endpoint does one of the following:

• Has transferred exactly the amount of data expected
• Transfers a packet with a payload size less than wMaxPacketSize or transfers a zero-length packet

Hi @erwango, it seems that in order to solve the bug, the hal should take care of transfers that don't finish with a short packet, send a Zero length packet. Tested it on my stm32, and it seems that it solved the issue.

From the above quoted, what makes you think that a bulk transfer consisting of multiples of endpoint MPS, should always be terminated with a ZLP? And that this is always interpreted as successful? It depends on your host application, and there are applications that simply expect a stream (without ZLP in between). Unfortunately, you do not tell us how your host application does USB requests and neither what you use on Zephyr side (usb_write(), usb_transfer()). IMO this should be converted to a discussion.

[erwango]

Converting to discussion

[JonathanVanWin]

@aurel32 has provided a reasonable answer. I used the lower level usb_write functions that don't send the ZLP, but now I added another call to usb_write with ZLP.
It sounds reasonable the application of sending transfers bigger than the ram buffer.

[erwango]

Thanks for the heads up.