Skip to content

Conversation

raissi-oussema
Copy link
Contributor

@raissi-oussema raissi-oussema commented Dec 7, 2024

DoIP Protocol Overview

The Diagnostic over IP (DoIP) protocol is used in automotive diagnostic systems to facilitate communication between diagnostic tools and ECUs (Electronic Control Units) over IP-based networks. It enables remote diagnostics, configuration, and software updates over Ethernet, offering an efficient and scalable solution for modern vehicles.

Header Structure (8 bytes)

protocol version (1 byte)
invert protocol version (1 byte)
payload type (2 bytes)
payload length (4 bytes)

Pyload types / code / structure

  • "Generic DOIP header Nack" (0x0000) (header + nackCode (1 byte : required))
  • "Vehicle identification request" (0x0001) (header)
  • "Vehicle identification request with EID" (0x0002) (header + EID (6 bytes : required))
  • "Vehicle identification request with VIN" (0x0003) (header + VIN (17 bytes : required))
  • "Vehicle announcement message" (0x0004) (header + VIN (17 bytes : required) + logical address (2bytes : required) + EID (6 bytes : required) + GID (6 bytes : required) + further action (1 bytes : required)+ sync status (1 bytes : optional)).
  • "Routing activation request" (0x0005) (header + source address (2 bytes : required) + activation type (1 byte:required) + reservedISO (4 bytes : required) + reservedOEM (4 bytes : optional)).
  • "Routing activation response" (0x0006) (header + tester adress(2 bytes : required) + response code (1 byte:required) + reservedISO (4 bytes : required) + reservedOEM (4 bytes : optional)).
  • "Alive check request" (0x0007) (header)
  • "Alive check response" (0x0008) (header + source address (2 byte : required))
  • "DOIP entity status request" (0x4001) (header)
  • "DOIP entity status response" (0x4002) (header + node type (1 byte : required) + max concurrent socket (1 byte : required) + currently opened sockets (1 byte : required) + max data size(4 bytes : optional))
  • "Diagnostic power mode request information" (0x4003) (header)
  • "Diagnostic power mode response information" (0x4004) (header + power mode (1 byte : required))
  • "Diagnostic message" (0x8001) (header + source address (2 bytes : required) + target address (2 bytes : required) + UDS message (n bytes : required ; n > 1))
  • "Diagnostic message Ack" (0x8002) (header + source address (2 bytes : required) + target address (2 bytes : required) + ack code (1 byte : required)) + previous message (n bytes : optional ; n>0))
  • "Diagnostic message Nack" (0x8003) (header + source address (2 bytes : required) + target address (2 bytes : required) + Nack code (1 byte : required)) + previous message (n bytes : optional ; n>0))

@Dimi1010
Copy link
Collaborator

Dimi1010 commented Dec 8, 2024

As per the contributing guidelines, please retarget the PR to the dev branch instead of the master.

@tigercosmos tigercosmos changed the base branch from master to dev December 8, 2024 09:37
@egecetin egecetin closed this Dec 8, 2024
@egecetin egecetin reopened this Dec 8, 2024
@egecetin egecetin linked an issue Dec 8, 2024 that may be closed by this pull request
@raissi-oussema
Copy link
Contributor Author

Observed several issues in the CI pipelines, likely due to missing definitions for htobe16 and other endianness conversion macros. All tests and pre-commit checks are passed in my linux machine.
Adding an import for "EndianPortable.h" in last PR is expected to resolve these problems.

Copy link

codecov bot commented Dec 10, 2024

Codecov Report

❌ Patch coverage is 97.82828% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.43%. Comparing base (be201c4) to head (82a0fcd).
⚠️ Report is 1 commits behind head on dev.

Files with missing lines Patch % Lines
Packet++/src/DoIpLayer.cpp 96.74% 30 Missing and 1 partial ⚠️
Packet++/header/DoIpLayer.h 96.16% 10 Missing ⚠️
Packet++/src/TcpLayer.cpp 83.33% 1 Missing ⚠️
Packet++/src/UdpLayer.cpp 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1655      +/-   ##
==========================================
+ Coverage   82.88%   83.43%   +0.54%     
==========================================
  Files         292      295       +3     
  Lines       51834    53814    +1980     
  Branches    11293    11863     +570     
==========================================
+ Hits        42965    44902    +1937     
- Misses       7671     8093     +422     
+ Partials     1198      819     -379     
Flag Coverage Δ
alpine320 75.38% <95.01%> (+0.68%) ⬆️
fedora42 75.49% <94.92%> (+0.64%) ⬆️
macos-13 81.63% <95.71%> (+0.53%) ⬆️
macos-14 81.63% <95.71%> (+0.53%) ⬆️
macos-15 81.64% <95.71%> (+0.53%) ⬆️
mingw32 70.23% <71.72%> (+0.05%) ⬆️
mingw64 70.23% <71.81%> (+0.06%) ⬆️
rhel94 75.22% <94.27%> (+0.66%) ⬆️
ubuntu2004 59.18% <66.49%> (+0.30%) ⬆️
ubuntu2004-zstd 59.26% <66.49%> (+0.29%) ⬆️
ubuntu2204 75.15% <94.28%> (+0.65%) ⬆️
ubuntu2204-icpx 60.94% <62.77%> (+0.07%) ⬆️
ubuntu2404 75.41% <95.02%> (+0.70%) ⬆️
ubuntu2404-arm64 75.40% <95.02%> (+0.68%) ⬆️
unittest 83.43% <97.82%> (+0.54%) ⬆️
windows-2022 85.43% <95.75%> (+0.40%) ⬆️
windows-2025 85.45% <95.75%> (+0.39%) ⬆️
winpcap 85.45% <95.75%> (+0.39%) ⬆️
xdp 52.85% <94.28%> (+1.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@seladb
Copy link
Owner

seladb commented Jan 3, 2025

@raissi-oussema are you planning to continue working on this PR?

@raissi-oussema
Copy link
Contributor Author

Hi, I was engaged with other tasks, but I plan to get back to this PR soon. Thanks for your understanding!

raissi-oussema and others added 2 commits January 6, 2025 01:52
.improve  maps searchs for doipEnumsToStrings
.cover more uses cases based on codecov feedback
@raissi-oussema
Copy link
Contributor Author

Design suggestions or code improvements are always welcome and greatly appreciated.

@seladb
Copy link
Owner

seladb commented Jan 8, 2025

@raissi-oussema to make it easier to review, do you think you can add some documentation on the DoIP protocol to the PR body?

It'd mostly be helpful to get more details on the header structure and different possible message

@raissi-oussema
Copy link
Contributor Author

@seladb I need support for CI pipelines, I can't figure out why are they still failing. And a clear documentation was successfully added to PR body to make it easier for you to start the code review.

@Dimi1010
Copy link
Collaborator

Dimi1010 commented Jan 12, 2025

@seladb I need support for CI pipelines, I can't figure out why are they still failing. And a clear documentation was successfully added to PR body to make it easier for you to start the code review.

Doxigen pipeline:
/__w/PcapPlusPlus/PcapPlusPlus/Packet++/header/DoIpLayer.h:42: error: Compound pcpp::DoIpLayer is not documented. (warning treated as error, aborting now)

XDP pipeline:
That is an issue with the CI image. Merge latest changes from dev branch and it should be fine.

VS pipeline:
Tbh, no idea om that one. It seems the opencoverage download link stopped working for a bit or something.

@raissi-oussema
Copy link
Contributor Author

@seladb I need support for CI pipelines, I can't figure out why are they still failing. And a clear documentation was successfully added to PR body to make it easier for you to start the code review.

Doxigen pipeline: /__w/PcapPlusPlus/PcapPlusPlus/Packet++/header/DoIpLayer.h:42: error: Compound pcpp::DoIpLayer is not documented. (warning treated as error, aborting now)

XDP pipeline: That is an issue with the CI image. Merge latest changes from dev branch and it should be fine.

VS pipeline: Tbh, no idea om that one. It seems the opencoverage download link stopped working for a bit or something.

What could be the problem for dioxygen pipeline, doipLayer.h is well documented [line 42] ?

// implement abstract methods

/**
* TODO, parse UDS layer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for another PR? If so, should the remaining data be parsed as a generic payload layer for now?

Copy link
Contributor Author

@raissi-oussema raissi-oussema Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m_NextLayer is intended to be the UDS layer, which has not been implemented yet. In the future, as more knowledge is gained, either I or another contributor may add this functionality. For now, I suggest parsing it as a generic layer, as you mentioned.
PS: the nextLayer will be parsed only when the payloadType is 0x8001.

Copy link
Contributor Author

@raissi-oussema raissi-oussema Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your feedback is highly appertiated, what do you think about adding this snippet of code:

void  parseNextLayer() override
		{
			if (getPayloadType() == DoIpPayloadTypes::DIAGNOSTIC_MESSAGE_TYPE)
			{
				size_t headerLen = sizeof(doiphdr);

				if (m_DataLen <= headerLen + 2 /*source address size*/ + 2 /*target address size*/)
					return;

				uint8_t* payload = m_Data + (headerLen + 2 + 2);
				size_t payloadLen = m_DataLen - (headerLen + 2 + 2);
				m_NextLayer = new PayloadLayer(payload, payloadLen, this, m_Packet);
				return;
			}
		}`

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your feedback is highly appertiated, what do you think about adding this snippet of code:

void  parseNextLayer() override
		{
			if (getPayloadType() == DoIpPayloadTypes::DIAGNOSTIC_MESSAGE_TYPE)
			{
				size_t headerLen = sizeof(doiphdr);

				if (m_DataLen <= headerLen + 2 /*source address size*/ + 2 /*target address size*/)
					return;

				uint8_t* payload = m_Data + (headerLen + 2 + 2);
				size_t payloadLen = m_DataLen - (headerLen + 2 + 2);
				m_NextLayer = new PayloadLayer(payload, payloadLen, this, m_Packet);
				return;
			}
		}`

Seems good. Can the source address and target address used in that type of message be accessed from this layer? Since we are excluding it from the generic payload.

A minor tip, having headerLen be marked as constexpr might allow some compiler optimizations (such as the arithmetic using it + a literal being computed during compilation and hardcoded if possible).

Also why the return statement that is at the end of the block anyway?

Copy link
Contributor Author

@raissi-oussema raissi-oussema Jan 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dimi1010 , this approach appears both safer and cleaner. It allows us to construct a DiagnosticMessage directly from the current layer, providing direct access to the diagnostic data. Using this data, we can then build a generic PayloadLayer.

void parseNextLayer() override
{
	DiagnosticMessageData diagnosticMessage;

	if (diagnosticMessage.buildFromLayer(this))
	{
		m_NextLayer = new PayloadLayer(diagnosticMessage.diagnosticData.data(),
				                             diagnosticMessage.diagnosticData.size(), this, m_Packet);
	}
}

buildFromLayer safely parses the current layer and verifies whether it represents a valid diagnostic message.
what do you think ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but my question stands. DiagnosticMessageData has two other members (sourceAddress and targetAddress) which at the moment I don't see how the user can access them easily. They have neither accessors in the current DoIPLayer or are included as part of the UDSLayer (currently PayloadLayer).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sourceAddress and targetAddress are public members of DiagnosticMessageData class just like diagnosticData and they are not part of the UDS layer:

uint16_t sourceAddress;              /**< Source address of the message. */
uint16_t targetAddress;              /**< Target address for the diagnostic message. */
std::vector<uint8_t> diagnosticData; /**< Diagnostic message data with dynamic length. */

the user can access these fields by the method buildFromLayer .
I've made this dummy function just to show how to access to these fields :

void DoIpLayer::resolveDiagMessageFields()
	{
		DiagnosticMessageData diagnosticMessage;
		if (diagnosticMessage.buildFromLayer(this))
		{
			uint16_t srcAddr = diagnosticMessage.sourceAddress;
			uint16_t targetAddr = diagnosticMessage.targetAddress;
			std::vector<uint8_t> diagData = diagnosticMessage.diagnosticData;
		}
	}

Do you think this implementation need more improvement ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, if i am understanding correctly the sequence is this?

  1. User somehow receives a DoIPLayer from a Packet
  2. User checks the payload type (via DoIPLayer::getPayloadType())
  3. Depending on the payload type user uses T::buildFromLayer(DoIPLayer) (T being the corresponding message struct) to populate the data from the layer into the struct.

Am I understanding the flow correctly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes you are absolutely correct.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that works fine. 👍

Co-authored-by: Dimitar Krastev <dimi1010100@gmail.com>
@raissi-oussema raissi-oussema requested a review from seladb May 25, 2025 22:06
@raissi-oussema raissi-oussema requested a review from seladb June 12, 2025 18:36
@seladb
Copy link
Owner

seladb commented Jun 15, 2025

@raissi-oussema please also note this comment: #1655 (comment)

@seladb
Copy link
Owner

seladb commented Jul 11, 2025

@raissi-oussema I think this PR is almost ready to merge. Can you address the remaining comments?

@raissi-oussema
Copy link
Contributor Author

@raissi-oussema I think this PR is almost ready to merge. Can you address the remaining comments?

I was tied up with other tasks, but I’ll take care of the remaining work over the next two days. Thanks for your patience!

@raissi-oussema raissi-oussema requested a review from seladb July 16, 2025 22:52
@raissi-oussema
Copy link
Contributor Author

@seladb, could you have a look, I've addressed the last comments and I made some improvements.

Copy link
Owner

@seladb seladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also see the open comments:

@raissi-oussema raissi-oussema requested a review from seladb July 28, 2025 23:36
Copy link
Owner

@seladb seladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@raissi-oussema just 2 last comments and I think we can merge this PR!

@raissi-oussema raissi-oussema requested a review from seladb August 1, 2025 15:26
@seladb seladb self-requested a review August 2, 2025 18:37
@seladb seladb merged commit 5868b83 into seladb:dev Aug 3, 2025
78 of 79 checks passed
@seladb
Copy link
Owner

seladb commented Aug 3, 2025

@raissi-oussema thank you so much for working on this PR! 🙏
I really appreciate your hard work and dedication, and I'm sorry it took a bit long to get this PR merged.
Great work, and thank you again for contributing to this project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

support of doip (diagnostic over IP protocol)

5 participants