Skip to content

Commit a2ef954

Browse files
committed
User manual - update coverage statements
* Update counts of total rules, supported rules and implemented rules. * Expand description of what makes a rule supportable. * Report which rules are not implemented.
1 parent 0af1ef3 commit a2ef954

File tree

2 files changed

+16
-13
lines changed

2 files changed

+16
-13
lines changed

docs/user_manual.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
| 0.15.0 | 2023-05-24 | Mauro Baluda | Clarify AUTOSAR C++ supported versions. |
2424
| 0.16.0 | 2023-07-03 | Luke Cartey | Remove reference to LGTM, update the name of the query pack |
2525
| 0.17.0 | 2023-08-16 | Luke Cartey | Update list of supported compiler configurations. |
26+
| 0.87.0 | 2024-01-30 | Luke Cartey | Update product description and coverage table. |
2627

2728
## Release information
2829

@@ -48,15 +49,16 @@ A _coding standard_ is a set of rules or guidelines which restrict or prohibit t
4849

4950
The _CodeQL Coding Standards_ product is a set of CodeQL queries for identifying contraventions of rules in the following coding standards:
5051

51-
| Standard | Version | Total rules | Total supportable rules | Status |
52-
| -------------------------------------------------------------------------------------------------------------------- | ------- | ----------- | ----------------------- | ----------------- |
53-
| [AUTOSAR C++](https://www.autosar.org/fileadmin/standards/R22-11/AP/AUTOSAR_RS_CPP14Guidelines.pdf) | [^1] R22-11, R21-11, R20-11, R19-11, R19-03 | 397 | 375 | Implemented |
54-
| [CERT-C++](https://resources.sei.cmu.edu/downloads/secure-coding/assets/sei-cert-cpp-coding-standard-2016-v01.pdf) | 2016 | 83 | 83 | Implemented |
55-
| [CERT C](https://resources.sei.cmu.edu/downloads/secure-coding/assets/sei-cert-c-coding-standard-2016-v01.pdf) | 2016 | 99 | 99 | Under development |
56-
| [MISRA C](https://www.misra.org.uk/product/misra-c2012-third-edition-first-revision/) | 2012 | 172 | 169 | Under development |
52+
| Standard | Version | Rules | Supportable rules | Implemented rules | Status |
53+
| -------------------------------------------------------------------------------------------------------------------- | ------- | ----------- | ----------------------- | ----------------- | ------- |
54+
| [AUTOSAR C++](https://www.autosar.org/fileadmin/standards/R22-11/AP/AUTOSAR_RS_CPP14Guidelines.pdf) | [^1] R22-11, R21-11, R20-11, R19-11, R19-03 | 397 | 372 | 370[^2] | Implemented |
55+
| [CERT-C++](https://resources.sei.cmu.edu/downloads/secure-coding/assets/sei-cert-cpp-coding-standard-2016-v01.pdf) | 2016 | 83 | 82 | 82 | Implemented |
56+
| [CERT C](https://resources.sei.cmu.edu/downloads/secure-coding/assets/sei-cert-c-coding-standard-2016-v01.pdf) | 2016 | 99 | 97 | 97 | Implemented |
57+
| [MISRA C](https://www.misra.org.uk/product/misra-c2012-third-edition-first-revision/) | 2012 | 175 | 164 | 162[^3] | Implemented |
5758

59+
Not all rules in these standards are amenable to static analysis by CodeQL - some rules require external or domain specific knowledge to validate, or refer to properties which are not present in the our representation of the codebase under analysis. In addition, some rules are natively enforced by our supported set of compilers. As CodeQL requires that the program under analysis compiles, we are unable to implement queries for these rules, and doing so would be redundant.
5860

59-
Not all rules in these standards are amenable to static analysis by CodeQL - some rules require external or domain specific knowledge to validate, or refer to properties which are not present in the our representation of the codebase under analysis. For each rule we therefore identify whether it is supportable or not. Furthermore, a rule can be supported in two ways:
61+
For each rule we therefore identify whether it is supportable or not. Furthermore, a rule can be supported in two ways:
6062

6163
- **Automated** - the queries for the rule find contraventions directly.
6264
- **Audit only** - the queries for the rule does not find contraventions directly, but instead report a list of _candidates_ that can be used as input into a manual audit. For example, `A10-0-1` (_Public inheritance shall be used to implement 'is-a' relationship_) is not directly amenable to static analysis, but CodeQL can be used to produce a list of all the locations that use public inheritance so they can be manually reviewed.
@@ -66,6 +68,8 @@ Each supported rule is implemented as one or more CodeQL queries, with each quer
6668
The datasheet _"CodeQL Coding Standards: supported rules"_, provided with each release, lists which rules are supported for that particular release, and the _scope of analysis_ for that rule.
6769

6870
[^1]: AUTOSAR C++ versions R22-11, R21-11, R20-11, R19-11 and R19-03 are all identical as indicated in the document change history.
71+
[^2]: The unimplemented supportable AUTOSAR rules are `A7-1-8` and `A8-2-1`. These rules require additional support in the CodeQL CLI to ensure the required information is available in the CodeQL database to identify violations of these rules.
72+
[^3]: The unimplemented supportable MISRA C 2012 rules are `Rule 9.5` and `Dir 4.14`. `Rule 9.5` requires additional support in the CodeQL CLI to ensure the required information is available in the CodeQL database to identify violations of these rules. `Dir 4.14` is covered by the default CodeQL queries, which identify potential security vulnerabilities caused by not validating external input.
6973

7074
## Supported environment
7175

rules.csv

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -500,7 +500,7 @@ c,CERT-C,CON41-C,Yes,Rule,,,Wrap functions that can fail spuriously in a loop,CO
500500
c,CERT-C,CON43-C,OutOfScope,Rule,,,Do not allow data races in multithreaded code,,,,
501501
c,CERT-C,DCL30-C,Yes,Rule,,,Declare objects with appropriate storage durations,,Declarations8,Hard,
502502
c,CERT-C,DCL31-C,Yes,Rule,,,Declare identifiers before using them,,Declarations1,Medium,
503-
c,CERT-C,DCL36-C,No,Rule,,,Do not declare an identifier with conflicting linkage classifications,,,,
503+
c,CERT-C,DCL36-C,No,Rule,,,Do not declare an identifier with conflicting linkage classifications,,,,Compiler enforced in all supported compilers
504504
c,CERT-C,DCL37-C,Yes,Rule,,,Do not declare or define a reserved identifier,,Declarations1,Easy,
505505
c,CERT-C,DCL38-C,Yes,Rule,,,Use the correct syntax when declaring a flexible array member,,Declarations2,Easy,
506506
c,CERT-C,DCL39-C,Yes,Rule,,,Avoid information leakage when passing a structure across a trust boundary,,Declarations7,Hard,
@@ -585,7 +585,7 @@ c,CERT-C,POS51-C,OutOfScope,Rule,,,Avoid deadlock with POSIX threads by locking
585585
c,CERT-C,POS52-C,OutOfScope,Rule,,,Do not perform operations that can block while holding a POSIX lock,,,,
586586
c,CERT-C,POS53-C,OutOfScope,Rule,,,Do not use more than one mutex for concurrent waiting operations on a condition variable,,,,
587587
c,CERT-C,POS54-C,OutOfScope,Rule,,,Detect and handle POSIX library errors,,,,
588-
c,CERT-C,PRE30-C,No,Rule,,,Do not create a universal character name through concatenation,,,Medium,
588+
c,CERT-C,PRE30-C,No,Rule,,,Do not create a universal character name through concatenation,,,Medium,Compiler enforced in all supported compilers
589589
c,CERT-C,PRE31-C,Yes,Rule,,,Avoid side effects in arguments to unsafe macros,RULE-13-2,SideEffects4,Medium,
590590
c,CERT-C,PRE32-C,Yes,Rule,,,Do not use preprocessor directives in invocations of function-like macros,,Preprocessor5,Hard,
591591
c,CERT-C,SIG30-C,Yes,Rule,,,Call only asynchronous-safe functions within signal handlers,,SignalHandlers,Medium,
@@ -614,8 +614,8 @@ c,MISRA-C-2012,DIR-4-9,Yes,Advisory,,,A function should be used in preference to
614614
c,MISRA-C-2012,DIR-4-10,Yes,Required,,,Precautions shall be taken in order to prevent the contents of a header file being included more than once,M16-2-3,Preprocessor2,Medium,
615615
c,MISRA-C-2012,DIR-4-11,Yes,Required,,,The validity of values passed to library functions shall be checked,,Contracts,Hard,
616616
c,MISRA-C-2012,DIR-4-12,Yes,Required,,,Dynamic memory allocation shall not be used,,Banned,Medium,
617-
c,MISRA-C-2012,DIR-4-13,Yes,Advisory,,,Functions which are designed to provide operations on a resource should be called in an appropriate sequence,,Contracts,Hard,
618-
c,MISRA-C-2012,DIR-4-14,Yes,Required,,,The validity of values received from external sources shall be checked,,Contracts,Hard,
617+
c,MISRA-C-2012,DIR-4-13,No,Advisory,,,Functions which are designed to provide operations on a resource should be called in an appropriate sequence,,,,Rule 22.1, 22.2 and 22.6 cover aspects of this rule. In other cases this is a design issue and needs to be checked manually.
618+
c,MISRA-C-2012,DIR-4-14,Yes,Required,,,The validity of values received from external sources shall be checked,,Contracts,Hard,This is supported by CodeQLs default C security queries.
619619
c,MISRA-C-2012,RULE-1-1,No,Required,,,"The program shall contain no violations of the standard C syntax and constraints, and shall not exceed the implementation's translation limits",,,Easy,"This should be checked via the compiler output, rather than CodeQL, which adds unnecessary steps."
620620
c,MISRA-C-2012,RULE-1-2,Yes,Advisory,,,Language extensions should not be used,,Language3,Hard,
621621
c,MISRA-C-2012,RULE-1-3,Yes,Required,,,There shall be no occurrence of undefined or critical unspecified behaviour,,Language3,Hard,
@@ -773,5 +773,4 @@ c,MISRA-C-2012,RULE-22-6,Yes,Mandatory,,,The value of a pointer to a FILE shall
773773
c,MISRA-C-2012,RULE-22-7,Yes,Required,,,The macro EOF shall only be compared with the unmodified return value from any Standard Library function capable of returning EOF,,IO3,Hard,
774774
c,MISRA-C-2012,RULE-22-8,Yes,Required,,,The value of errno shall be set to zero prior to a call to an errno-setting-function,ERR30-C,Contracts3,Medium,
775775
c,MISRA-C-2012,RULE-22-9,Yes,Required,,,The value of errno shall be tested against zero after calling an errno-setting-function,,Contracts3,Medium,
776-
c,MISRA-C-2012,RULE-22-10,Yes,Required,,,The value of errno shall only be tested when the last function to be called was an errno-setting-function,,Contracts3,Medium,
777-
,,,,,,,,,,0,
776+
c,MISRA-C-2012,RULE-22-10,Yes,Required,,,The value of errno shall only be tested when the last function to be called was an errno-setting-function,,Contracts3,Medium,

0 commit comments

Comments
 (0)