[Linux-kernel-mentees] [PATCH v4 00/14] cocci: Improve C parsing of attributes
Jaskaran Singh
jaskaransingh7654321 at gmail.com
Fri Jun 5 13:43:08 UTC 2020
This patch series aims to improve parsing of attributes in C by
Coccinelle's C parser.
These parsing errors were discovered by running a build of Coccinelle's
master branch on gitlab to parse the source code of Linux v5.5-rc4 and Linux
v5.6-rc7. Coccinelle currently manages attributes similar to comments, so
to explicity state what the attributes are to the C parser, a
MACROANNOTATION hint was used in Coccinelle's standard.h file. The hint
enables the parser to correctly identify attributes, and thus potentially
handle them differently in the AST.
Upon collection of these parsing errors, The GNU C Compiler's
grammar was taken as a reference for introducing attributes in correct
places in Coccinelle's C parser.
Upon applying the MACROANNOTATION hint to a majority of the attributes in
Coccinelle's standard.h file and parsing the source code of Linux
v5.6-rc7, the difference between the stats (of --parse-c) before and
after applying this patch series are as follows:
Before:
NB total files = 28268; perfect = 25697; pbs = 2570; timeout = 0; =========> 90%
nb good = 19160664, nb passed = 135235 =========> 0.70% passed
nb good = 19160664, nb bad = 129685 =========> 99.33% good or passed
After:
NB total files = 28268; perfect = 25699; pbs = 2568; timeout = 0; =========> 90%
nb good = 19160856, nb passed = 71716 =========> 0.37% passed
nb good = 19160856, nb bad = 129493 =========> 99.33% good or passed
What this patch series does not do:
- Add attributes from all the new productions into the C AST. Some of
these are dropped.
- Reflect changes from the C parser in the SmPL parser.
Separate patches will be sent for the above.
Changes in v4:
-------------
Remove the following patches from the series and add/merge them to [1].
parsing_c: visitor_c: Add vk_attribute and vk_attribute_s
parsing_c: lib_parsing_c: Add ii_of_attr
engine: Add distrf_attr to PARAM functor
engine: cocci_vs_c: Add allminus argument to attribute_list
parsing_c: parser: Make abstract_declarator pass attributes
cocci: Add parameter attributes to C AST
parsing_c: parser: Place parameter attributes in C AST
parsing_c: visitor_c: Visit parameter attributes
parsing_c: unparse_hrule: Add parameter attributes in record
parsing_c: type_c: Add parameter attributes to record
engine: cocci_vs_c: "Match" parameter attributes
engine: c_vs_c: Match parameter attributes
parsing_c: parser: Add rule for optional end attributes
- Add to series.
parsing_c: parser: Add field declaration end attributes production
- Use end_attributes_opt to avoid duplicate code.
parsing_c: parser: Handle struct/union/enum end attributes
- Use end_attributes_opt to avoid duplicate code.
parsing_c: parser: Add MacroDecl end attributes production
- Use end_attributes_opt to avoid duplicate code.
parsing_c: parser: cpp_other end attributes production
- Use end_attributes_opt to avoid duplicate code.
Changes in v3:
-------------
parsing_c: parser: Place parameter attributes in C AST
- Remove the "attr is ignored" comments in the declaratorp rule.
- Place the attributes from the "declarator attributes" production
in the declaratorp rule in the C AST as well.
Changes in v2:
--------------
parsing_c: parser: Add attribute production in spec_qualif_list
- This patch applied on its own causes a build error. Fix the error by
removing the let binding for abstract_declaratort in the type_name
rule
parsing_c: parser: Make abstract_declarator pass attributes
- Reflect changes from the above build error fix.
Jaskaran Singh (14):
parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens
parsing_c: parsing_hacks: Label end attributes correctly
parsing_c: parsing_hacks: Commentize attributes before qualif/type
parsing_c: parser: Add rule for multiple end attributes
parsing_c: parser: Add expression statement attributes
parsing_c: parser: Add rule for optional end attributes
parsing_c: parser: Add attribute production in spec_qualif_list
parsing_c: parser: Add init_declarator_attrs rule
parsing_c: parser: Add field declaration end attributes production
parsing_c: parser: Handle struct/union/enum end attributes
parsing_c: parser: Add MacroDecl end attributes production
parsing_c: parser: cpp_other end attributes production
parsing_c: cpp_token_c: Introduce MACROANNOTATION hint
cocci: standard.h: Annotate attributes with MACROANNOTATION
parsing_c/cpp_token_c.ml | 2
parsing_c/parse_c.ml | 4 +
parsing_c/parser_c.mly | 91 ++++++++++++++++++++++++-----------
parsing_c/parsing_hacks.ml | 37 ++++++++++++++
standard.h | 116 ++++++++++++++++++++++-----------------------
5 files changed, 164 insertions(+), 86 deletions(-)
More information about the Linux-kernel-mentees
mailing list