Merge pull request #2260 from strictdoc-project/stanislaw/docs

docs: document the migration path: [SECTION] -> [[SECTION]]

Author: stanislaw

docs/strictdoc_01_user_guide.sdoc

@@ -14,6 +14,7 @@ ELEMENTS:
 - TAG: SECTION
   PROPERTIES:
     IS_COMPOSITE: True
+    PREFIX: None
     VIEW_STYLE: Narrative
   FIELDS:
   - TITLE: MID
@@ -544,15 +545,40 @@ TITLE: Document structure
 [TEXT]
 MID: 5419c0c9bfcc4922ac5aa79ddb333188
 STATEMENT: >>>
-An SDoc document consists of a ``[DOCUMENT]`` declaration followed by a sequence of nodes:
+An SDoc document consists of a ``[DOCUMENT]`` declaration followed by a sequence of **leaf nodes** or **composite nodes**.
 
-- Lead nodes: ``[TEXT]`` or ``[REQUIREMENT]``
-- Composite nodes: ``[COMPOSITE_REQUIREMENT]``
-- Section nodes that group other nodes recursively: ``[SECTION]``.
+A **leaf node** cannot contain any child nodes. It is simply a node with text fields. The default leaf nodes are defined as ``[TEXT]`` or ``[REQUIREMENT]``. Any other ``[...]`` custom leaf node can be registered by a user in a document grammar.
 
-Each construct is described in more detail below.
+Example of a lead node:
+
+.. code-block:: text
+
+    [REQUIREMENT]
+    STATEMENT: StrictDoc shall be based on a document model.
+
+A **composite node** can contain other composite or leaf nodes. The default composite node is ``[[SECTION]]`` which models a section/chapter in a document. Any other ``[[...]]`` composite node can be registered by a user in a document grammar.
+
+Example of a composite node:
+
+.. code-block:: text
+
+    [[SECTION]]
+    TITLE: Document model
+
+    [REQUIREMENT]
+    STATEMENT: StrictDoc shall be based on a document model.
+
+    [[/SECTION]]
+
+The document elements are described below.
 <<<
 
+[[/SECTION]]
+
+[[SECTION]]
+MID: ec9dcd4ad28a4852bc47723e667006df
+TITLE: Syntax rules
+
 [[SECTION]]
 MID: 926894ac8efc4e99943741691b3d4efe
 UID: SECTION-UG-Strict-rule-1
@@ -561,7 +587,7 @@ TITLE: Strict rule #1: One empty line between all nodes
 [TEXT]
 MID: 2ebd1f2d3c8746a383d71232756c3bd3
 STATEMENT: >>>
-StrictDoc's grammar requires each node, such as ``[REQUIREMENT]``, ``[SECTION]``,
+StrictDoc's grammar requires each node, such as ``[REQUIREMENT]``, ``[[SECTION]]``,
 etc., to be separated with exactly one empty line from the nodes surrounding it.
 This rule is valid for all nodes. Absence of an empty line or presence of more
 than one empty line between two nodes will result in an SDoc parsing error.
@@ -577,11 +603,13 @@ TITLE: Strict rule #2: No content is allowed outside of SDoc grammar
 [TEXT]
 MID: 686eb01cf7db44c4bd1edd72ccb02ed0
 STATEMENT: >>>
-StrictDoc's grammar does not allow any content to be written outside of the SDoc
-grammatical constructs. It is assumed that the critical content shall always be
-written in form of requirements:
-``[REQUIREMENT]`` and ``[COMPOSITE_REQUIREMENT]``. Non-critical content shall
-be specified using ``[TEXT]`` nodes.
+StrictDoc's grammar does not allow any content to be written outside the SDoc grammatical constructs.
+
+It is assumed that the critical content shall always be
+written in form of leaf nodes and composite nodes, such as
+``[REQUIREMENT]`` and ``[[SECTION]]``.
+
+Non-critical content shall be specified using the ``[TEXT]`` leaf nodes.
 <<<
 
 [[/SECTION]]
@@ -1228,7 +1256,11 @@ TITLE: Section
 [TEXT]
 MID: 14e5fea31b1c41bf8c743f18e51053e0
 STATEMENT: >>>
-The ``[SECTION]`` element is used for creating document chapters and grouping
+.. warning::
+
+    As of 2025 Q2, StrictDoc is switching to a new ``[[SECTION]]`` syntax for declaring sections (previously ``[SECTION]``). If you are an existing user, please follow the migration guide [LINK: SECTION-UG-NODE-MIGRATION].
+
+The ``[[SECTION]]`` element can used for creating document chapters and grouping
 requirements into logical groups. It is equivalent to the use of ``#``, ``##``,
 ``###``, etc., in Markdown and ``====``, ``----``, ``~~~~`` in RST.
 
@@ -3849,6 +3881,91 @@ The existing workaround for this problem is to increase a number of semaphores i
 MID: b55c23b113d54802b5717f7a82a45bd2
 TITLE: Appendices
 
+[[SECTION]]
+MID: 2ed20461aeb4462b812432c08aa006be
+UID: SECTION-UG-NODE-MIGRATION
+TITLE: [[NODE]] migration
+
+[TEXT]
+MID: 9d55b69bf789487e814210da06fc4164
+STATEMENT: >>>
+As of 2025 Q2, the following changes are made to StrictDoc's grammar in a partially backward-incompatible manner. The relevant GitHub issue related to the migration:
+
+`Migration: SDoc model: Merge Section, Node, Composite Node into just Node #2193 <https://github.com/strictdoc-project/strictdoc/issues/2193>`_
+
+1\) The ``[[CUSTOM_NODE]]`` syntax with two square brackets around the element name has been introduced for composite nodes.
+
+Example of declaring a composite node in a grammar and using it in a document:
+
+.. code-block::
+
+    [GRAMMAR]
+    ELEMENTS:
+    - TAG: SECTION
+      PROPERTIES:
+        IS_COMPOSITE: True
+        PREFIX: None
+        VIEW_STYLE: Narrative
+      FIELDS:
+      - TITLE: MID
+        TYPE: String
+        REQUIRED: False
+      - TITLE: UID
+        TYPE: String
+        REQUIRED: False
+      - TITLE: TITLE
+        TYPE: String
+        REQUIRED: True
+
+     [[SECTION]]
+     TITLE: Section #1
+
+     [[/SECTION]]
+
+Using the new syntax, it is now possible to declare any custom composite element, such as what ``[COMPOSITE_REQUIREMENT]`` used to provide before.
+
+2\) The previously used ``[SECTION]`` syntax with one square bracket around the SECTION is now DEPRECATED, but the old behavior is still supported. The users are encouraged to migrate all ``[SECTION]`` elements to the new ``[[SECTION]]`` syntax. To support the new behavior, the grammar must include a declaration for a composite SECTION element similar to the example provided above.
+
+3\) The ``[COMPOSITE_REQUIREMENT]`` node has been removed from the codebase. This is a backward-incompatible change, but it is assumed that no users should be affected since this feature has never been developed enough to be useful.
+<<<
+
+[[SECTION]]
+MID: 6f15ef6876ba48598d5560aaf4664ec9
+TITLE: How to migrate from [SECTION] to [[SECTION]]
+
+[TEXT]
+MID: 19fe5099aa034eddb960cc88953ace0e
+STATEMENT: >>>
+The migration from ``[SECTION]`` to the new composite ``[[SECTION]]`` syntax is mostly straightforward and can be done manually. However, StrictDoc also provides a dedicated configuration option to assist with this migration.
+
+1\) **Manual approach**
+
+Update all instances of ``[SECTION]`` to ``[[SECTION]]`` manually, or use a combination of ``find`` and ``sed``. Additionally, make sure to update all document grammar declarations to include the new composite ``[[SECTION]]`` element.
+
+2\) **Semi-automated approach (experimental)**
+
+StrictDoc can automatically migrate SDoc files if the project configuration option ``section_behavior`` is set as follows:
+
+.. code-block::
+
+    [project]
+    section_behavior = "[[SECTION]]"
+
+With this option enabled, StrictDoc's ``export`` command will convert all ``[SECTION]`` nodes to ``[[SECTION]]`` when writing SDoc files.
+
+To perform the migration, run the export command:
+
+.. code-block::
+
+    strictdoc export . --formats=sdoc
+
+The resulting files will have all ``[SECTION]`` nodes replaced with ``[[SECTION]]``.
+<<<
+
+[[/SECTION]]
+
+[[/SECTION]]
+
 [[SECTION]]
 MID: a72ae310493a42ba8ecdc7de2c42d4f2
 UID: SECTION-UG-FREETEXT-TEXT

strictdoc/backend/sdoc/models/document_grammar.py

@@ -581,6 +581,22 @@ def create_default_section_element(
                 required="False",
             )
         )
+        fields.append(
+            GrammarElementFieldString(
+                parent=None,
+                title=RequirementFieldName.LEVEL,
+                human_title=None,
+                required="False",
+            )
+        )
+        fields.append(
+            GrammarElementFieldString(
+                parent=None,
+                title=RequirementFieldName.PREFIX,
+                human_title=None,
+                required="False",
+            )
+        )
         fields.append(
             GrammarElementFieldString(
                 parent=None,

strictdoc/backend/sdoc/processor.py

@@ -24,7 +24,9 @@
 
 
 class ParseContext:
-    def __init__(self, path_to_sdoc_file: Optional[str]):
+    def __init__(
+        self, path_to_sdoc_file: Optional[str], migrate_sections: bool = False
+    ):
         self.path_to_sdoc_file: Optional[str] = path_to_sdoc_file
         self.path_to_sdoc_dir: Optional[str] = None
         if path_to_sdoc_file is not None:
@@ -37,6 +39,7 @@ def __init__(self, path_to_sdoc_file: Optional[str]):
         self.document_has_requirements = False
 
         self.fragments_from_files: List[SDocDocumentFromFileIF] = []
+        self.migrate_sections: bool = migrate_sections
 
 
 class SDocParsingProcessor:
@@ -46,7 +49,10 @@ def __init__(self, parse_context: ParseContext):
     def process_document(self, document: SDocDocument):
         document.grammar = (
             self.parse_context.document_grammar
-            or DocumentGrammar.create_default(document)
+            or DocumentGrammar.create_default(
+                document,
+                create_section_element=self.parse_context.migrate_sections,
+            )
         )
         self.parse_context.document = document
         document.ng_including_document_reference = (

strictdoc/backend/sdoc/reader.py

@@ -29,8 +29,10 @@ class SDReader:
     )
 
     @staticmethod
-    def _read(input_string, file_path=None):
-        parse_context = ParseContext(path_to_sdoc_file=file_path)
+    def _read(input_string, file_path=None, migrate_sections: bool = False):
+        parse_context = ParseContext(
+            path_to_sdoc_file=file_path, migrate_sections=migrate_sections
+        )
         processor = SDocParsingProcessor(parse_context=parse_context)
         SDReader.meta_model.register_obj_processors(
             processor.get_default_processors()
@@ -84,7 +86,9 @@ def read_from_file(
 
         try:
             sdoc, parse_context = self.read_with_parse_context(
-                sdoc_content, file_path=file_path
+                sdoc_content,
+                file_path=file_path,
+                migrate_sections=project_config.is_new_section_behavior(),
             )
 
             sdoc.fragments_from_files = parse_context.fragments_from_files
@@ -118,9 +122,6 @@ def read_from_file(
 
     @staticmethod
     def convert(section: SDocSection) -> SDocNode:
-        """
-        FIXME: Handle LEVEL, REQ_PREFIX
-        """
         fields = []
 
         if section.mid_permanent:
@@ -141,6 +142,27 @@ def convert(section: SDocSection) -> SDocNode:
                     multiline=False,
                 )
             )
+        if section.ng_resolved_custom_level is not None:
+            fields.append(
+                SDocNodeField.create_from_string(
+                    None,
+                    field_name="LEVEL",
+                    field_value=section.ng_resolved_custom_level,
+                    multiline=False,
+                )
+            )
+        if (
+            section.requirement_prefix is not None
+            and len(section.requirement_prefix) > 0
+        ):
+            fields.append(
+                SDocNodeField.create_from_string(
+                    None,
+                    field_name="PREFIX",
+                    field_value=section.requirement_prefix,
+                    multiline=False,
+                )
+            )
         fields.append(
             SDocNodeField.create_from_string(
                 None,
@@ -160,6 +182,14 @@ def convert(section: SDocSection) -> SDocNode:
         )
         for field_ in fields:
             field_.parent = node
+
+        node.ng_including_document_reference = (
+            section.ng_including_document_reference
+        )
+        node.ng_document_reference = section.ng_document_reference
+        node.ng_level = section.ng_level
+        node.ng_resolved_custom_level = section.ng_resolved_custom_level
+
         return node
 
     @staticmethod

strictdoc/core/project_config.py

@@ -79,6 +79,7 @@ class ProjectConfig:
     DEFAULT_SERVER_PORT = 5111
     DEFAULT_BUNDLE_DOCUMENT_VERSION = "@GIT_VERSION (Git branch: @GIT_BRANCH)"
     DEFAULT_BUNDLE_DOCUMENT_COMMIT_DATE = "@GIT_COMMIT_DATETIME"
+    DEFAULT_SECTION_BEHAVIOR = "[SECTION]"
 
     def __init__(
         self,
@@ -107,6 +108,7 @@ def __init__(
         reqif_import_markup: Optional[str],
         config_last_update: Optional[datetime.datetime],
         chromedriver: Optional[str],
+        section_behavior: Optional[str],
     ):
         assert isinstance(environment, SDocRuntimeEnvironment)
         if source_root_path is not None:
@@ -193,6 +195,7 @@ def __init__(
         self.is_running_on_server: bool = False
         self.view: Optional[str] = None
         self.chromedriver: Optional[str] = chromedriver
+        self.section_behavior: Optional[str] = section_behavior
 
     @staticmethod
     def default_config(environment: SDocRuntimeEnvironment) -> "ProjectConfig":
@@ -221,6 +224,7 @@ def default_config(environment: SDocRuntimeEnvironment) -> "ProjectConfig":
             reqif_import_markup=None,
             config_last_update=None,
             chromedriver=None,
+            section_behavior=ProjectConfig.DEFAULT_SECTION_BEHAVIOR,
         )
 
     # Some server command settings can override the project config settings.
@@ -347,6 +351,9 @@ def is_activated_source_file_language_parsers(self) -> bool:
             ProjectFeature.SOURCE_FILE_LANGUAGE_PARSERS in self.project_features
         )
 
+    def is_new_section_behavior(self):
+        return self.section_behavior == "[[SECTION]]"
+
     def get_strictdoc_root_path(self) -> str:
         return self.environment.path_to_strictdoc
 
@@ -441,6 +448,8 @@ def _load_from_dictionary(
         reqif_import_markup: Optional[str] = None
         chromedriver: Optional[str] = None
 
+        section_behavior: str = ProjectConfig.DEFAULT_SECTION_BEHAVIOR
+
         if "project" in config_dict:
             project_content = config_dict["project"]
             project_title = project_content.get("title", project_title)
@@ -610,6 +619,11 @@ def _load_from_dictionary(
                     assert isinstance(test_report_root_entry_, dict)
                     test_report_root_dict.update(test_report_root_entry_)
 
+            section_behavior = project_content.get(
+                "section_behavior", section_behavior
+            )
+            assert section_behavior in ("[SECTION]", "[[SECTION]]")
+
         if "server" in config_dict:
             server_content = config_dict["server"]
             server_host = server_content.get("host", server_host)
@@ -673,4 +687,5 @@ def _load_from_dictionary(
             reqif_import_markup=reqif_import_markup,
             config_last_update=config_last_update,
             chromedriver=chromedriver,
+            section_behavior=section_behavior,
         )

tasks.py

@@ -107,7 +107,7 @@ def clean_itest_artifacts(context):
     run_invoke(
         context,
         """
-        git clean -dfX tests/integration/
+        git clean -dX --force --quiet tests/integration/
         """,
         warn=True,
     )