Definition of Done

[drewcdisc]

New Development requirements

• Any new or updated feature or functionality requires that documentation be updated to reflect the change. This includes new rule operators, which should include a relevant example to inform authors on their implementation and usage.
• The person who codes is always different from the person who reviews.
• The coder will run the regression test as part of the Pull Request.

Testing Framework

• New or updated feature or functionality is successfully implement according to the request in the Product Backlog Item (PBI)
• Product Owner acceptance for each PBI
• Unit Tests are successful
• Any new operator or builder functionality requires that we update our test suite. [TBD: This bullet point is to be implemented only after the Testing Environment Epic is completed within CORE. Other edits of this DoD will likely be needed once the new Testing is underway.]
  • After new code is successfully unit tested, the dev will create a new PBI for updating the Test Suite. The new PBI will ensure that a rule is run to test the new code. It must apply the label "Test Suite Update"
    • If the Rule already exists, the new PBI will be added to the Sprint Backlog.
    • If the Rule still needs to be developed, the new PBI will be added to the top of the Product Backlog.
    • During Sprint Planning the Team will always need to filter the backlog for the label "Test Suite Update" to ensure that any newly available rules are accounted for in testing during the Sprint.
  • Adding a rule(s) to trigger the new operator or the new dataset builder.
  • If the previous version of the testing dataset does not cover the new operator or builder functionality, then the new dataset needs to be added. Nick will have the final say on this point.
• Any rule changes need to be tested internally within the engine
• Be carefully explicit about a successful result, and also leave open the opportunity to discuss with the PO or stakeholders what success means.
• Issue should not break the system.
• We have assiduously tried to break the system [negative testing].
• System testing via the CLI that uses any new rule and the data to go along with it.
  • This is a place where negative data can also be used.
• We need a minimum level of capability so that the tools are useful. That capability is a completed validation for the rule and data.
• Long run-times always provide useful (repeated) feedback so that the user does not cancel a run due to lack of knowledge of a successful run.
• Full regression testing is run successfully.

Graceful exception handling

• We can catch errors so that they don't unwind the stack back to the user.
• Specifically with rules, we are wrapping a TRY around each rule execution, logging any error, and continuing on.

Absurd uses of the system

• Rules should catch when the data is absurd
• Other non-rule absurdities will not be caught (for now)

Non-rule PBIs

These PBIs will require additional unit testing that will become part of the regression test. However, no additional system testing is added.

Average data set size

Average rule cycles required

Regression testing

This includes both system tests and unit tests.

Branching strategies

• On the cdisc-rules-engine repo, features are developed in their own branches but when the feature is done it is merged into the dev branch. We are considering skipping the dev branch and going straight into main since we main merges don't trigger a release. Dev Team please make final determination around merge.
• On the other repos, features are developed in their own branch and merged into main for release.

Engineering Disciplines

Whereas several areas of this DoD discuss engineering disciplines that are defined in specific contexts (like unit and regression tests), this section is for launching discussions about other engineering disciplines that we define or discover. Please add to this section the things that we want to do and those we want to at least consider.

When we resolve to include any of these disciplines into our DoD, please move them into their appropriate contexts within the DoD document and remove them from this list. They should only start here, not live here.

eXtreme Programming (XP)

XP is a process that emphasizes empowering engineers to take ownership and responsibility for their code. It has several practices that integrate together in order to achieve the best results. It is not possible for all XP states to be achieved in our team. For instance, we are not a co-located organization, so mitigation has to be allowed for that. This document will not become a conversation about XP; instead we all need to learn about it on our own. There is a good summary here.

Still, those areas of XP (or ANY engineering discipline) that we choose to use do belong in this section on Engineering Disciplines, at least as a start for the discussion. So, please add them as needed. As we work out how we need to integrate them in our process, we will want to include them in the other areas of this document where they have the most appropriate impact.

Pair Programming

Pair programming is a discipline that we discussed during a retrospective. It is a practice where two people work on the same code. Traditionally, this would mean two people literally sitting together at one machine, one typing while the other watches and makes suggestions. The idea is that this practice implements a real time code review. We would have to come up with some collaborative coding tools in order to accomplish this.

Refactoring

Refactoring is the practice of improving code quality without changing code behavior. Since writing code is synonymous with creating bugs, improving code quality AND changing code behavior can end up in a tug-of-war over bug management. One area where we discussed refactoring is the practice of keeping our technical requirements (version dependencies on outside libraries or platform) up-to-date. This is discussed further in a section below.

Technical Requirements

We are introducing a policy that says we will not allow the CORE platform to fall more than two versions behind on any libraries or platforms in which we are dependent. As of this writing CORE is dependent on Python 3.10, but Python has already released 3.13. Therefore we will be creating a PBI for managing this refactoring of our code to be updated appropriately.

Developer responsibility

Keeping our code up-to-date with versions must rely on our engineering team. Our product owner is unlikely to track those dependencies. Therefore the Development Team must take responsibility for bringing these dependency issues before the PO in the form of PBIs in New Issues, and bring it up during Planning.