TfsWorkItemTypeValidatorTool – use field mapping and field value mapping in validation (#3010)

This PR solves the last two items in [improve work item type validation
list](#2951).

Validation now respects field mapings (`FieldToFieldMultiMap` and
`FieldToFieldMap`) and field value mappings (`FieldValueMap`) of
`FieldMappingTool`. For `FieldToFieldMap` mappings only the ones with
`SourceToTarget` mapping mode are used.

Configuration for validator `TfsWorkItemTypeValidatorTool` has changed a
bit:

- `SourceFieldMappings` is removed, This existed only to define field
mappings between source and target. Now real mappings from
`FieldMappingTool` are respected.
- `FixedTargetFields` was removed. It is not needed and it was not
really good because it worked with target fields and basically we
validate source fields in target work item type.
- `ExcludeSourceFields` was added. Basically it replaces previous two
properties and it is a list of source fields which will be excluded from
validation.

Fields excluded from validation `ExcludeSourceFields` are not really
excluded. They are still validated so user knows if there are some
issues, but even when the issues are found, they are reported only as
information instead of warning and the field itself is considered valid.
This is especially useful, if the user still wants to migrate such a
field, but it is not possible to define simple mapping for it. For
example user will be notified, that the field `X` do not exist in
target, but will configure it as excluded and define `FieldMergeMap` for
this field.

There was a tip, that `FieldSkipMap` can also be used, but it is not
possible. This mapping type is for target fields and we need to skip
fields in source work item types.

## Example log outputs

I tried to log as much thing as possible, so the user will know what to
do.

### Field exists and is valid

```
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'System.Description' (Description).
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Target field 'System.Description' (Description) exists in 'Bug' and is valid.
```

### Field is mapped and is valid

```
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'Kros.VersionNumber' (Kros Version Number).
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]     Source field 'Kros.VersionNumber' is mapped to 'Custom.KrosVersionNumber'.
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Target field 'Custom.KrosVersionNumber' (KrosVersionNumber) exists in 'Bug' and is valid.
```

### Field is missing in target

```
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'Microsoft.VSTS.Scheduling.Effort' (Effort).
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]     Missing field 'Microsoft.VSTS.Scheduling.Effort' in 'Bug'.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source field reference name: Microsoft.VSTS.Scheduling.Effort
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source field name: Effort
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Field type: Double
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Allowed values:
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Allowed values type: Double
```

### Allowed values for field are missing in target

```
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'System.State' (State).
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Allowed values in target field do not match allowed values in source. Checking field value maps.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]     Missing values are: 'Approved', 'Committed', 'Done', 'Removed'
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]       Value 'Approved' is mapped to 'New', which exists in target.
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]       Value 'Done' is mapped to 'Closd', which does not exists in target. This is probably invalid 'FieldValueMap' configuration.
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]       Value 'Removed' is mapped to 'Closed', which exists in target.
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]     Source field 'System.State' and target field 'System.State' have different allowed values.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source allowed values: 'Approved', 'Committed', 'Done', 'New', 'Removed'
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Target allowed values: 'Active', 'Closed', 'New', 'Resolved'
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Missing values in target are: 'Committed', 'Done'
```

### Field is excluded from validation

```
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]   Validating source field 'Microsoft.VSTS.Scheduling.Effort' (Effort).
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]     Missing field 'Microsoft.VSTS.Scheduling.Effort' in 'Bug'.
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Source field reference name: Microsoft.VSTS.Scheduling.Effort
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Source field name: Effort
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Field type: Double
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Allowed values:
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Allowed values type: Double
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]     Field 'Microsoft.VSTS.Scheduling.Effort' is excluded from validation, so it is considered valid.
```

### Final log if there are any issues

```
[08:21:14 ERR] [16.3.2-Local.4-7-g1ca7b77d] Some work item types or their fields are not valid in the target system (see previous logs).
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If the work item type does not exist in target system, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Create it there.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure mapping to another work item type which exists in target using 'WorkItemTypeMappingTool' configuration.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Exclude it from validation. To configure which work item types are validated, use either 'IncludeWorkItemTypes' or 'ExcludeWorkItemTypes' of 'TfsWorkItemTypeValidatorTool' configuration (but not both at the same time).
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If field is missing in target, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Add missing field to the target work item type.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure field mapping using 'FieldToFieldMultiMap' in 'FieldMappingTool' configuration. This is simpler method for source to target field mapping, which allows to map multiple fields at once.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure field mapping using 'FieldToFieldMap' in 'FieldMappingTool' configuration. Mapping mode must be of type 'SourceToTarget'.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If allowed values of the source and target fields do not match, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Add missing allowed values to the target field.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure value mapping. Add field maps of type 'FieldValueMap' to 'FieldMappingTool' configuration.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] To exclude field from validation, just configure it in 'ExcludeSourceFields' of 'TfsWorkItemTypeValidatorTool' configuration. If field is excluded from validation, all the issues are still logged, just the result of validation is 'valid'.
```

Author: MrHinsh

docs/data/classes/reference.tools.tfsworkitemtypevalidatortool.yaml

@@ -53,19 +53,10 @@ configurationSamples:
     {
       "$type": "TfsWorkItemTypeValidatorToolOptions",
       "Enabled": true,
-      "IncludeWorkItemtypes": [],
-      "ExcludeWorkItemtypes": [],
+      "IncludeWorkItemTypes": [],
+      "ExcludeWorkItemTypes": [],
       "ExcludeDefaultWorkItemTypes": true,
-      "SourceFieldMappings": {
-        "User Story": {
-          "Microsoft.VSTS.Common.Prirucka": "Custom.Prirucka"
-        }
-      },
-      "FixedTargetFields": {
-        "User Story": [
-          "Custom.Prirucka"
-        ]
-      }
+      "ExcludeSourceFields": {}
     }
   sampleFor: MigrationTools.Tools.TfsWorkItemTypeValidatorToolOptions
 description: >-
@@ -83,48 +74,36 @@ options:
 - parameterName: ExcludeDefaultWorkItemTypes
   type: Boolean
   description: >-
-    If `true`, some work item types will be automatically added to `ExcludeWorkItemtypes` list.
+    If `true`, some work item types will be automatically added to `ExcludeWorkItemTypes` list.
      Work item types excluded by default are: Code Review Request, Code Review Response, Feedback Request,
      Feedback Response, Shared Parameter, Shared Steps.
   defaultValue: missing XML code comments
   isRequired: false
   dotNetType: System.Boolean, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
-- parameterName: ExcludeWorkItemtypes
-  type: List
-  description: List of work item types which will be excluded from validation.
-  defaultValue: missing XML code comments
-  isRequired: false
-  dotNetType: System.Collections.Generic.List`1[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
-- parameterName: FixedTargetFields
+- parameterName: ExcludeSourceFields
   type: Dictionary
   description: >-
-    List of target fields that are considered as `fixed`.
-     A field marked as fixed will not stop the migration if differences are found.
-     Instead of a warning, only an informational message will be logged.
+    List of fields in source work itemt types, that are excluded from validation.
+     Fields excluded from validation are still validated and all found issues are logged.
+     But the result of the validation is 'valid' and the issues are logged as information instead of warning.
 
-     Use this list when you already know about the differences and have resolved them,
-     for example by using `FieldMappingTool`.
-
-     The key is the target work item type name.
-     You can also use `*` to define fixed fields that apply to all work item types.
+     The key is the source work item type name.
+     You can also use `*` to exclude fields from all source work item types.
   defaultValue: null
   isRequired: false
   dotNetType: System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Collections.Generic.List`1[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
-- parameterName: IncludeWorkItemtypes
+- parameterName: ExcludeWorkItemTypes
   type: List
-  description: List of work item types which will be validated. If this list is empty, all work item types will be validated.
-  defaultValue: null
+  description: List of work item types which will be excluded from validation.
+  defaultValue: missing XML code comments
   isRequired: false
   dotNetType: System.Collections.Generic.List`1[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
-- parameterName: SourceFieldMappings
-  type: Dictionary
-  description: >-
-    Field reference name mappings. Key is work item type name, value is dictionary of mapping source filed name to
-     target field name. Target field name can be empty string to indicate that this field will not be validated in target.
-     As work item type name, you can use `*` to define mappings which will be applied to all work item types.
+- parameterName: IncludeWorkItemTypes
+  type: List
+  description: List of work item types which will be validated. If this list is empty, all work item types will be validated.
   defaultValue: null
   isRequired: false
-  dotNetType: System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
+  dotNetType: System.Collections.Generic.List`1[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e
 status: missing XML code comments
 processingTarget: missing XML code comments
 classFile: src/MigrationTools.Clients.TfsObjectModel/Tools/TfsWorkItemTypeValidatorTool.cs

docs/static/schema/configuration.schema.json

@@ -2182,22 +2182,6 @@
                 }
               }
             },
-            "TfsChangeSetMappingTool": {
-              "title": "TfsChangeSetMappingTool",
-              "description": "missing XML code comments",
-              "type": "object",
-              "properties": {
-                "ChangeSetMappingFile": {
-                  "description": "missing XML code comments",
-                  "type": "string"
-                },
-                "Enabled": {
-                  "description": "If set to `true` then the tool will run. Set to `false` and the processor will not run.",
-                  "type": "boolean",
-                  "default": "true"
-                }
-              }
-            },
             "TfsEmbededImagesTool": {
               "title": "TfsEmbededImagesTool",
               "description": "missing XML code comments",
@@ -2234,6 +2218,22 @@
                 }
               }
             },
+            "TfsChangeSetMappingTool": {
+              "title": "TfsChangeSetMappingTool",
+              "description": "missing XML code comments",
+              "type": "object",
+              "properties": {
+                "Enabled": {
+                  "description": "If set to `true` then the tool will run. Set to `false` and the processor will not run.",
+                  "type": "boolean",
+                  "default": "true"
+                },
+                "ChangeSetMappingFile": {
+                  "description": "missing XML code comments",
+                  "type": "string"
+                }
+              }
+            },
             "TfsNodeStructureTool": {
               "title": "TfsNodeStructureTool",
               "description": "Tool for creating missing area and iteration path nodes in the target project during migration. Configurable through TfsNodeStructureToolOptions to specify which node types to create.",
@@ -2469,20 +2469,11 @@
                   "default": "true"
                 },
                 "ExcludeDefaultWorkItemTypes": {
-                  "description": "If `true`, some work item types will be automatically added to `ExcludeWorkItemtypes` list.\r\n Work item types excluded by default are: Code Review Request, Code Review Response, Feedback Request,\r\n Feedback Response, Shared Parameter, Shared Steps.",
+                  "description": "If `true`, some work item types will be automatically added to `ExcludeWorkItemTypes` list.\r\n Work item types excluded by default are: Code Review Request, Code Review Response, Feedback Request,\r\n Feedback Response, Shared Parameter, Shared Steps.",
                   "type": "boolean"
                 },
-                "ExcludeWorkItemtypes": {
-                  "description": "List of work item types which will be excluded from validation.",
-                  "type": "array",
-                  "prefixItems": [
-                    {
-                      "type": "string"
-                    }
-                  ]
-                },
-                "FixedTargetFields": {
-                  "description": "List of target fields that are considered as `fixed`.\r\n A field marked as fixed will not stop the migration if differences are found.\r\n Instead of a warning, only an informational message will be logged.\r\n\n Use this list when you already know about the differences and have resolved them,\r\n for example by using `FieldMappingTool`.\r\n\n The key is the target work item type name.\r\n You can also use `*` to define fixed fields that apply to all work item types.",
+                "ExcludeSourceFields": {
+                  "description": "List of fields in source work itemt types, that are excluded from validation.\r\n Fields excluded from validation are still validated and all found issues are logged.\r\n But the result of the validation is 'valid' and the issues are logged as information instead of warning.\r\n\n The key is the source work item type name.\r\n You can also use `*` to exclude fields from all source work item types.",
                   "type": "object",
                   "default": "null",
                   "additionalProperties": {
@@ -2494,26 +2485,24 @@
                     ]
                   }
                 },
-                "IncludeWorkItemtypes": {
-                  "description": "List of work item types which will be validated. If this list is empty, all work item types will be validated.",
+                "ExcludeWorkItemTypes": {
+                  "description": "List of work item types which will be excluded from validation.",
                   "type": "array",
-                  "default": "null",
                   "prefixItems": [
                     {
                       "type": "string"
                     }
                   ]
                 },
-                "SourceFieldMappings": {
-                  "description": "Field reference name mappings. Key is work item type name, value is dictionary of mapping source filed name to\r\n target field name. Target field name can be empty string to indicate that this field will not be validated in target.\r\n As work item type name, you can use `*` to define mappings which will be applied to all work item types.",
-                  "type": "object",
+                "IncludeWorkItemTypes": {
+                  "description": "List of work item types which will be validated. If this list is empty, all work item types will be validated.",
+                  "type": "array",
                   "default": "null",
-                  "additionalProperties": {
-                    "type": "object",
-                    "additionalProperties": {
+                  "prefixItems": [
+                    {
                       "type": "string"
                     }
-                  }
+                  ]
                 }
               }
             },

docs/static/schema/schema.tools.tfsworkitemtypevalidatortool.json

@@ -11,27 +11,22 @@
       "default": "true"
     },
     "ExcludeDefaultWorkItemTypes": {
-      "description": "If `true`, some work item types will be automatically added to `ExcludeWorkItemtypes` list.\r\n Work item types excluded by default are: Code Review Request, Code Review Response, Feedback Request,\r\n Feedback Response, Shared Parameter, Shared Steps.",
+      "description": "If `true`, some work item types will be automatically added to `ExcludeWorkItemTypes` list.\r\n Work item types excluded by default are: Code Review Request, Code Review Response, Feedback Request,\r\n Feedback Response, Shared Parameter, Shared Steps.",
       "type": "boolean"
     },
-    "ExcludeWorkItemtypes": {
-      "description": "List of work item types which will be excluded from validation.",
-      "type": "array"
-    },
-    "FixedTargetFields": {
-      "description": "List of target fields that are considered as `fixed`.\r\n A field marked as fixed will not stop the migration if differences are found.\r\n Instead of a warning, only an informational message will be logged.\r\n\n Use this list when you already know about the differences and have resolved them,\r\n for example by using `FieldMappingTool`.\r\n\n The key is the target work item type name.\r\n You can also use `*` to define fixed fields that apply to all work item types.",
+    "ExcludeSourceFields": {
+      "description": "List of fields in source work itemt types, that are excluded from validation.\r\n Fields excluded from validation are still validated and all found issues are logged.\r\n But the result of the validation is 'valid' and the issues are logged as information instead of warning.\r\n\n The key is the source work item type name.\r\n You can also use `*` to exclude fields from all source work item types.",
       "type": "object",
       "default": "null"
     },
-    "IncludeWorkItemtypes": {
+    "ExcludeWorkItemTypes": {
+      "description": "List of work item types which will be excluded from validation.",
+      "type": "array"
+    },
+    "IncludeWorkItemTypes": {
       "description": "List of work item types which will be validated. If this list is empty, all work item types will be validated.",
       "type": "array",
       "default": "null"
-    },
-    "SourceFieldMappings": {
-      "description": "Field reference name mappings. Key is work item type name, value is dictionary of mapping source filed name to\r\n target field name. Target field name can be empty string to indicate that this field will not be validated in target.\r\n As work item type name, you can use `*` to define mappings which will be applied to all work item types.",
-      "type": "object",
-      "default": "null"
     }
   }
 }

src/MigrationTools.Clients.TfsObjectModel.Tests/Tools/FieldReferenceNameMappingToolOptionsTests.cs

@@ -259,11 +259,13 @@ private void ValidateWorkItemTypes()
                 .ToProject()
                 .WorkItemTypes
                 .Cast<WorkItemType>()
+                .OrderBy(wit => wit.Name)
                 .ToList();
             var targetWits = Target.WorkItems.Project
                 .ToProject()
                 .WorkItemTypes
                 .Cast<WorkItemType>()
+                .OrderBy(wit => wit.Name)
                 .ToList();
 
             // Reflected work item ID field is mandatory for migration, so it is validated even if the validator tool is disabled.

src/MigrationTools.Clients.TfsObjectModel/Processors/TfsWorkItemMigrationProcessor.cs

@@ -41,11 +41,13 @@ protected override void InternalExecute()
                 .ToProject()
                 .WorkItemTypes
                 .Cast<WorkItemType>()
+                .OrderBy(wit => wit.Name)
                 .ToList();
             List<WorkItemType> targetWits = Target.WorkItems.Project
                 .ToProject()
                 .WorkItemTypes
                 .Cast<WorkItemType>()
+                .OrderBy(wit => wit.Name)
                 .ToList();
             bool containsReflectedWorkItemId = CommonTools.WorkItemTypeValidatorTool
                 .ValidateReflectedWorkItemIdField(sourceWits, targetWits, Target.Options.ReflectedWorkItemIdField);