You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are various extractors, modules, and helper functions exposed by the framework that can be imported and used in your project. The framework also exposes a logger designed for outputting log statements to the console during extraction. The framework's exports can be seen in the [index.js file](https://github.com/mcode/mcode-extraction-framework/blob/master/src/index.js).
30
39
@@ -43,6 +52,96 @@ const cancerDiseaseStatusExtractor = new CSVCancerDiseaseStatusExtractor('path-t
43
52
})();
44
53
```
45
54
55
+
## Client Usage
56
+
57
+
The framework also contains the [MCODEClient](src/client/MCODEClient.js) which has registered all of the extractors in this repo. Once you have exported CSV data and updated your configuration file, use the mCODE Extraction client by running the following:
58
+
59
+
```bash
60
+
node src/cli/cli.js [options]
61
+
```
62
+
63
+
To see all the options that can be used with the mCODE client, run the following:
64
+
65
+
```bash
66
+
node src/cli/cli.js --help
67
+
```
68
+
69
+
### First Time User Guide
70
+
71
+
To run the mCODE Extraction Client you must have:
72
+
73
+
1. A CSV file at `data/patient-mrns.csv` that contains a list of all MRN's to be extracted;
74
+
2. CSV files containing the patient data to transform into mCODE data;
75
+
3. A configuration file that points to your data and provides additional information.
76
+
77
+
#### CSV File Formats: Steps 1 & 2
78
+
79
+
CSV data for each extractor is expected in the `data` directory. The CSV schema for each data element is located in [docs](docs). The two patient CSV files below are necessary for the client to run.
80
+
81
+
-`data/patient-mrns.csv` which adheres to the [Patient MRN's CSV Schema](docs/patient-mrns.csv)
82
+
-`data/patient-information.csv` which adheres to the [Patient CSV Schema](docs/patient.csv)
83
+
84
+
Examples files for these extractor can be found in the [`test/sample-client-data`](test/sample-client-data) directory. Files there provide examples of the values that are expected in each column, but they are not based on any real patient data.
85
+
86
+
#### Configuration Files: Step 3
87
+
88
+
After exporting your CSV files to the `data` directory, kickstart the creation of a configuration file by renaming the provided `mcode-csv-config.example.json` to `csv.config.json`. Then, ensure the following configuration parameters are properly set:
89
+
90
+
1.`patientIdCsvPath` should provide a file path to a CSV file containing MRN's for relevant patients;
91
+
2. For each extractor, `filePath:` should provide a file path to a CSV file containing that corresponding extractor's data;
92
+
93
+
For instructions on setting up an email notification trigger whenever an error is encountered in extraction, see the [Email Notification](#Email-Notification) section below.
94
+
95
+
### Configuration Deep Dive
96
+
97
+
Each deployment of the mCODE Extraction Client needs a configuration file. This file will specify basic information that every run will use. The configuration file can live in the `config` directory or any directory you prefer. An illustrative example file can be found in [`config/mcode-csv-config.example.json`](config/mcode-csv-config.example.json).
98
+
99
+
To specify which patients the client should extract data for, the configuration file _must_ point to a CSV file containing MRNs for each patient. The format for this file can be found [here](https://github.com/mcode/mcode-extraction-framework/blob/master/docs). An example of this file can be found in [`test/sample-client-data/patient-mrns.csv`](test/sample-client-data/patient-mrns.csv).
100
+
101
+
Each extractor uses various methods to gather data and format that data into [mCODE](http://hl7.org/fhir/us/mcode/index.html) profiled resources. The `observation` extractor formats data into a general [FHIR R4](http://hl7.org/fhir/R4) profile. Extractors may require additional configuration items that can be specified in the configuration file.
102
+
103
+
### Email Notification
104
+
105
+
The mCODE Extraction Client supports sending an email using the SMTP protocol when there are errors during data extraction.
106
+
The connection to the SMTP server is considered authenticated from the start. Currently, there is no support for providing authentication information separately through configuration.
107
+
108
+
In order to send an email, users must specify the hostname or IP address of an SMTP server to connect to and the email addresses to send the email to. Optionally, users can specify the port to connect to and the email address to send from. These fields must be specified in the `notificationInfo` object in the configuration file. Below is more information on each field that can be specified. Further information can be found in the [`nodemailer` documentation](https://nodemailer.com/) for the [SMTP transport](https://nodemailer.com/smtp/) and [message configuration](https://nodemailer.com/message/).
109
+
110
+
-`host`: The hostname or IP address of an SMTP server to connect to
111
+
-`port`: (Optional) The port to connect to (defaults to 587)
112
+
-`to`: Comma separated list or an array of recipients email addresses that will appear on the _To:_ field
113
+
-`from`: (Optional) The email address of the sender. All email addresses can be plain `'sender@server.com'` or formatted `'"Sender Name" sender@server.com'` (defaults to mcode-extraction-errors@mitre.org, which cannot receive reply emails)
114
+
115
+
An example of this object can be found in [`config/mcode-csv-config.example.json`](config/mcode-csv-config.example.json).
116
+
117
+
If the `notificationInfo` object is provided in configuration, an email will be sent using the specified options if any errors occur during data extraction. If any required field is missing in the object (`host` or `to`), an email cannot be sent. If you prefer to not have an email sent even if errors occur, you can choose to not include the `notificationInfo` object in your configuration file.
118
+
119
+
### Logging Successful Extractions
120
+
121
+
Whenever the mCODE Extraction Client successfully runs, a log is kept of the given date range of the extraction. Users will need to specify the location of the file to save this information. The default location is in a `logs` directory in a file called `run-logs.json`. Initially, this file's contents should be an empty array, `[]`. Users will need to create this file before running the mCODE Extraction Client with `from-date` and/or `to-date` for the first time.
122
+
123
+
Users can specify a different location for the file by using the `--run-log-filepath <path>` CLI option. For example:
The mCODE Extraction Client will extract all data that is provided in the CSV files by default, regardless of any dates associated with each row of data. It is recommended that any required date filtering is performed outside of the scope of this client.
132
+
133
+
If for any reason a user is required to specify a date range to be extracted through this client, users _must_ add a `dateRecorded` column in every data CSV file. This column will indicate when each row of data was added to the CSV file. Note that this date _does not_ correspond to any date associated with the data element.
134
+
135
+
#### CLI From-Date and To-Date (NOT recommended use)
136
+
137
+
If any filtering on data elements in CSV files is required, the `entries-filter` option must be used. The remaining instructions in this section assume this flag is provided.
138
+
139
+
If a `from-date` is provided as an option when running the mCODE Extraction Client, it will be used to filter out any data elements that are recorded before that date based on the `dateRecorded` column in the CSV files. If a `to-date` is provided as an option, it will be used to filter out any data elements that are recorded after that date based on the `dateRecorded` column in the CSV files. If no `to-date` is provided, the default is today. If no `from-date` is provided, the mCODE Extraction Client will look to a run log file (details [above](#Logging-Successful-Extractions)) to find the most recent run and use the `to-date` of that run as the `from-date` for the current run, allowing users to only run the extraction on data elements that were not included in previous runs. If there are no previous run times logged, a `from-date` needs to be provided when running the extraction when the `entries-filter` option is provided. If the `entries-filter` option is not provided, any `from-date` and `to-date` options will be ignored, none of the data elements will be filtered by date, and a successful run will not be logged since there is no specified date range. An example running the client with the `from-date` and `to-date` is as follows:
0 commit comments