Allow a DataSource to be used to provide values in generated items

[andrewserff]

Sometimes, we have a need to generate values that are based on some other source of data. For example, we might have an existing source of data that provides information about airports around the world. We might want to use the actual names, locations, or callsigns of those airports when generating events. So we would like to have a new Data Source that you could provide to the generator that could be pulled from when each event is generated. Then you should be able to reference the values within the items pulled from the data source in your generated events.

[andrewserff]

I've added initial support for Data Sources which you specify in the config. Currently supporting only CSV files as a Data Source but could be expanded to support other types. The Data Source is loaded, then at event generation time, a random item is pulled from the data source (in the csv case, a single row is pulled) and then loaded into a map of key/values. You can reference these values using a new source.key-name property type in your generation config. As an example, I'm using this database of Airports from here: https://ourairports.com/data/ (specifically the navaids.csv). Then provide a config like:

{
    "workflows": [{
            "workflowName": "test",
            "workflowFilename": "dataSourceWorkflow.json",
            "instances": 1
        }
   ],
    "producers": [{
        "type":"logger"
    }],
    "source": {
        "type": "csv",
        "filename": "/tmp/navaids.csv"
    }
}

Then in your Workflow config, you can reference data like:

{
    "eventFrequency": 500,
    "varyEventFrequency": true,
    "repeatWorkflow": true,
    "timeBetweenRepeat": 500,
    "varyRepeatFrequency": true,
    "steps": [{
            "config": [{
                    "timestamp": "nowTimestamp()",
                    "system": "source.name",
                    "country": "source.iso_country",
                    "latitude": "source.latitude_deg",
                    "longitude": "source.longitude_deg",
                    "state": "random(\"ACTIVE\",\"INACTIVE\")"                    
                }],
            "duration": 0
        }]
}

This will generate data like:

{"timestamp":1705357053522,"system":"Jenpeg","country":"CA","latitude":54.52000045776367,"longitude":-98.04340362548828,"state":"ACTIVE"}
{"timestamp":1705357054382,"system":"Lecce","country":"IT","latitude":40.2484016418457,"longitude":18.130699157714844,"state":"INACTIVE"}

I would like to also add the ability to add deviation around a source value. For instance, if you ask for the latitude from the source, but don't want the exact latitude but would rather a point close to point provided by the source. I'll have to see if I have time to also add this capability.

I also still need to update the README to add how to use this new feature before submitting a PR.