Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
cb015bf
add: main file for conftool.
asllop Mar 28, 2025
ec62b17
docs: document questionnaire process.
asllop Mar 28, 2025
cb1b1cd
add: base class for yaml to model mapping.
asllop Mar 31, 2025
030e2f7
add: support for lists of objects.
asllop Mar 31, 2025
daac841
add: support for enums.
asllop Mar 31, 2025
d374ce5
add: support for lists of non base model items.
asllop Mar 31, 2025
f5660c9
add: models for config and all subtypes.
asllop Mar 31, 2025
e1c5df6
add: model integrity check system.
asllop Mar 31, 2025
e7735ba
add: integrity checks for config and service_schedule models.
asllop Mar 31, 2025
37f08bc
update: improve invalid enum error message.
asllop Apr 1, 2025
ad9396e
chore: factorize code.
asllop Apr 1, 2025
da35027
chore: reorganize project.
asllop Apr 1, 2025
9b5cf90
docs: add conftool project readme.
asllop Apr 1, 2025
3cacaf9
add: support for newtype base models.
asllop Apr 1, 2025
4cfb017
add: validation checks for redis and arguments models.
asllop Apr 1, 2025
72796b9
fix: accidental code.
asllop Apr 1, 2025
cbfbd9c
add: checks for inner types of list and dict.
asllop Apr 1, 2025
64ebc71
chore: factorize code.
asllop Apr 1, 2025
aec3de0
add: specific dict handler.
asllop Apr 1, 2025
523301c
add: handle None case gracefully.
asllop Apr 2, 2025
84435a2
update: Exception class.
asllop Apr 2, 2025
08cf1fd
add: `newrelic` config key.
asllop Apr 2, 2025
96e4c19
update: newrelic required config.
asllop Apr 2, 2025
338549d
docs: update instance config documentation.
asllop Apr 2, 2025
bd3425c
add: instance checks.
asllop Apr 2, 2025
618d198
update: instance arguments checks.
asllop Apr 2, 2025
d9a64b1
update: rename newrelic model class.
asllop Apr 2, 2025
d381383
add: auth checks. custom enum base class.
asllop Apr 2, 2025
bf7a378
update: redis config checks.
asllop Apr 2, 2025
a8c7cb5
add: event_type attribute to limits model.
asllop Apr 2, 2025
95e6a6c
add: checks for query config.
asllop Apr 2, 2025
e2802a7
chore: split up model classes into different files.
asllop Apr 3, 2025
a9a31fa
chore: rename enum module.
asllop Apr 3, 2025
f27ab62
chore: split up newrelic model.
asllop Apr 3, 2025
852b85d
update: capture ConfigException for parsing errors.
asllop Apr 3, 2025
b2a560f
add: base class for questions.
asllop Apr 3, 2025
3e36e7f
add: skippable questions.
asllop Apr 3, 2025
9b4a540
chore: encapsulate prompt_toolkit-dependant code.
asllop Apr 4, 2025
c4e3112
add: prompts for int and bool.
asllop Apr 4, 2025
2db05cc
chore: encapsulate question askers and model.
asllop Apr 4, 2025
da67995
add: string questions.
asllop Apr 4, 2025
8253c7b
update: improve skippable validation.
asllop Apr 4, 2025
310ea80
add: raw input questions.
asllop Apr 4, 2025
a95a945
add: config model to YAML serialization.
asllop Apr 8, 2025
87e8167
chore: move to_dict function to model package.
asllop Apr 8, 2025
b84a222
add: to_yaml method to config model
asllop Apr 8, 2025
39c1f9b
fix: return none from ask methods.
asllop Apr 8, 2025
9c73b32
add: text module.
asllop Apr 9, 2025
d23b955
add: instance questions.
asllop Apr 9, 2025
af59e95
fix: ask enum, use symbolic variant names to prompt.
asllop Apr 10, 2025
9303518
add: auth questions.
asllop Apr 10, 2025
f7932da
chore: refactor.
asllop Apr 10, 2025
e509c20
add: cache/redis questions.
asllop Apr 14, 2025
1046421
add: rest of arguments simple questions.
asllop Apr 14, 2025
83772a8
add: question basic questions.
asllop Apr 14, 2025
502fb73
add: ask for query list.
asllop Apr 15, 2025
a662928
add: newrelic config questions.
asllop Apr 15, 2025
fdeb045
add: read list of IDs.
asllop Apr 15, 2025
90c052c
fix: to YAML con version of "__inner_val__" models.
asllop Apr 15, 2025
74afae1
add: ask a dictionary of values.
asllop Apr 15, 2025
b2c4614
add: limits questions.
asllop Apr 15, 2025
c68d351
add: instance level service schedule conf.
asllop Apr 15, 2025
6b95d47
fix: NR api endpoint is required.
asllop Apr 16, 2025
127a655
chore: improve messages and remove comments.
asllop Apr 16, 2025
f30aa97
add: question hierarchy.
asllop Apr 16, 2025
b30f5b2
add: visual styles.
asllop Apr 17, 2025
f6a09f2
fix: service_chedule check rules.
asllop Apr 17, 2025
2e80d57
add: auth warning message on config validation.
asllop Apr 17, 2025
0427afb
update: cli arguments.
asllop Apr 17, 2025
c44485a
docs: update cli arguments.
asllop Apr 17, 2025
fa4648d
chore: reorganize questions.
asllop Apr 17, 2025
fa0cb3e
add: write data to file.
asllop Apr 17, 2025
544d572
fix: issue with python 3.13
asllop Apr 23, 2025
b08bb63
update: model for token url.
asllop Apr 23, 2025
b249053
docs: update readme.
asllop Apr 23, 2025
a0dd82f
docs: update README.
asllop Apr 24, 2025
1250df3
fix: review change requests.
asllop May 7, 2025
3cef699
chore: refactor id_list_check.
asllop May 8, 2025
c88ab10
fix: account id and license key aren't required, can be set as env vars.
asllop May 8, 2025
673dfc8
chore: remove pass keywords.
asllop May 8, 2025
83704da
chore: reorganize code in main.
asllop May 8, 2025
79ececd
fix: redis port var names.
asllop May 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -386,7 +386,15 @@ to specify the directory containing the [configuration file](#configyml).

Several configuration files are used to control the behavior of the exporter.

#### `config.yml`
#### Guided configuration

In order to simplify the configuration process, we provided a CLI application:
The [ConfTool](./conftool/).

Please follow the provided instructions in the [README](./conftool/README.md) for
installation and usage.

#### Manual configuration

The main configuration for the exporter is the `config.yml` file. In fact, it
does not need to be named `config.yml` although that is the default name if a
Expand Down Expand Up @@ -595,6 +603,17 @@ The `labels` parameter is a set of key/value pairs. The value of this parameter
is a YAML mapping. Each key/value pair is added to all logs and events generated
by the exporter.

###### `service_schedule`

| Description | Valid Values | Required | Default |
| --- | --- | --- | --- |
| Schedule configuration used by the built-in scheduler | YAML Mapping | N | N/a |

Instance-specific service schedule configuration. When present it has precedence
over the general `service_schedule` config.

Check [`service_schedule`](#service_schedule) for format description.

##### Instance arguments

The main configuration of an instance is specified in the `arguments` attribute
Expand Down
2 changes: 2 additions & 0 deletions conftool/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.yml
*.yaml
45 changes: 45 additions & 0 deletions conftool/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# ConfTool

Configuration tool for the New Relic Salesforce Exporter.

## Requirements

Tested with Python versions 3.9 and 3.13.

## Install

From `conftool` folder:

- Create a virtual environment ([venv](https://virtualenv.pypa.io/en/latest/installation.html) is required):

```
python<version> -m venv my_env
```

- Activate the environment:

```
source my_env/bin/activate
```

- Then install dependencies:

```
pip install -r requirements.txt
```

## Usage

From repo's root folder.

1. To create new config file:

```
python -m conftool path/to/config.yml
```

1. To validate an existing config file:

```
python -m conftool path/to/config.yml --check
```
1 change: 1 addition & 0 deletions conftool/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
VERSION = "0.1.0"
111 changes: 111 additions & 0 deletions conftool/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
from conftool.model.data_format import DataFormatModel
from . import VERSION
from .model.config import ConfigModel
from .model.exception import ConfigException
from .form import questionnaire
from .form.format import print_warning, print_fail, print_ok
from .form.text import t_warning_missing_auth, t_warning_missing_account_id, \
t_warning_missing_license

import argparse
import os.path

TITLE = f"New Relic Salesforce Exporter Config Tool {VERSION}"
DESCRIPTION = f"{TITLE}. Create or check configuration files."

def main():
parser = argparse.ArgumentParser(description=DESCRIPTION)
parser.add_argument('config_file', type=str, help='File path. Config YAML file to create or check.')
parser.add_argument('-c', '--check', action='store_true', help='Check configuration.')
args = parser.parse_args()

print(f"{TITLE}\n")

if args.check:
check_config(args.config_file)
else:
create_config(args.config_file)

def create_config(config_file: str):
print("Creating new config file...\n")

if os.path.isfile(config_file):
print("Error: Config file already exists.")
exit(1)

try:
conf = questionnaire.run()
except KeyboardInterrupt:
print("\nAborted.")
exit(1)

conf_yml = conf.to_yaml()

print_config(conf_yml)

try:
with open(config_file, "w+") as file:
try:
file.write(conf_yml)
file.close()
except (IOError, OSError):
print("Error: Could not write to file.")
exit(1)
except (FileNotFoundError, PermissionError, OSError):
print("Error: Could not open file.")
exit(1)

print_ok("Done.")
print()

def check_config(config_file: str):
print("Validating config file...\n")

if not os.path.isfile(config_file):
print("Error: Config file doesn't exist.")
exit(1)

try:
config_yaml_str = read_file(config_file)
except Exception as err:
print("Error opening file:", err)
exit(1)

try:
config_model = ConfigModel.from_yaml(config_yaml_str)
except ConfigException as err:
print_fail(str(err))
print()
exit(1)

for index,i in enumerate(config_model.instances):
if i.arguments.auth is None:
print(f"At instance #{index + 1}:")
print_warning(t_warning_missing_auth)
print()

if config_model.newrelic.data_format == DataFormatModel.EVENTS and \
config_model.newrelic.account_id is None:
print_warning(t_warning_missing_account_id)

if config_model.newrelic.license_key is None:
print_warning(t_warning_missing_license)

# Serialize model into YAML
serialized_yaml = config_model.to_yaml()
print_config(serialized_yaml)

print_ok("Validation OK!")
print()

def read_file(file_name: str) -> str:
with open(file_name) as file:
return file.read()

def print_config(conf):
print('---- CONFIG FILE:\n')
print(conf)
print('----')
print()

if __name__ == "__main__": main()
30 changes: 30 additions & 0 deletions conftool/form/format.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
class Color:
BG_RED = '\x1b[41m'
FG_BOLD_RED = '\x1b[1;31m'

FG_BOLD_WHITE = '\x1b[1;37m'

BG_BLUE = '\x1b[44m'

BG_YELLOW = '\x1b[43m'

FG_BOLD_BLACK = '\x1b[1;30m'

FG_BOLD_GREEN = '\x1b[1;32m'

RESET = '\x1b[0m'

def print_title(msg: str):
print(Color.BG_RED + Color.FG_BOLD_WHITE + ' ' + msg + ' ' + Color.RESET + '\n')

def print_statement(msg: str):
print(Color.BG_BLUE + Color.FG_BOLD_WHITE + msg + Color.RESET + "\n")

def print_warning(msg: str):
print(Color.BG_YELLOW + Color.FG_BOLD_BLACK + "WARNING: " + msg + Color.RESET + "\n")

def print_fail(msg: str):
print(Color.FG_BOLD_RED + "FAILED: " + msg + Color.RESET)

def print_ok(msg: str):
print(Color.FG_BOLD_GREEN + msg + Color.RESET)
146 changes: 146 additions & 0 deletions conftool/form/prompt.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
from prompt_toolkit.validation import Validator, ValidationError
from prompt_toolkit import prompt

def prompt_list(message: str, options: list[str], required: bool) -> int:
for i,option in enumerate(options):
print(str(i+1) + ") " + str(option))
validator = ListNumberValidator(1, len(options), not required)
response = prompt(message + " ", validator=validator)
if response is None or response == "":
return None
else:
return int(response)

def prompt_int(message: str, min: int, max: int, required: bool) -> int:
validator = NumberRangeValidator(min, max, not required)
response = prompt(message + " ", validator=validator)
if response is None or response == "":
return None
else:
return int(response)

def prompt_bool(message: str, required: bool) -> int:
validator = BooleanValidator(not required)
response = prompt(message + " ", validator=validator)
if response is None or response == "":
return None
else:
return response == "y" or response == "Y"

def prompt_str(message: str, checker, required: bool) -> int:
validator = StringValidator(checker, not required)
response = prompt(message + " ", validator=validator)
if response == "":
return None
else:
return response

def prompt_any(message: str, required: bool) -> int:
validator = SkippableValidator(not required)
response = prompt(message + " ", validator=validator)
if response == "":
return None
else:
return response

class SkippableValidator(Validator):
skippable: bool

def __init__(self, skippable: bool):
self.skippable = skippable
super().__init__()

def validate(self, document):
text = document.text
if not self.skippable and (text is None or text == ""):
raise ValidationError(
message=f"Input can't be empty",
cursor_position=0
)

def can_skip(self, text):
return self.skippable and (text is None or text == "")

class ListNumberValidator(SkippableValidator):
min: int
max: int

def __init__(self, min: int, max: int, skippable: bool):
super().__init__(skippable)
self.min = min
self.max = max

def validate(self, document):
super().validate(document)
text = document.text
if self.can_skip(text):
return
if not self.digit_in_range(text):
raise ValidationError(
message="Input must be a number in the list",
cursor_position=0
)

def digit_in_range(self, text: str) -> bool:
if text.isdecimal():
n = int(text)
return n >= self.min and n <= self.max
else:
return False

class NumberRangeValidator(SkippableValidator):
min: int
max: int

def __init__(self, min: int, max: int, skippable: bool):
super().__init__(skippable)
self.min = min
self.max = max

def validate(self, document):
super().validate(document)
text = document.text
if self.can_skip(text):
return
if not self.number_in_range(text):
raise ValidationError(
message=f"Input must be a number in the range [{self.min},{self.max}]",
cursor_position=0
)

def number_in_range(self, text: str) -> bool:
if text.isdecimal():
n = int(text)
return n >= self.min and n <= self.max
else:
return False

class BooleanValidator(SkippableValidator):
def validate(self, document):
super().validate(document)
text = document.text
if self.can_skip(text):
return
if text not in {"y", "Y", "n", "N"}:
raise ValidationError(
message=f"Input must be a 'Y', 'y', 'N' or 'n'",
cursor_position=0
)

class StringValidator(SkippableValidator):
checker: None

def __init__(self, checker, skippable: bool):
super().__init__(skippable)
self.checker = checker

def validate(self, document):
super().validate(document)
text = document.text
if self.can_skip(text):
return
if not self.checker(text):
raise ValidationError(
message=f"Wrong format",
cursor_position=0
)
Loading
Loading