-
Notifications
You must be signed in to change notification settings - Fork 21
chore: config generator tool #66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
cb015bf
add: main file for conftool.
asllop ec62b17
docs: document questionnaire process.
asllop cb1b1cd
add: base class for yaml to model mapping.
asllop 030e2f7
add: support for lists of objects.
asllop daac841
add: support for enums.
asllop d374ce5
add: support for lists of non base model items.
asllop f5660c9
add: models for config and all subtypes.
asllop e1c5df6
add: model integrity check system.
asllop e7735ba
add: integrity checks for config and service_schedule models.
asllop 37f08bc
update: improve invalid enum error message.
asllop ad9396e
chore: factorize code.
asllop da35027
chore: reorganize project.
asllop 9b5cf90
docs: add conftool project readme.
asllop 3cacaf9
add: support for newtype base models.
asllop 4cfb017
add: validation checks for redis and arguments models.
asllop 72796b9
fix: accidental code.
asllop cbfbd9c
add: checks for inner types of list and dict.
asllop 64ebc71
chore: factorize code.
asllop aec3de0
add: specific dict handler.
asllop 523301c
add: handle None case gracefully.
asllop 84435a2
update: Exception class.
asllop 08cf1fd
add: `newrelic` config key.
asllop 96e4c19
update: newrelic required config.
asllop 338549d
docs: update instance config documentation.
asllop bd3425c
add: instance checks.
asllop 618d198
update: instance arguments checks.
asllop d9a64b1
update: rename newrelic model class.
asllop d381383
add: auth checks. custom enum base class.
asllop bf7a378
update: redis config checks.
asllop a8c7cb5
add: event_type attribute to limits model.
asllop 95e6a6c
add: checks for query config.
asllop e2802a7
chore: split up model classes into different files.
asllop a9a31fa
chore: rename enum module.
asllop f27ab62
chore: split up newrelic model.
asllop 852b85d
update: capture ConfigException for parsing errors.
asllop b2a560f
add: base class for questions.
asllop 3e36e7f
add: skippable questions.
asllop 9b4a540
chore: encapsulate prompt_toolkit-dependant code.
asllop c4e3112
add: prompts for int and bool.
asllop 2db05cc
chore: encapsulate question askers and model.
asllop da67995
add: string questions.
asllop 8253c7b
update: improve skippable validation.
asllop 310ea80
add: raw input questions.
asllop a95a945
add: config model to YAML serialization.
asllop 87e8167
chore: move to_dict function to model package.
asllop b84a222
add: to_yaml method to config model
asllop 39c1f9b
fix: return none from ask methods.
asllop 9c73b32
add: text module.
asllop d23b955
add: instance questions.
asllop af59e95
fix: ask enum, use symbolic variant names to prompt.
asllop 9303518
add: auth questions.
asllop f7932da
chore: refactor.
asllop e509c20
add: cache/redis questions.
asllop 1046421
add: rest of arguments simple questions.
asllop 83772a8
add: question basic questions.
asllop 502fb73
add: ask for query list.
asllop a662928
add: newrelic config questions.
asllop fdeb045
add: read list of IDs.
asllop 90c052c
fix: to YAML con version of "__inner_val__" models.
asllop 74afae1
add: ask a dictionary of values.
asllop b2c4614
add: limits questions.
asllop c68d351
add: instance level service schedule conf.
asllop 6b95d47
fix: NR api endpoint is required.
asllop 127a655
chore: improve messages and remove comments.
asllop f30aa97
add: question hierarchy.
asllop b30f5b2
add: visual styles.
asllop f6a09f2
fix: service_chedule check rules.
asllop 2e80d57
add: auth warning message on config validation.
asllop 0427afb
update: cli arguments.
asllop c44485a
docs: update cli arguments.
asllop fa4648d
chore: reorganize questions.
asllop fa0cb3e
add: write data to file.
asllop 544d572
fix: issue with python 3.13
asllop b08bb63
update: model for token url.
asllop b249053
docs: update readme.
asllop a0dd82f
docs: update README.
asllop 1250df3
fix: review change requests.
asllop 3cef699
chore: refactor id_list_check.
asllop c88ab10
fix: account id and license key aren't required, can be set as env vars.
asllop 673dfc8
chore: remove pass keywords.
asllop 83704da
chore: reorganize code in main.
asllop 79ececd
fix: redis port var names.
asllop File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
*.yml | ||
*.yaml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# ConfTool | ||
|
||
Configuration tool for the New Relic Salesforce Exporter. | ||
|
||
## Requirements | ||
|
||
Tested with Python versions 3.9 and 3.13. | ||
|
||
## Install | ||
|
||
From `conftool` folder: | ||
|
||
- Create a virtual environment ([venv](https://virtualenv.pypa.io/en/latest/installation.html) is required): | ||
|
||
``` | ||
python<version> -m venv my_env | ||
``` | ||
|
||
- Activate the environment: | ||
|
||
``` | ||
source my_env/bin/activate | ||
``` | ||
|
||
- Then install dependencies: | ||
|
||
``` | ||
pip install -r requirements.txt | ||
``` | ||
|
||
## Usage | ||
|
||
From repo's root folder. | ||
|
||
1. To create new config file: | ||
|
||
``` | ||
python -m conftool path/to/config.yml | ||
``` | ||
|
||
1. To validate an existing config file: | ||
|
||
``` | ||
python -m conftool path/to/config.yml --check | ||
sdewitt-newrelic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
VERSION = "0.1.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
from conftool.model.data_format import DataFormatModel | ||
from . import VERSION | ||
from .model.config import ConfigModel | ||
from .model.exception import ConfigException | ||
from .form import questionnaire | ||
from .form.format import print_warning, print_fail, print_ok | ||
from .form.text import t_warning_missing_auth, t_warning_missing_account_id, \ | ||
t_warning_missing_license | ||
|
||
import argparse | ||
import os.path | ||
|
||
TITLE = f"New Relic Salesforce Exporter Config Tool {VERSION}" | ||
DESCRIPTION = f"{TITLE}. Create or check configuration files." | ||
|
||
def main(): | ||
parser = argparse.ArgumentParser(description=DESCRIPTION) | ||
parser.add_argument('config_file', type=str, help='File path. Config YAML file to create or check.') | ||
parser.add_argument('-c', '--check', action='store_true', help='Check configuration.') | ||
args = parser.parse_args() | ||
|
||
print(f"{TITLE}\n") | ||
|
||
if args.check: | ||
check_config(args.config_file) | ||
else: | ||
create_config(args.config_file) | ||
|
||
def create_config(config_file: str): | ||
print("Creating new config file...\n") | ||
|
||
if os.path.isfile(config_file): | ||
print("Error: Config file already exists.") | ||
exit(1) | ||
|
||
try: | ||
conf = questionnaire.run() | ||
except KeyboardInterrupt: | ||
print("\nAborted.") | ||
exit(1) | ||
|
||
conf_yml = conf.to_yaml() | ||
|
||
print_config(conf_yml) | ||
|
||
try: | ||
with open(config_file, "w+") as file: | ||
try: | ||
file.write(conf_yml) | ||
file.close() | ||
except (IOError, OSError): | ||
print("Error: Could not write to file.") | ||
exit(1) | ||
except (FileNotFoundError, PermissionError, OSError): | ||
print("Error: Could not open file.") | ||
exit(1) | ||
|
||
print_ok("Done.") | ||
print() | ||
|
||
def check_config(config_file: str): | ||
print("Validating config file...\n") | ||
|
||
if not os.path.isfile(config_file): | ||
print("Error: Config file doesn't exist.") | ||
exit(1) | ||
|
||
try: | ||
config_yaml_str = read_file(config_file) | ||
except Exception as err: | ||
print("Error opening file:", err) | ||
exit(1) | ||
|
||
try: | ||
config_model = ConfigModel.from_yaml(config_yaml_str) | ||
except ConfigException as err: | ||
print_fail(str(err)) | ||
print() | ||
exit(1) | ||
|
||
for index,i in enumerate(config_model.instances): | ||
if i.arguments.auth is None: | ||
print(f"At instance #{index + 1}:") | ||
print_warning(t_warning_missing_auth) | ||
print() | ||
|
||
if config_model.newrelic.data_format == DataFormatModel.EVENTS and \ | ||
config_model.newrelic.account_id is None: | ||
print_warning(t_warning_missing_account_id) | ||
|
||
if config_model.newrelic.license_key is None: | ||
print_warning(t_warning_missing_license) | ||
|
||
# Serialize model into YAML | ||
serialized_yaml = config_model.to_yaml() | ||
print_config(serialized_yaml) | ||
|
||
print_ok("Validation OK!") | ||
print() | ||
|
||
def read_file(file_name: str) -> str: | ||
with open(file_name) as file: | ||
return file.read() | ||
|
||
def print_config(conf): | ||
print('---- CONFIG FILE:\n') | ||
print(conf) | ||
print('----') | ||
sdewitt-newrelic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
print() | ||
|
||
if __name__ == "__main__": main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
class Color: | ||
BG_RED = '\x1b[41m' | ||
FG_BOLD_RED = '\x1b[1;31m' | ||
|
||
FG_BOLD_WHITE = '\x1b[1;37m' | ||
|
||
BG_BLUE = '\x1b[44m' | ||
|
||
BG_YELLOW = '\x1b[43m' | ||
|
||
FG_BOLD_BLACK = '\x1b[1;30m' | ||
|
||
FG_BOLD_GREEN = '\x1b[1;32m' | ||
|
||
RESET = '\x1b[0m' | ||
|
||
def print_title(msg: str): | ||
print(Color.BG_RED + Color.FG_BOLD_WHITE + ' ' + msg + ' ' + Color.RESET + '\n') | ||
|
||
def print_statement(msg: str): | ||
print(Color.BG_BLUE + Color.FG_BOLD_WHITE + msg + Color.RESET + "\n") | ||
|
||
def print_warning(msg: str): | ||
print(Color.BG_YELLOW + Color.FG_BOLD_BLACK + "WARNING: " + msg + Color.RESET + "\n") | ||
|
||
def print_fail(msg: str): | ||
print(Color.FG_BOLD_RED + "FAILED: " + msg + Color.RESET) | ||
|
||
def print_ok(msg: str): | ||
print(Color.FG_BOLD_GREEN + msg + Color.RESET) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
from prompt_toolkit.validation import Validator, ValidationError | ||
from prompt_toolkit import prompt | ||
|
||
def prompt_list(message: str, options: list[str], required: bool) -> int: | ||
for i,option in enumerate(options): | ||
print(str(i+1) + ") " + str(option)) | ||
validator = ListNumberValidator(1, len(options), not required) | ||
response = prompt(message + " ", validator=validator) | ||
if response is None or response == "": | ||
sdewitt-newrelic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return None | ||
else: | ||
return int(response) | ||
|
||
def prompt_int(message: str, min: int, max: int, required: bool) -> int: | ||
validator = NumberRangeValidator(min, max, not required) | ||
response = prompt(message + " ", validator=validator) | ||
if response is None or response == "": | ||
return None | ||
else: | ||
return int(response) | ||
|
||
def prompt_bool(message: str, required: bool) -> int: | ||
validator = BooleanValidator(not required) | ||
response = prompt(message + " ", validator=validator) | ||
if response is None or response == "": | ||
return None | ||
else: | ||
return response == "y" or response == "Y" | ||
|
||
def prompt_str(message: str, checker, required: bool) -> int: | ||
validator = StringValidator(checker, not required) | ||
response = prompt(message + " ", validator=validator) | ||
if response == "": | ||
return None | ||
else: | ||
return response | ||
|
||
def prompt_any(message: str, required: bool) -> int: | ||
validator = SkippableValidator(not required) | ||
response = prompt(message + " ", validator=validator) | ||
if response == "": | ||
return None | ||
else: | ||
return response | ||
|
||
class SkippableValidator(Validator): | ||
skippable: bool | ||
|
||
def __init__(self, skippable: bool): | ||
self.skippable = skippable | ||
super().__init__() | ||
|
||
def validate(self, document): | ||
text = document.text | ||
if not self.skippable and (text is None or text == ""): | ||
raise ValidationError( | ||
message=f"Input can't be empty", | ||
cursor_position=0 | ||
) | ||
|
||
def can_skip(self, text): | ||
return self.skippable and (text is None or text == "") | ||
|
||
class ListNumberValidator(SkippableValidator): | ||
min: int | ||
max: int | ||
|
||
def __init__(self, min: int, max: int, skippable: bool): | ||
super().__init__(skippable) | ||
self.min = min | ||
self.max = max | ||
|
||
def validate(self, document): | ||
super().validate(document) | ||
text = document.text | ||
sdewitt-newrelic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if self.can_skip(text): | ||
return | ||
if not self.digit_in_range(text): | ||
raise ValidationError( | ||
message="Input must be a number in the list", | ||
cursor_position=0 | ||
) | ||
|
||
def digit_in_range(self, text: str) -> bool: | ||
if text.isdecimal(): | ||
n = int(text) | ||
return n >= self.min and n <= self.max | ||
else: | ||
return False | ||
|
||
class NumberRangeValidator(SkippableValidator): | ||
min: int | ||
max: int | ||
|
||
def __init__(self, min: int, max: int, skippable: bool): | ||
super().__init__(skippable) | ||
self.min = min | ||
self.max = max | ||
|
||
def validate(self, document): | ||
super().validate(document) | ||
text = document.text | ||
if self.can_skip(text): | ||
return | ||
if not self.number_in_range(text): | ||
raise ValidationError( | ||
message=f"Input must be a number in the range [{self.min},{self.max}]", | ||
cursor_position=0 | ||
) | ||
|
||
def number_in_range(self, text: str) -> bool: | ||
if text.isdecimal(): | ||
n = int(text) | ||
return n >= self.min and n <= self.max | ||
else: | ||
return False | ||
|
||
class BooleanValidator(SkippableValidator): | ||
def validate(self, document): | ||
super().validate(document) | ||
text = document.text | ||
if self.can_skip(text): | ||
return | ||
if text not in {"y", "Y", "n", "N"}: | ||
sdewitt-newrelic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
raise ValidationError( | ||
message=f"Input must be a 'Y', 'y', 'N' or 'n'", | ||
cursor_position=0 | ||
) | ||
|
||
class StringValidator(SkippableValidator): | ||
checker: None | ||
|
||
def __init__(self, checker, skippable: bool): | ||
super().__init__(skippable) | ||
self.checker = checker | ||
|
||
def validate(self, document): | ||
super().validate(document) | ||
text = document.text | ||
if self.can_skip(text): | ||
return | ||
if not self.checker(text): | ||
raise ValidationError( | ||
message=f"Wrong format", | ||
cursor_position=0 | ||
) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.