This is a paragraph.
+ + + + """ + + +def save_some_data(example_text: str): + # function requires file_name and data as a string to be sent. + # see documentation for additonal information + save_text(file_name="your-file-name.txt", data=example_text) + + +def open_some_data(the_file_name: str) -> str: + # function requires file_name and a string will be returned + # see documentation for additonal information + result: str = open_text(file_name=the_file_name) + return result + + +if __name__ == "__main__": + save_some_data(example_text) + opened_file: str = open_some_data("your-file-name.txt") + print(opened_file) +``` diff --git a/docs/recipes/emailValidation.md b/docs/examples/validate_emails.md similarity index 71% rename from docs/recipes/emailValidation.md rename to docs/examples/validate_emails.md index 3a8cbde9..6c1fb13f 100644 --- a/docs/recipes/emailValidation.md +++ b/docs/examples/validate_emails.md @@ -1,46 +1,67 @@ -# Validating Email Addresses -Example of how to use in a script +# validate_emails Example + +# Email Validation Example Script + +This module demonstrates how to validate a list of email addresses using various configurations. It leverages the `validate_email_address` function from the `dsg_lib.common_functions.email_validation` module to perform the validation. + +The script is designed to: +- Validate a predefined list of email addresses. +- Use multiple configurations to test different validation scenarios. +- Measure and display the time taken to validate all email addresses. +- Print the validation results in a sorted order for better readability. + +## Features + +- **Email Validation**: Checks the validity of email addresses based on various configurations. +- **Custom Configurations**: Supports multiple validation options such as deliverability checks, allowing quoted local parts, and more. +- **Performance Measurement**: Tracks the time taken to validate all email addresses. +- **Result Sorting**: Outputs the validation results in a sorted format for easier analysis. + +## Usage + +Run the script as a standalone module: + +```bash +$ python validate_emails.py +``` + +## Attributes + +### Email Addresses +A predefined list of email addresses to validate. The list includes: +- Valid email addresses. +- Invalid email addresses. +- Edge cases such as emails with non-ASCII characters, quoted local parts, and domain literals. + +### Configurations +A list of dictionaries, where each dictionary represents a validation configuration. Configuration options include: +- `check_deliverability` (bool): Whether to check if the email address is deliverable. +- `test_environment` (bool): Whether the function is being run in a test environment. +- `allow_smtputf8` (bool): Whether to allow non-ASCII characters in the email address. +- `allow_empty_local` (bool): Whether to allow email addresses with an empty local part. +- `allow_quoted_local` (bool): Whether to allow email addresses with a quoted local part. +- `allow_display_name` (bool): Whether to allow email addresses with a display name. +- `allow_domain_literal` (bool): Whether to allow email addresses with a domain literal. +- `globally_deliverable` (bool): Whether the email address should be globally deliverable. +- `timeout` (int): The timeout for the validation in seconds. +- `dns_type` (str): The type of DNS to use for the validation. Can be `'dns'` or `'timeout'`. + +## Functions + +### `validate_email_address(email: str, **kwargs: dict) -> dict` +Validates an email address using the provided configuration and returns a dictionary with the results. + +## Example Output + +The script outputs the validation results in a sorted order, along with the time taken for the validation process. Each result includes: +- The email address. +- The validation status. +- Additional metadata based on the configuration used. + +## License +This module is licensed under the MIT License. ```python -# -*- coding: utf-8 -*- -""" -This module is used to validate a list of email addresses using various configurations. - -The module imports the `validate_email_address` function from the `dsg_lib.common_functions.email_validation` -module and uses it to validate a list of email addresses. The email addresses and configurations are hard-coded -into the module. - -The module measures the time taken to validate all the email addresses with all the configurations and prints -the results in a sorted order. - -The module can be run as a standalone script. - -Example: - $ python validate_emails.py - -Attributes: - email_addresses (list of str): A list of email addresses to validate. - configurations (list of dict): A list of configurations to use for validation. Each configuration is a - dictionary with the following keys: - - check_deliverability (bool): Whether to check if the email address is deliverable. - - test_environment (bool): Whether the function is being run in a test environment. - - allow_smtputf8 (bool): Whether to allow non-ASCII characters in the email address. - - allow_empty_local (bool): Whether to allow email addresses with an empty local part. - - allow_quoted_local (bool): Whether to allow email addresses with a quoted local part. - - allow_display_name (bool): Whether to allow email addresses with a display name. - - allow_domain_literal (bool): Whether to allow email addresses with a domain literal. - - globally_deliverable (bool): Whether the email address should be globally deliverable. - - timeout (int): The timeout for the validation in seconds. - - dns_type (str): The type of DNS to use for the validation. Can be 'dns' or 'timeout'. - -Functions: - validate_email_address(email: str, **kwargs: dict) -> dict: Validates an email address using the provided - configuration and returns a dictionary with the results. - -Author: Mike Ryan -Date: 2024/05/16 -License: MIT -""" import pprint import time @@ -225,5 +246,4 @@ if __name__ == "__main__": pprint.pprint(v, indent=4) print(f"Time taken: {t1 - t0:.2f}") - ``` diff --git a/docs/index.md b/docs/index.md index d2af5c27..ca53e3a3 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,8 +9,8 @@ Support Python Versions  [](https://github.com/astral-sh/ruff) -[](./reports/coverage/index.html) -[](./reports/coverage/index.html) +[](./reports/coverage/index.html) +[](./reports/coverage/index.html) CI/CD Pipeline: @@ -29,6 +29,7 @@ SonarCloud: # DevSetGo Common Library + `devsetgo_lib` is a versatile library designed to provide common functions for Python applications. Its main goal is to increase reusability and reduce the need to rewrite the same functions across multiple applications. This also allows for quick defect resolution and propagation of fixes across all dependent projects. Read the Full Documentation [here](https://devsetgo.github.io/devsetgo_lib/). @@ -40,6 +41,9 @@ Read the Full Documentation [here](https://devsetgo.github.io/devsetgo_lib/). - **CSV, JSON, and Text File Functions**: Create, read, write, and manipulate various file types with ease. - **Folder Functions**: Create and remove directories, list directory contents, and manage file system operations efficiently. + - **File Moving**: + Move files from one directory to another, with an option to compress the file being moved. + - **Logging**: Comprehensive logging setup using the [Loguru Library]('https://loguru.readthedocs.io/en/stable/overview.html'). Provides extensive customization options for log configuration, including log rotation, retention, and formatting. Includes improvements for multiprocessing environments to ensure log messages are handled correctly across multiple processes. @@ -58,6 +62,14 @@ Read the Full Documentation [here](https://devsetgo.github.io/devsetgo_lib/). - Configuration and management of asynchronous database sessions. - CRUD operations with async support. +## Quick Reference + +- Logging & Config Setup +- FastAPI Endpoints +- Calendar & Date Utilities +- Pattern Matching +- CSV & JSON Helpers + --- ## Installation diff --git a/docs/recipes/asyncDatabase.md b/docs/recipes/asyncDatabase.md deleted file mode 100644 index bcf895fc..00000000 --- a/docs/recipes/asyncDatabase.md +++ /dev/null @@ -1,282 +0,0 @@ -# Examples -Here are a few examples of how to use the database functions - -## Asyncio Script Example -Example of how to use in a script - -```python -import asyncio -from sqlalchemy import select -from dsg_lib.async_database_functions import database_config, async_database, database_operations - -# Configuration -config = { - "database_uri": "sqlite+aiosqlite:///:memory:?cache=shared", - "echo": False, - "future": True, - "pool_recycle": 3600, -} - -# Create a DBConfig instance -db_config = database_config.DBConfig(config) - -# Create an AsyncDatabase instance -async_db = async_database.AsyncDatabase(db_config) - -# Create a DatabaseOperations instance -db_ops = database_operations.DatabaseOperations(async_db) - -# User class -class User(async_db.Base): - __tablename__ = "users" - first_name = Column(String, unique=False, index=True) - last_name = Column(String, unique=False, index=True) - email = Column(String, unique=True, index=True, nullable=True) - -# Async function to get all users -async def get_all_users(): - # Create a select query - query = select(User) - - # Execute the query and fetch all results - users = await db_ops.read_query(query) - - # Print the users - for user in users: - print(f"User: {user.first_name} {user.last_name}, Email: {user.email}") - -# Run the async function -asyncio.run(get_all_users()) - -``` - - -## FastAPI Example - -```python -# -*- coding: utf-8 -*- - -from contextlib import asynccontextmanager - -from fastapi import FastAPI -from fastapi.responses import RedirectResponse -from loguru import logger -from tqdm import tqdm - -from dsg_lib import logging_config - -logging_config.config_log( - logging_level="Debug", log_serializer=False, log_name="log.log" -) - - -@asynccontextmanager -async def lifespan(app: FastAPI): - logger.info("starting up") - # Create the tables in the database - await async_db.create_tables() - - create_users = True - if create_users: - await create_a_bunch_of_users(single_entry=23, many_entries=100) - yield - logger.info("shutting down") - - -# Create an instance of the FastAPI class -app = FastAPI( - title="FastAPI Example", # The title of the API - description="This is an example of a FastAPI application using the DevSetGo Toolkit.", # A brief description of the API - version="0.1.0", # The version of the API - docs_url="/docs", # The URL where the API documentation will be served - redoc_url="/redoc", # The URL where the ReDoc documentation will be served - openapi_url="/openapi.json", # The URL where the OpenAPI schema will be served - debug=True, # Enable debug mode - middleware=[], # A list of middleware to include in the application - routes=[], # A list of routes to include in the application - lifespan=lifespan, -) - - -@app.get("/") -async def root(): - """ - Root endpoint of API - Returns: - Redrects to openapi document - """ - # redirect to openapi docs - logger.info("Redirecting to OpenAPI docs") - response = RedirectResponse(url="/docs") - return response - -from sqlalchemy import Column, Delete, Select, String, Update - -from dsg_lib import ( - async_database, - base_schema, - database_config, - database_operations, -) - -# Create a DBConfig instance -config = { - # "database_uri": "postgresql+asyncpg://postgres:postgres@postgresdb/postgres", - "database_uri": "sqlite+aiosqlite:///:memory:?cache=shared", - "echo": False, - "future": True, - # "pool_pre_ping": True, - # "pool_size": 10, - # "max_overflow": 10, - "pool_recycle": 3600, - # "pool_timeout": 30, -} - -db_config = database_config.DBConfig(config) -# Create an AsyncDatabase instance -async_db = async_database.AsyncDatabase(db_config) - -# Create a DatabaseOperations instance -db_ops = database_operations.DatabaseOperations(async_db) - - -# User class inherits from SchemaBase and async_db.Base -# This class represents the User table in the database -class User(base_schema.SchemaBase, async_db.Base): - __tablename__ = "users" # Name of the table in the database - - # Define the columns of the table - first_name = Column(String, unique=False, index=True) # First name of the user - last_name = Column(String, unique=False, index=True) # Last name of the user - email = Column( - String, unique=True, index=True, nullable=True - ) # Email of the user, must be unique - -async def create_a_bunch_of_users(single_entry=0, many_entries=0): - logger.info(f"single_entry: {single_entry}") - await async_db.create_tables() - # Create a list to hold the user data - - # Create a loop to generate user data - - for i in tqdm(range(single_entry), desc="executing one"): - value = secrets.token_hex(16) - user = User( - first_name=f"First{value}", - last_name=f"Last{value}", - email=f"user{value}@example.com", - ) - logger.info(f"created_users: {user}") - await db_ops.create_one(user) - - users = [] - # Create a loop to generate user data - for i in tqdm(range(many_entries), desc="executing many"): - value_one = secrets.token_hex(4) - value_two = secrets.token_hex(8) - user = User( - first_name=f"First{value_one}{i}{value_two}", - last_name=f"Last{value_one}{i}{value_two}", - email=f"user{value_one}{i}{value_two}@example.com", - ) - logger.info(f"created_users: {user.first_name}") - users.append(user) - - # Use db_ops to add the users to the database - await db_ops.create_many(users) - - -@app.get("/database/get-count") -async def get_count(): - count = await db_ops.count_query(Select(User)) - return {"count": count} - - -# endpoint to get list of user -@app.get("/database/get-all") -async def get_all(offset: int = 0, limit: int = 100): - records = await db_ops.read_query(Select(User).offset(offset).limit(limit)) - return {"records": records} - - -@app.get("/database/get-primary-key") -async def table_primary_key(): - pk = await db_ops.get_primary_keys(User) - return {"pk": pk} - - -@app.get("/database/get-column-details") -async def table_column_details(): - columns = await db_ops.get_columns_details(User) - return {"columns": columns} - - -@app.get("/database/get-tables") -async def table_table_details(): - tables = await db_ops.get_table_names() - return {"table_names": tables} - - -@app.get("/database/get-one-record") -async def get_one_record(record_id: str): - record = await db_ops.get_one_record(Select(User).where(User.pkid == record_id)) - return {"record": record} - -if __name__ == "__main__": - import uvicorn - - uvicorn.run(app, host="127.0.0.1", port=5000) -``` - - -### Configuration Examples - -```python -# SQLite in memory database -config = { - "database_uri": "sqlite+aiosqlite:///:memory:?cache=shared", - "echo": False, - "future": True, - "pool_recycle": 3600, -} -``` - -```python -# PostgreSQL database -config = { - "database_uri": "postgresql+asyncpg://postgres:postgres@postgresdb/postgres", - "echo": False, - "future": True, - "pool_recycle": 3600, -} -``` - -```python -# MySQL database -config = { - "database_uri": "mysql+aiomysql://root:root@localhost/test", - "echo": False, - "future": True, - "pool_recycle": 3600, -} -``` - -```python -# SQL Server database -config = { - "database_uri": "mssql+aiomssql://sa:yourStrong(!)Password@localhost:1433/master", - "echo": False, - "future": True, - "pool_recycle": 3600, -} -``` - -```python -# Oracle database -config = { - "database_uri": "oracle+oracledb_async://scott:tiger@localhost/?service_name=XEPDB1", - "echo": False, - "future": True, - "pool_recycle": 3600, -} -``` diff --git a/docs/recipes/loggingExample.md b/docs/recipes/loggingExample.md deleted file mode 100644 index af5b515c..00000000 --- a/docs/recipes/loggingExample.md +++ /dev/null @@ -1,118 +0,0 @@ -# Logging Example - -```python -# -*- coding: utf-8 -*- -""" -Author: Mike Ryan -Date: 2024/05/16 -License: MIT -""" -import logging -import multiprocessing -import secrets -import threading - -# from loguru import logger -# import logging as logger -from . import logger -from tqdm import tqdm - -from dsg_lib.common_functions import logging_config - -# Configure logging as before -logging_config.config_log( - logging_directory='log', - log_name='log', - logging_level='DEBUG', - log_rotation='100 MB', - log_retention='10 days', - log_backtrace=True, - log_serializer=True, - log_diagnose=True, - # app_name='my_app', - # append_app_name=True, - intercept_standard_logging=True, - enqueue=True, -) - - -# @logger.catch -def div_zero(x, y): - try: - return x / y - except ZeroDivisionError as e: - logger.error(f'{e}') - logging.error(f'{e}') - - -# @logger.catch -def div_zero_two(x, y): - try: - return x / y - except ZeroDivisionError as e: - logger.error(f'{e}') - logging.error(f'{e}') - - -def log_big_string(lqty=100, size=256): - big_string = secrets.token_urlsafe(size) - for _ in range(lqty): - logging.debug(f'Lets make this a big message {big_string}') - div_zero(x=1, y=0) - div_zero_two(x=1, y=0) - # after configuring logging - # use loguru to log messages - logger.debug('This is a loguru debug message') - logger.info('This is an loguru info message') - logger.error('This is an loguru error message') - logger.warning('This is a loguru warning message') - logger.critical('This is a loguru critical message') - - # will intercept all standard logging messages also - logging.debug('This is a standard logging debug message') - logging.info('This is an standard logging info message') - logging.error('This is an standard logging error message') - logging.warning('This is a standard logging warning message') - logging.critical('This is a standard logging critical message') - - -def worker(wqty=1000, lqty=100, size=256): - for _ in tqdm(range(wqty), ascii=True, leave=True): # Adjusted for demonstration - log_big_string(lqty=lqty, size=size) - - -def main(wqty: int = 100, lqty: int = 10, size: int = 256, workers: int = 16, thread_test: bool = False, process_test: bool = False): - if process_test: - processes = [] - # Create worker processes - for _ in tqdm(range(workers), desc="Multi-Processing Start", leave=True): - p = multiprocessing.Process( - target=worker, args=(wqty, lqty, size,)) - processes.append(p) - p.start() - - for p in tqdm((processes), desc="Multi-Processing Start", leave=False): - p.join(timeout=60) # Timeout after 60 seconds - if p.is_alive(): - logger.error(f"Process {p.name} is hanging. Terminating.") - p.terminate() - p.join() - - if thread_test: - threads = [] - for _ in tqdm(range(workers), desc="Threading Start", leave=True): # Create worker threads - t = threading.Thread(target=worker, args=(wqty, lqty, size,)) - threads.append(t) - t.start() - - for t in tqdm(threads, desc="Threading Gather", leave=False): - t.join() - - -if __name__ == "__main__": - from time import time - start = time() - main(wqty=5, lqty=50, size=64, workers=8, thread_test=False, process_test=True) - print(f"Execution time: {time()-start:.2f} seconds") - -``` diff --git a/docs/release-notes.md b/docs/release-notes.md index 1a464213..5f78c94c 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -4,6 +4,37 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## Latest Changes +### New Functions and Updated Documentation ([v2025.4.5.1](https://github.com/devsetgo/devsetgo_lib/releases/tag/v2025.4.5.1)) + +#### What's Changed +* Fix of Calendar Version Pattern (#485) @devsetgo +* Working on Improving Documentation and Tests (#474) @devsetgo +* pip(deps): bump pytest-asyncio from 0.25.3 to 0.26.0 (#480) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump mkdocstrings[python,shell] from 0.27.0 to 0.29.1 (#481) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump black from 24.10.0 to 25.1.0 (#482) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump ruff from 0.9.9 to 0.11.2 (#483) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pre-commit from 4.0.1 to 4.2.0 (#484) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pytest-asyncio from 0.25.0 to 0.25.3 (#475) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump ruff from 0.9.4 to 0.9.9 (#476) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump flake8 from 7.1.1 to 7.1.2 (#477) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump twine from 6.0.1 to 6.1.0 (#478) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump structlog from 24.4.0 to 25.1.0 (#479) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pymdown-extensions from 10.13 to 10.14.3 (#473) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump python-json-logger from 2.0.7 to 3.2.1 (#469) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump ruff from 0.8.1 to 0.9.4 (#470) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pytest from 8.3.3 to 8.3.4 (#471) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pygments from 2.18.0 to 2.19.1 (#472) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump loguru from 0.7.2 to 0.7.3 (#463) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump hatchling from 1.26.3 to 1.27.0 (#464) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump bumpcalver from 2024.11.8 to 2024.12.14.1 (#466) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pymdown-extensions from 10.12 to 10.13 (#465) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump pytest-asyncio from 0.24.0 to 0.25.0 (#467) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump mkdocs-material from 9.5.46 to 9.5.47 (#460) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump twine from 5.1.1 to 6.0.1 (#461) @[dependabot[bot]](https://github.com/apps/dependabot) +* pip(deps): bump ruff from 0.8.0 to 0.8.1 (#462) @[dependabot[bot]](https://github.com/apps/dependabot) + +Published Date: 2025 April 05, 20:56 + ### Adding new db functions ([v2024.11.28.1](https://github.com/devsetgo/devsetgo_lib/releases/tag/v2024.11.28.1)) #### What's Changed @@ -569,3 +600,359 @@ Major changes are in PR #304 Published Date: 2023 April 01, 00:27 + +### Open CSV enhancements and library updates ([v0.9.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.9.0)) + +# What's Changed +* fix of latest changes (#288) @devsetgo +* Open_CSV Enhancements (#287) @devsetgo +* pip(deps): bump pytest-cov from 3.0.0 to 4.0.0 (#274) @dependabot +* pip(deps): bump mkdocs-material from 8.4.2 to 8.5.5 (#276) @dependabot +* pip(deps): bump autoflake from 1.5.3 to 1.6.1 (#275) @dependabot +* pip(deps): bump tqdm from 4.64.0 to 4.64.1 (#273) @dependabot +* pip(deps): bump pytest from 7.1.2 to 7.1.3 (#272) @dependabot +* pip(deps): bump mkdocs from 1.3.1 to 1.4.0 (#271) @dependabot +* pip(deps): bump tox from 3.25.1 to 3.26.0 (#269) @dependabot +* pip(deps): bump pylint from 2.15.0 to 2.15.3 (#270) @dependabot +* pip(deps): bump mkdocs-material from 8.3.9 to 8.4.2 (#268) @dependabot +* pip(deps): bump autopep8 from 1.6.0 to 1.7.0 (#264) @dependabot +* pip(deps): bump pylint from 2.14.5 to 2.15.0 (#265) @dependabot +* pip(deps): bump autoflake from 1.4 to 1.5.3 (#263) @dependabot +* pip(deps): bump black from 22.6.0 to 22.8.0 (#267) @dependabot +* pip(deps): bump flake8 from 5.0.1 to 5.0.4 (#266) @dependabot +* pip(deps): bump pre-commit from 2.19.0 to 2.20.0 (#260) @dependabot +* pip(deps): bump mkdocs from 1.3.0 to 1.3.1 (#261) @dependabot +* pip(deps): bump flake8 from 4.0.1 to 5.0.1 (#259) @dependabot +* pip(deps): bump mkdocs-material from 8.3.8 to 8.3.9 (#258) @dependabot +* pip(deps): bump pylint from 2.14.4 to 2.14.5 (#262) @dependabot +* pip(deps): bump twine from 4.0.0 to 4.0.1 (#252) @dependabot +* pip(deps): bump pylint from 2.14.0 to 2.14.4 (#251) @dependabot +* pip(deps): bump mkdocs-material from 8.2.16 to 8.3.8 (#253) @dependabot +* pip(deps): bump black from 22.3.0 to 22.6.0 (#254) @dependabot +* pip(deps): bump tox from 3.25.0 to 3.25.1 (#255) @dependabot +* pip(deps): bump watchdog from 2.1.8 to 2.1.9 (#256) @dependabot +* github actionts(deps): bump actions/setup-python from 3 to 4 (#257) @dependabot +* pip(deps): bump pylint from 2.13.7 to 2.14.0 (#250) @dependabot +* pip(deps): bump watchdog from 2.1.7 to 2.1.8 (#246) @dependabot +* pip(deps): bump pre-commit from 2.18.1 to 2.19.0 (#248) @dependabot +* pip(deps): bump mkdocs-material from 8.2.12 to 8.2.16 (#249) @dependabot +* pip(deps): bump tox from 3.24.5 to 3.25.0 (#242) @dependabot +* pip(deps): bump pre-commit from 2.17.0 to 2.18.1 (#243) @dependabot +* pip(deps): bump click from 8.1.2 to 8.1.3 (#245) @dependabot +* pip(deps): bump pylint from 2.13.4 to 2.13.7 (#240) @dependabot +* pip(deps): bump tqdm from 4.63.1 to 4.64.0 (#244) @dependabot +* pip(deps): bump mkdocs-material from 8.2.8 to 8.2.12 (#241) @dependabot +* pip(deps): bump pytest from 7.1.1 to 7.1.2 (#239) @dependabot +* pip(deps): bump watchdog from 2.1.6 to 2.1.7 (#238) @dependabot +* pip(deps): bump pylint from 2.12.2 to 2.13.4 (#237) @dependabot +* pip(deps): bump mkdocs from 1.2.3 to 1.3.0 (#234) @dependabot +* pip(deps): bump tqdm from 4.63.0 to 4.63.1 (#233) @dependabot +* pip(deps): bump black from 22.1.0 to 22.3.0 (#236) @dependabot +* pip(deps): bump pytest from 7.0.1 to 7.1.1 (#231) @dependabot +* pip(deps): bump click from 8.0.4 to 8.1.2 (#235) @dependabot +* pip(deps): bump mkdocs-material from 8.2.5 to 8.2.8 (#232) @dependabot +* pip(deps): bump twine from 3.8.0 to 4.0.0 (#230) @dependabot +* document updates (#229) @devsetgo + + +Published Date: 2022 December 04, 16:55 + +### Additional Logging Configuration ([v0.8.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.8.0)) + +# What's Changed +* New Logging Configuration items (#228) @devsetgo +* pip(deps): bump tqdm from 4.62.3 to 4.63.0 (#224) @dependabot +* pip(deps): bump mkdocs-material from 8.2.3 to 8.2.4 (#227) @dependabot +* github actionts(deps): bump actions/setup-python from 2.3.1 to 3 (#226) @dependabot +* pip(deps): bump mkdocs-material from 8.1.9 to 8.2.3 (#225) @dependabot +* pip(deps): bump twine from 3.7.1 to 3.8.0 (#223) @dependabot +* pip(deps): bump pytest from 6.2.5 to 7.0.1 (#222) @dependabot +* pip(deps): bump pytest-runner from 5.3.1 to 6.0.0 (#221) @dependabot +* pip(deps): bump loguru from 0.5.3 to 0.6.0 (#218) @dependabot +* pip(deps): bump black from 21.12b0 to 22.1.0 (#219) @dependabot +* pip(deps): bump mkdocs-material from 8.1.8 to 8.1.9 (#220) @dependabot + + +Published Date: 2022 March 12, 21:07 + +### ([v0.7.1](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.7.1)) + +# What's Changed +* Bump version: 0.7.0 → 0.7.1 (#217) @devsetgo +* Hotfix for setup file (#216) @devsetgo + + +Published Date: 2022 January 29, 01:51 + +### Logging to Beta Testing ([v0.7.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.7.0)) + +Logging is now has basic unit tests and is more ready to use with live application. + +# What's Changed +* Adding Logging Config (#215) @devsetgo +* pip(deps): bump pre-commit from 2.15.0 to 2.16.0 (#210) @dependabot +* pip(deps): bump pylint from 2.12.1 to 2.12.2 (#211) @dependabot +* pip(deps): bump tox from 3.24.4 to 3.24.5 (#212) @dependabot +* pip(deps): bump black from 21.11b1 to 21.12b0 (#213) @dependabot +* pip(deps): bump twine from 3.6.0 to 3.7.1 (#214) @dependabot +* pip(deps): bump twine from 3.5.0 to 3.6.0 (#204) @dependabot +* pip(deps): bump coverage-badge from 1.0.2 to 1.1.0 (#205) @dependabot +* pip(deps): bump mkdocs-material from 7.3.6 to 8.0.2 (#206) @dependabot +* pip(deps): bump pylint from 2.11.1 to 2.12.1 (#207) @dependabot +* pip(deps): bump black from 21.10b0 to 21.11b1 (#208) @dependabot +* github actionts(deps): bump actions/setup-python from 2.2.2 to 2.3.1 (#209) @dependabot +* Dev (#203) @devsetgo +* pip(deps): bump tox from 3.24.3 to 3.24.4 (#193) @dependabot +* pip(deps): bump tqdm from 4.62.2 to 4.62.3 (#194) @dependabot +* pip(deps): bump pylint from 2.10.2 to 2.11.1 (#195) @dependabot +* pip(deps): bump mkdocs-material from 7.2.6 to 7.3.0 (#196) @dependabot +* pip(deps): bump black from 21.8b0 to 21.9b0 (#197) @dependabot +* pip(deps): bump mkdocs-material from 7.2.4 to 7.2.6 (#189) @dependabot +* pip(deps): bump pytest from 6.2.4 to 6.2.5 (#191) @dependabot +* pip(deps): bump watchdog from 2.1.3 to 2.1.5 (#192) @dependabot +* pip(deps): bump tox from 3.24.1 to 3.24.3 (#190) @dependabot +* pip(deps): bump pre-commit from 2.14.0 to 2.15.0 (#188) @dependabot +* pip(deps): bump black from 21.7b0 to 21.8b0 (#187) @dependabot +* pip(deps): bump pylint from 2.9.6 to 2.10.2 (#184) @dependabot +* pip(deps): bump tqdm from 4.62.0 to 4.62.2 (#185) @dependabot +* github actionts(deps): bump actions/setup-python from 1 to 2.2.2 (#182) @dependabot +* Bump wheel from 0.36.2 to 0.37.0 (#180) @dependabot +* Bump mkdocs-material from 7.2.2 to 7.2.4 (#181) @dependabot +* Bump tox from 3.24.0 to 3.24.1 (#177) @dependabot +* Bump mkdocs-material from 7.2.1 to 7.2.2 (#178) @dependabot +* Bump pre-commit from 2.13.0 to 2.14.0 (#179) @dependabot +* Bump pylint from 2.9.5 to 2.9.6 (#176) @dependabot +* Bump tqdm from 4.61.2 to 4.62.0 (#175) @dependabot +* Bump mkdocs-material from 7.1.10 to 7.2.1 (#174) @dependabot +* Bump twine from 3.4.1 to 3.4.2 (#171) @dependabot +* Bump pylint from 2.9.3 to 2.9.5 (#170) @dependabot +* Bump mkdocs from 1.2.1 to 1.2.2 (#173) @dependabot +* documentation update (#169) @devsetgo +* README fix (#168) @devsetgo + + +Published Date: 2022 January 29, 01:42 + +### Logging Configuration ([v0.6.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.6.0)) + +# What's Changed +* Adding Logging and Cleanup (#167) @devsetgo +* Bump tqdm from 4.61.1 to 4.61.2 (#166) @dependabot +* Bump pylint from 2.8.3 to 2.9.3 (#165) @dependabot +* Bump watchdog from 2.1.2 to 2.1.3 (#164) @dependabot +* Bump mkdocs-material from 7.1.8 to 7.1.9 (#163) @dependabot +* Bump tqdm from 4.61.0 to 4.61.1 (#162) @dependabot +* Bump mkdocs-material from 7.1.7 to 7.1.8 (#161) @dependabot +* Bump mkdocs from 1.1.2 to 1.2.1 (#159) @dependabot +* Bump black from 21.5b2 to 21.6b0 (#158) @dependabot +* Bump mkdocs-material from 7.1.6 to 7.1.7 (#160) @dependabot +* Bump pytest-cov from 2.12.0 to 2.12.1 (#154) @dependabot +* Bump pylint from 2.8.2 to 2.8.3 (#155) @dependabot +* Bump black from 21.5b1 to 21.5b2 (#156) @dependabot +* Bump mkdocs-material from 7.1.5 to 7.1.6 (#157) @dependabot +* Bump tqdm from 4.60.0 to 4.61.0 (#153) @dependabot +* Bump pre-commit from 2.12.1 to 2.13.0 (#151) @dependabot +* Bump pytest-runner from 5.3.0 to 5.3.1 (#152) @dependabot +* Bump mkdocs-material from 7.1.4 to 7.1.5 (#150) @dependabot +* Bump watchdog from 2.1.1 to 2.1.2 (#149) @dependabot +* Bump click from 7.1.2 to 8.0.1 (#148) @dependabot +* Bump black from 21.5b0 to 21.5b1 (#147) @dependabot +* Bump watchdog from 2.1.0 to 2.1.1 (#146) @dependabot +* Bump pytest-cov from 2.11.1 to 2.12.0 (#145) @dependabot +* Bump flake8 from 3.9.1 to 3.9.2 (#143) @dependabot +* Bump pytest from 6.2.3 to 6.2.4 (#139) @dependabot +* Bump watchdog from 2.0.3 to 2.1.0 (#138) @dependabot +* Bump black from 21.4b2 to 21.5b0 (#140) @dependabot +* Bump mkdocs-material from 7.1.3 to 7.1.4 (#141) @dependabot +* Dev (#142) @devsetgo +* Bump tox from 3.23.0 to 3.23.1 (#137) @dependabot +* Bump autopep8 from 1.5.6 to 1.5.7 (#136) @dependabot +* Bump pylint from 2.7.4 to 2.8.2 (#135) @dependabot +* Bump black from 20.8b1 to 21.4b2 (#134) @dependabot +* Bump mkdocs-material from 7.1.2 to 7.1.3 (#133) @dependabot +* Adding SonarCloud Code Coverage (#130) @devsetgo +* Bump mkdocs-material from 7.1.1 to 7.1.2 (#132) @dependabot +* Bump watchdog from 2.0.2 to 2.0.3 (#131) @dependabot +* Bump pre-commit from 2.12.0 to 2.12.1 (#129) @dependabot +* Bump flake8 from 3.9.0 to 3.9.1 (#128) @dependabot +* Bump mkdocs-material from 7.1.0 to 7.1.1 (#127) @dependabot +* Bump tqdm from 4.59.0 to 4.60.0 (#124) @dependabot +* Bump pytest from 6.2.2 to 6.2.3 (#125) @dependabot +* Bump pre-commit from 2.11.1 to 2.12.0 (#126) @dependabot +* Bump pylint from 2.7.2 to 2.7.4 (#122) @dependabot +* Bump mkdocs-material from 7.0.6 to 7.1.0 (#123) @dependabot +* Bump mkdocs-material from 7.0.5 to 7.0.6 (#121) @dependabot +* Bump flake8 from 3.8.4 to 3.9.0 (#120) @dependabot +* Bump twine from 3.3.0 to 3.4.1 (#118) @dependabot +* Bump autopep8 from 1.5.5 to 1.5.6 (#119) @dependabot + + +Published Date: 2021 July 16, 23:44 + +### Fixing Publish ([v0.5.0-2](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.5.0-2)) + +# What's Changed +* adding update for publish (#117) @devsetgo + + +Published Date: 2021 March 18, 17:19 + +### Calendar and RegEx Function + Documentation ([v0.5.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.5.0)) + +# What's Changed +* Adding Calendar Functions (#116) @devsetgo +* Bump pre-commit from 2.10.1 to 2.11.1 (#113) @dependabot +* update to Saturday (#115) @devsetgo +* Bump tqdm from 4.58.0 to 4.59.0 (#112) @dependabot +* Bump mkdocs-material from 7.0.4 to 7.0.5 (#114) @dependabot +* fixes for mkdoc material update (#111) @devsetgo +* Bump tox from 3.22.0 to 3.23.0 (#109) @dependabot +* Bump mkdocs-material from 7.0.2 to 7.0.4 (#108) @dependabot +* Bump pylint from 2.7.1 to 2.7.2 (#107) @dependabot +* Bump coverage from 5.4 to 5.5 (#110) @dependabot +* Bump pylint from 2.6.2 to 2.7.1 (#103) @dependabot +* Bump mkdocs-material from 6.2.8 to 7.0.2 (#104) @dependabot +* Bump watchdog from 2.0.1 to 2.0.2 (#105) @dependabot +* Bump tqdm from 4.57.0 to 4.58.0 (#106) @dependabot +* Bump tox from 3.21.4 to 3.22.0 (#101) @dependabot +* Bump watchdog from 2.0.0 to 2.0.1 (#99) @dependabot +* Bump pylint from 2.6.0 to 2.6.2 (#102) @dependabot +* Bump tqdm from 4.56.2 to 4.57.0 (#100) @dependabot +* Bump pytest-runner from 5.2 to 5.3.0 (#98) @dependabot +* Bump tqdm from 4.56.0 to 4.56.2 (#97) @dependabot +* Bump watchdog from 1.0.2 to 2.0.0 (#96) @dependabot +* Bump pre-commit from 2.10.0 to 2.10.1 (#95) @dependabot +* Bump mkdocs-material from 6.2.6 to 6.2.8 (#94) @dependabot +* Bump tox from 3.21.3 to 3.21.4 (#93) @dependabot +* Bump autopep8 from 1.5.4 to 1.5.5 (#92) @dependabot +* Bump tox from 3.21.2 to 3.21.3 (#87) @dependabot +* Bump mkdocs-material from 6.2.5 to 6.2.6 (#88) @dependabot +* Bump pytest from 6.2.1 to 6.2.2 (#89) @dependabot +* Bump coverage from 5.3.1 to 5.4 (#91) @dependabot +* Bump pre-commit from 2.9.3 to 2.10.0 (#90) @dependabot +* Bump tox from 3.21.1 to 3.21.2 (#84) @dependabot +* Bump mkdocs-material from 6.2.4 to 6.2.5 (#85) @dependabot +* Bump pytest-cov from 2.10.1 to 2.11.1 (#86) @dependabot +* Bump tox from 3.20.1 to 3.21.1 (#81) @dependabot +* Bump mkdocs-material from 6.2.3 to 6.2.4 (#82) @dependabot +* Bump tqdm from 4.55.1 to 4.56.0 (#83) @dependabot +* Bump tqdm from 4.55.0 to 4.55.1 (#80) @dependabot +* Bump mkdocs-material from 6.2.2 to 6.2.3 (#79) @dependabot + + +Published Date: 2021 March 18, 17:06 + +### Minor updates and library updates. ([v0.4.1](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.4.1)) + +# What's Changed +* Updates and Minor updates (#78) @devsetgo +* Bump tqdm from 4.54.1 to 4.55.0 (#77) @dependabot +* Bump twine from 3.2.0 to 3.3.0 (#76) @dependabot +* Bump coverage from 5.3 to 5.3.1 (#74) @dependabot +* Bump mkdocs-material from 6.1.7 to 6.2.2 (#75) @dependabot +* Bump watchdog from 0.10.4 to 1.0.2 (#73) @dependabot +* Bump pytest from 6.1.2 to 6.2.1 (#71) @dependabot +* Bump wheel from 0.36.1 to 0.36.2 (#70) @dependabot +* Bump tqdm from 4.54.0 to 4.54.1 (#67) @dependabot +* Bump mkdocs-material from 6.1.6 to 6.1.7 (#68) @dependabot +* Bump pre-commit from 2.9.2 to 2.9.3 (#69) @dependabot +* Bump wheel from 0.36.0 to 0.36.1 (#66) @dependabot +* Bump wheel from 0.35.1 to 0.36.0 (#64) @dependabot +* Bump tqdm from 4.53.0 to 4.54.0 (#65) @dependabot +* Bump pre-commit from 2.8.2 to 2.9.2 (#61) @dependabot +* Bump mkdocs-material from 6.1.5 to 6.1.6 (#60) @dependabot +* Bump tqdm from 4.52.0 to 4.53.0 (#62) @dependabot +* Bump watchdog from 0.10.3 to 0.10.4 (#63) @dependabot +* Bump tqdm from 4.51.0 to 4.52.0 (#59) @dependabot +* Bump mkdocs-material from 6.1.4 to 6.1.5 (#58) @dependabot +* Bump mkdocs-material from 6.1.2 to 6.1.4 (#57) @dependabot +* Bump pre-commit from 2.8.0 to 2.8.2 (#55) @dependabot +* Bump mkdocs-material from 6.1.0 to 6.1.2 (#56) @dependabot +* Bump pytest from 6.1.1 to 6.1.2 (#52) @dependabot +* Bump pre-commit from 2.7.1 to 2.8.0 (#53) @dependabot +* Bump tqdm from 4.50.2 to 4.51.0 (#54) @dependabot +* Bump mkdocs-material from 6.0.2 to 6.1.0 (#51) @dependabot +* Bump tqdm from 4.50.1 to 4.50.2 (#49) @dependabot +* Bump tox from 3.20.0 to 3.20.1 (#50) @dependabot +* Bump pytest from 6.1.0 to 6.1.1 (#48) @dependabot +* Bump mkdocs-material from 6.0.1 to 6.0.2 (#47) @dependabot +* Bump flake8 from 3.8.3 to 3.8.4 (#45) @dependabot +* Bump tqdm from 4.50.0 to 4.50.1 (#44) @dependabot +* Bump bump2version from 1.0.0 to 1.0.1 (#46) @dependabot +* Bump tqdm from 4.49.0 to 4.50.0 (#42) @dependabot +* Bump black from 19.10b0 to 20.8b1 (#43) @dependabot +* Bump tqdm from 4.46.0 to 4.49.0 (#40) @dependabot +* Bump pytest from 5.4.2 to 6.1.0 (#39) @dependabot +* Bump coverage from 5.1 to 5.3 (#38) @dependabot +* Bump autoflake from 1.3.1 to 1.4 (#41) @dependabot +* Bump twine from 3.1.1 to 3.2.0 (#37) @dependabot +* Bump wheel from 0.34.2 to 0.35.1 (#34) @dependabot +* Bump pytest-cov from 2.9.0 to 2.10.1 (#36) @dependabot +* Bump watchdog from 0.10.2 to 0.10.3 (#35) @dependabot +* Bump mkdocs-material from 5.2.2 to 6.0.1 (#33) @dependabot +* Bump pylint from 2.5.2 to 2.6.0 (#32) @dependabot-preview +* Bump pre-commit from 2.4.0 to 2.7.1 (#31) @dependabot-preview +* Bump tox from 3.15.1 to 3.20.0 (#30) @dependabot-preview +* Bump flake8 from 3.8.2 to 3.8.3 (#29) @dependabot-preview +* Bump autopep8 from 1.5.2 to 1.5.4 (#28) @dependabot-preview + + +Published Date: 2020 December 26, 23:51 + +### 0.4.0 - save_csv options ([v0.4.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.4.0)) + +## [0.4.0] - Examples and Data +### Added +- skipping version 0.3.0 and adding to 0.4.0 +- Adding delimiter option to save_csv + - Tests to check if delimiter > 1 character + - set ',' if none +- Adding quotechar option to save_csv +- Tests to check if quotechar > 1 character + - set '"' if none +- Add test of non-list to save_csv + +## [0.3.0] - Examples and Data +### Added +- Adding examples (see examples folder) +- Adding file_function documentation +- Adding documents site - https://devsetgo.github.io/devsetgo_lib/ + + +Published Date: 2020 April 16, 21:54 + +### Improvements ([v0.2.0](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.2.0)) + +- Improved Tests +- Improved Errors +- Adding more logging + +Published Date: 2020 January 26, 21:08 + +### v0.1.1 ([v0.1.1](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.1.1)) + +- New documentation +- fixes to pypi deployment + + + + +Published Date: 2020 January 26, 17:26 + +### Beta Release ([v0.1.0b2](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.1.0b2)) + +Basic Function (file and folder) +Publish to PyPi (fixing PyPi publishing issues) +Needs documentation. + + +Published Date: 2020 January 26, 13:03 + +### Pypi Beta Release ([v0.1.0b](https://github.com/devsetgo/devsetgo_lib/releases/tag/v0.1.0b)) + +Change to semantic versioning +- Publish to Pypi +- Base Functions + + +Published Date: 2020 January 26, 12:53 diff --git a/examples/cal_example.py b/examples/cal_example.py index ec84c3e5..0f08f68c 100644 --- a/examples/cal_example.py +++ b/examples/cal_example.py @@ -1,8 +1,79 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# Overview + +This module demonstrates the usage of the `calendar_functions` module from the `dsg_lib.common_functions` package. +It provides examples of how to work with months, both by their numeric representation and their names. + +The module includes two main functions: + +1. **`calendar_check_number`**: + - Iterates through a predefined list of month numbers (`month_list`) and uses the `get_month` function from `calendar_functions` to retrieve the corresponding month name. + - It then prints the result for each number in the list. + - Example: + - Input: `1` + - Output: `"January"` + - Input: `13` (invalid) + - Output: Depends on the implementation of `get_month` (e.g., `"Invalid Month"`). + +2. **`calendar_check_name`**: + - Iterates through a predefined list of month names (`month_names`) and uses the `get_month_number` function from `calendar_functions` to retrieve the corresponding numeric representation of the month. + - It then prints the result for each name in the list. + - Example: + - Input: `"january"` + - Output: `1` + - Input: `"bob"` (invalid) + - Output: Depends on the implementation of `get_month_number` (e.g., `"Invalid Month Name"`). + +# Features + +- **Validation of Inputs**: + The module demonstrates how to handle invalid inputs, such as: + - Numbers outside the valid range of months (1-12). + - Invalid month names that do not correspond to any recognized month. + +- **Testing and Debugging**: + This module can be used to test and validate the robustness of the `calendar_functions` module by providing a variety of valid and invalid inputs. + +# Usage + +- Run the script directly to see the output of the two functions. +- Modify the `month_list` or `month_names` variables to test with different inputs. + +# Dependencies + +- **`dsg_lib.common_functions.calendar_functions`**: + - This module must be available and contain the following functions: + 1. `get_month`: Accepts a numeric month (e.g., `1`) and returns the corresponding month name (e.g., `"January"`). + 2. `get_month_number`: Accepts a month name (e.g., `"january"`) and returns the corresponding numeric representation (e.g., `1`). + +# Example Output + +## For `calendar_check_number`: +If `month_list = [0, 1, 2, 3, 13]`, the output might be: +``` +Invalid Month +January +February +March +Invalid Month +``` + +## For `calendar_check_name`: +If `month_names = ["january", "february", "bob"]`, the output might be: +``` +1 +2 +Invalid Month Name +``` + +# Notes + +- Ensure that the `calendar_functions` module is correctly implemented and imported. +- The behavior for invalid inputs depends on the implementation of `get_month` and `get_month_number`. + +## License +This module is licensed under the MIT License. """ from dsg_lib.common_functions import calendar_functions diff --git a/examples/csv_example.py b/examples/csv_example.py index e3bd9919..ec6cd8a8 100644 --- a/examples/csv_example.py +++ b/examples/csv_example.py @@ -1,8 +1,59 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# CSV Example Module + +This module provides examples of how to work with CSV files using the `dsg_lib` library. It includes functions for saving data to a CSV file, opening and reading data from a CSV file, and creating sample files for testing purposes. The module is designed to demonstrate the usage of the `file_functions` and `logging_config` utilities provided by `dsg_lib`. + +## Functions + +### `save_some_data(example_list: list)` +Saves a list of data to a CSV file. The function uses the `save_csv` utility from `dsg_lib` to write the data to a file. The file is saved with a specified delimiter and quote character. + +- **Parameters**: + - `example_list` (list): A list of lists containing the data to be saved. +- **Notes**: + - The file is saved in the `/data` directory with the name `your-file-name.csv`. + - The delimiter used is `|`, and the quote character is `"`. + - Refer to the `save_csv` documentation for additional options. + +### `open_some_data(the_file_name: str) -> dict` +Opens a CSV file and returns its contents as a dictionary. This function assumes the CSV file has a header row and uses the `open_csv` utility from `dsg_lib`. + +- **Parameters**: + - `the_file_name` (str): The name of the CSV file to open. +- **Returns**: + - `dict`: A dictionary representation of the CSV file's contents. +- **Notes**: + - Additional options such as delimiter, quote level, and space handling can be configured. + - Refer to the Python CSV documentation for more details: [Python CSV Documentation](https://docs.python.org/3/library/csv.html). + +### `sample_files()` +Creates sample files for testing purposes. This function uses the `create_sample_files` utility from `dsg_lib`. + +- **Notes**: + - The sample file is named `test_sample` and contains 1000 rows of data. + +## Example Usage + +```python +if __name__ == "__main__": + # Save example data to a CSV file + save_some_data(example_list) + + # Open and read data from a CSV file + opened_file = open_some_data("your-file-name.csv") + print(opened_file) + + # Create sample files for testing + sample_files() +``` + +## Logging + +The module configures logging using the `config_log` utility from `dsg_lib`. The logging level is set to `DEBUG` to provide detailed information during execution. + +## License +This module is licensed under the MIT License. """ from dsg_lib.common_functions.file_functions import ( create_sample_files, diff --git a/examples/csv_example_with_timer.py b/examples/csv_example_with_timer.py index 193e5934..7a0e9ee2 100644 --- a/examples/csv_example_with_timer.py +++ b/examples/csv_example_with_timer.py @@ -1,8 +1,54 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# CSV Example with Timer + +This module demonstrates how to generate and save CSV files at regular intervals using Python. +It includes functionality to create sample data, save it to a CSV file, and repeat the process +indefinitely with a specified delay. + +## Features + +- **Dynamic Data Generation**: The `create_sample_list` function generates a list of lists + with a customizable number of rows. Each row contains sample data with predefined headers. + +- **Automated File Saving**: The `save_data_with_timer` function saves the generated data + to a CSV file every 5 seconds. Each file is uniquely named with a timestamp to avoid + overwriting. + +- **Customizable CSV Format**: The CSV files are saved with a pipe (`|`) as the delimiter + and double quotes (`"`) as the quote character. + +- **Logging Support**: The module uses a logging configuration to provide debug-level + logging for better traceability. + +## Use Case + +This module is ideal for scenarios where continuous data generation and saving are required, +such as testing, simulations, or data pipeline prototyping. + +## Directory Structure + +The generated CSV files are saved in the following directory: +``` +/workspaces/devsetgo_lib/data/move/source +``` + +## How to Run + +To execute the script, simply run it as a standalone program: +```bash +python csv_example_with_timer.py +``` + +The script will continuously generate and save CSV files until manually stopped. + +## Dependencies + +- `dsg_lib.common_functions.file_functions.save_csv`: A utility function to save data to a CSV file. +- `dsg_lib.common_functions.logging_config.config_log`: A utility function to configure logging. + +## License +This module is licensed under the MIT License. """ import time from datetime import datetime @@ -37,6 +83,19 @@ def create_sample_list(qty=10): def save_data_with_timer(): """ Saves a new CSV file every 5 seconds with a unique timestamped name. + + This function generates a sample list of data with a random number of rows + (between 10 and 100,000) using the `create_sample_list` function. It then + saves this data to a CSV file in the specified directory. The file name + includes a timestamp to ensure uniqueness. The CSV file is saved with a + pipe (`|`) as the delimiter and double quotes (`"`) as the quote character. + + The process repeats indefinitely, with a 5-second delay between each file + creation. This function is useful for testing or simulating scenarios where + data is continuously generated and saved to disk. + + The saved files are stored in the `/workspaces/devsetgo_lib/data/move/source` + directory. """ while True: example_list = create_sample_list(qty=random.randint(10, 100000)) diff --git a/examples/fastapi_example.py b/examples/fastapi_example.py index e843d812..1ed9cdf6 100644 --- a/examples/fastapi_example.py +++ b/examples/fastapi_example.py @@ -1,8 +1,69 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# FastAPI Example Module + +This module demonstrates the use of FastAPI in conjunction with the DevSetGo Toolkit to create a fully functional API. +It includes examples of database operations, user management, and system health endpoints. The module is designed to +showcase best practices for building scalable and maintainable FastAPI applications. + +## Features + +- **Database Integration**: + - Uses SQLAlchemy for ORM and database interactions. + - Supports SQLite (in-memory) for demonstration purposes. + - Includes models for `User` and `Address` tables with relationships. + +- **API Endpoints**: + - CRUD operations for `User` records. + - Bulk operations for creating and deleting records. + - System health endpoints for monitoring uptime, heap dumps, and status. + - Robots.txt endpoint for bot management. + +- **Logging**: + - Configured using `loguru` for structured and detailed logging. + - Logs API requests, database operations, and system events. + +- **Asynchronous Operations**: + - Fully asynchronous database operations using `asyncpg` and `aiosqlite`. + - Asynchronous lifespan management for startup and shutdown events. + +- **Configuration**: + - Modular configuration for database, logging, and API behavior. + - Bot management configuration for controlling access to the API. + +## Usage + +1. **Run the Application**: + Use the following command to start the FastAPI application: + ```bash + uvicorn fastapi_example:app --host 127.0.0.1 --port 5001 + ``` + +2. **Access the API**: + - OpenAPI Documentation: [http://127.0.0.1:5001/docs](http://127.0.0.1:5001/docs) + - ReDoc Documentation: [http://127.0.0.1:5001/redoc](http://127.0.0.1:5001/redoc) + +3. **Database Operations**: + - Use the provided endpoints to perform CRUD operations on the `User` and `Address` tables. + - Example endpoints include: + - `/database/create-one-record` + - `/database/get-all` + - `/database/delete-one-record` + +4. **Health Monitoring**: + - Access system health endpoints under `/api/health`. + +## Dependencies + +- `FastAPI`: Web framework for building APIs. +- `SQLAlchemy`: ORM for database interactions. +- `loguru`: Logging library for structured logs. +- `tqdm`: Progress bar for bulk operations. +- `pydantic`: Data validation and settings management. +- `DevSetGo Toolkit`: Custom library for database and common utility functions. + +## License +This module is licensed under the MIT License. """ import datetime import secrets diff --git a/examples/file_monitor.py b/examples/file_monitor.py index 05da7f9f..1b5aefdf 100644 --- a/examples/file_monitor.py +++ b/examples/file_monitor.py @@ -1,3 +1,65 @@ +""" +# File Monitor Example + +This module demonstrates the usage of the `process_files_flow` function from the `dsg_lib.common_functions.file_mover` library. +It monitors a source directory for files matching a specific pattern, processes them, and moves them to a destination directory, +optionally compressing the files during the process. + +## Features + +- **Directory Monitoring**: Watches a source directory for files matching a specified pattern (e.g., `*.csv`). +- **File Processing Flow**: Utilizes the `process_files_flow` function to handle file movement and optional compression. +- **Sample File Creation**: Periodically generates sample files in the source directory for testing purposes. +- **Asynchronous Execution**: Leverages Python's `asyncio` for concurrent tasks, such as file creation and processing. + +## Configuration + +The following constants can be configured to customize the behavior of the script: + +- `SOURCE_DIRECTORY`: Path to the directory where files are monitored. +- `TEMPORARY_DIRECTORY`: Path to a temporary directory used during file processing. +- `DESTINATION_DIRECTORY`: Path to the directory where processed files are moved. +- `FILE_PATTERN`: File pattern to monitor (e.g., `*.csv`). +- `COMPRESS_FILES`: Boolean flag to enable or disable file compression during processing. +- `CLEAR_SOURCE`: Boolean flag to clear the source directory before starting. + +## Usage + +1. Ensure the required directories exist. The script will create them if they do not. +2. Run the script to start monitoring the source directory and processing files. +3. The script will also create sample files in the source directory every 10 seconds for demonstration purposes. + +## Example + +To run the script: + +```bash +python file_monitor.py +``` + +Press `Ctrl+C` to stop the script. + +## Dependencies + +- `os` and `pathlib`: For file and directory operations. +- `asyncio`: For asynchronous task management. +- `loguru`: For logging. +- `dsg_lib.common_functions.file_mover`: For the file processing flow. + +## Notes + +- The script is designed for demonstration purposes and may require adjustments for production use. +- Ensure the `dsg_lib` library is installed and accessible in your environment. + +## Error Handling + +- The script gracefully handles `KeyboardInterrupt` to stop execution. +- The file creation task is canceled when the main function completes. + +## License +This module is licensed under the MIT License. +""" + import os import asyncio from pathlib import Path diff --git a/examples/json_example.py b/examples/json_example.py index b8be6e48..2cb9985c 100644 --- a/examples/json_example.py +++ b/examples/json_example.py @@ -1,8 +1,52 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# JSON Example Module + +This module demonstrates how to use the `open_json` and `save_json` functions from the `dsg_lib.common_functions.file_functions` package. It provides an example JSON structure and functions to save and load JSON data to and from a file. + +## Features + +- **Example JSON Data**: Contains a dictionary with information about historical figures and their contributions. +- **Save JSON Data**: Demonstrates saving JSON data to a file using the `save_json` function. +- **Open JSON Data**: Demonstrates loading JSON data from a file using the `open_json` function. + +## Example JSON Structure + +The `example_json` dictionary includes: +- A list of `super_cool_people` with details such as: + - `name`: The name of the person. + - `famous_for`: A brief description of their contributions. + - `birth_date`: Their date of birth. + - `death_date`: Their date of death. +- A `sources` field indicating the source of the information. + +## Functions + +### `save_some_data(example_json: str)` +Saves the provided JSON data to a file named `your-file-name.json`. + +### `open_some_data(the_file_name: str) -> dict` +Loads JSON data from the specified file and returns it as a dictionary. + +## Usage + +Run the module directly to: +1. Save the `example_json` data to a file. +2. Load the data back from the file. +3. Print the loaded data to the console. + +## Notes + +- Ensure the `dsg_lib` package is installed and accessible in your environment. +- Replace `"your-file-name.json"` with the desired file name when using the functions in a real-world scenario. + +## Example Execution + +```bash +python json_example.py +``` +## License +This module is licensed under the MIT License. """ from dsg_lib.common_functions.file_functions import open_json, save_json diff --git a/examples/log_example.py b/examples/log_example.py index ee18f16e..18a1bfbc 100644 --- a/examples/log_example.py +++ b/examples/log_example.py @@ -1,8 +1,64 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# Log Example Module + +This module demonstrates advanced logging configurations and usage in Python. It integrates both the `logging` module and `loguru` for robust logging capabilities. The module also showcases multi-threading and multi-processing for concurrent execution, while logging messages and handling exceptions. + +## Features + +- **Logging Configuration**: Configures logging with options for log rotation, retention, backtrace, and serialization. +- **Exception Handling**: Demonstrates exception handling with logging for `ZeroDivisionError`. +- **Concurrent Execution**: + - Multi-threading: Executes tasks concurrently using threads. + - Multi-processing: Executes tasks concurrently using processes. +- **Large Message Logging**: Logs large messages repeatedly to test logging performance. +- **Progress Tracking**: Uses `tqdm` to display progress bars for threads and processes. + +## Functions + +### `div_zero(x, y)` +Attempts to divide `x` by `y` and logs any `ZeroDivisionError` encountered. + +### `div_zero_two(x, y)` +Similar to `div_zero`, attempts to divide `x` by `y` and logs any `ZeroDivisionError` encountered. + +### `log_big_string(lqty=100, size=256)` +Logs a large string multiple times, demonstrating both standard logging and `loguru` logging. + +### `worker(wqty=1000, lqty=100, size=256)` +Executes the `log_big_string` function repeatedly, simulating a worker process or thread. + +### `main(wqty, lqty, size, workers, thread_test, process_test)` +Main entry point for the module. Configures and starts either multi-threading or multi-processing based on the provided arguments. + +## Usage + +Run the module directly to test its functionality. Example: + +```bash +python log_example.py +``` + +You can customize the parameters for workers, logging quantity, and message size by modifying the `main` function call in the `__main__` block. + +## Dependencies + +- `logging` +- `loguru` +- `multiprocessing` +- `threading` +- `secrets` +- `tqdm` +- `dsg_lib.common_functions` + +## Notes + +- Ensure the `dsg_lib` library is installed and accessible. +- Adjust the logging configuration as needed for your application. +- Use the `process_test` or `thread_test` flags to toggle between multi-processing and multi-threading. + +## License +This module is licensed under the MIT License. """ # from loguru import logger import logging diff --git a/examples/pattern_example.py b/examples/pattern_example.py index e8422443..a19c35df 100644 --- a/examples/pattern_example.py +++ b/examples/pattern_example.py @@ -1,8 +1,37 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# Pattern Example Module + +This module demonstrates the usage of the `pattern_between_two_char` function from the `dsg_lib.common_functions.patterns` package. It provides examples of how to extract patterns between specified characters in a given text block. + +## Features + +- **ASCII_LIST**: A comprehensive list of ASCII characters, which can be used for various text processing tasks. +- **pattern_find**: A utility function to find and pretty-print patterns between two specified characters in a text block. +- **run_examples**: A function that runs example use cases, including: + - Extracting patterns from a simple text block. + - Generating a large random text block and extracting patterns from it. + +## Usage + +To run the examples, execute this script directly. The output will demonstrate how patterns are extracted from text blocks. + +## Functions + +### `pattern_find(left_char: str, right_char: str, text_block: str)` +Finds and pretty-prints patterns between the specified `left_char` and `right_char` in the provided `text_block`. + +### `run_examples()` +Runs example use cases to showcase the functionality of the `pattern_between_two_char` function. + +## Example Output + +When running the script, you will see: +1. Patterns extracted from a predefined text block. +2. Patterns extracted from a randomly generated large text block. + +## License +This module is licensed under the MIT License. """ import pprint from random import randint diff --git a/examples/text_example.py b/examples/text_example.py index ac108bf8..f27a2689 100644 --- a/examples/text_example.py +++ b/examples/text_example.py @@ -1,8 +1,43 @@ # -*- coding: utf-8 -*- """ -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# Text Example Module + +This module demonstrates basic file operations using the `dsg_lib.common_functions.file_functions` library. +It provides examples of saving text data to a file and reading text data from a file. + +## Functions + +### `save_some_data(example_text: str)` +Saves the provided text data to a file. +- **Parameters**: + - `example_text` (str): The text data to be saved. +- **Behavior**: + Calls the `save_text` function from `dsg_lib.common_functions.file_functions` to save the data to a file named `your-file-name.txt`. + +### `open_some_data(the_file_name: str) -> str` +Reads text data from a specified file. +- **Parameters**: + - `the_file_name` (str): The name of the file to be read. +- **Returns**: + - `result` (str): The content of the file as a string. +- **Behavior**: + Calls the `open_text` function from `dsg_lib.common_functions.file_functions` to read the content of the file. + +## Example Usage + +```python +if __name__ == "__main__": + save_some_data(example_text) + opened_file: str = open_some_data("your-file-name.txt") + print(opened_file) +``` + +## Notes +- Ensure that the `dsg_lib` library is installed and accessible in your environment. +- The file operations assume that the file paths and permissions are correctly configured. + +## License +This module is licensed under the MIT License. """ from dsg_lib.common_functions.file_functions import open_text, save_text diff --git a/examples/validate_emails.py b/examples/validate_emails.py index 77f9e4fa..4b41c712 100644 --- a/examples/validate_emails.py +++ b/examples/validate_emails.py @@ -1,41 +1,65 @@ # -*- coding: utf-8 -*- """ -This module is used to validate a list of email addresses using various configurations. - -The module imports the `validate_email_address` function from the `dsg_lib.common_functions.email_validation` -module and uses it to validate a list of email addresses. The email addresses and configurations are hard-coded -into the module. - -The module measures the time taken to validate all the email addresses with all the configurations and prints -the results in a sorted order. - -The module can be run as a standalone script. - -Example: - $ python validate_emails.py - -Attributes: - email_addresses (list of str): A list of email addresses to validate. - configurations (list of dict): A list of configurations to use for validation. Each configuration is a - dictionary with the following keys: - - check_deliverability (bool): Whether to check if the email address is deliverable. - - test_environment (bool): Whether the function is being run in a test environment. - - allow_smtputf8 (bool): Whether to allow non-ASCII characters in the email address. - - allow_empty_local (bool): Whether to allow email addresses with an empty local part. - - allow_quoted_local (bool): Whether to allow email addresses with a quoted local part. - - allow_display_name (bool): Whether to allow email addresses with a display name. - - allow_domain_literal (bool): Whether to allow email addresses with a domain literal. - - globally_deliverable (bool): Whether the email address should be globally deliverable. - - timeout (int): The timeout for the validation in seconds. - - dns_type (str): The type of DNS to use for the validation. Can be 'dns' or 'timeout'. - -Functions: - validate_email_address(email: str, **kwargs: dict) -> dict: Validates an email address using the provided - configuration and returns a dictionary with the results. - -Author: Mike Ryan -Date: 2024/05/16 -License: MIT +# Email Validation Example Script + +This module demonstrates how to validate a list of email addresses using various configurations. It leverages the `validate_email_address` function from the `dsg_lib.common_functions.email_validation` module to perform the validation. + +The script is designed to: +- Validate a predefined list of email addresses. +- Use multiple configurations to test different validation scenarios. +- Measure and display the time taken to validate all email addresses. +- Print the validation results in a sorted order for better readability. + +## Features + +- **Email Validation**: Checks the validity of email addresses based on various configurations. +- **Custom Configurations**: Supports multiple validation options such as deliverability checks, allowing quoted local parts, and more. +- **Performance Measurement**: Tracks the time taken to validate all email addresses. +- **Result Sorting**: Outputs the validation results in a sorted format for easier analysis. + +## Usage + +Run the script as a standalone module: + +```bash +$ python validate_emails.py +``` + +## Attributes + +### Email Addresses +A predefined list of email addresses to validate. The list includes: +- Valid email addresses. +- Invalid email addresses. +- Edge cases such as emails with non-ASCII characters, quoted local parts, and domain literals. + +### Configurations +A list of dictionaries, where each dictionary represents a validation configuration. Configuration options include: +- `check_deliverability` (bool): Whether to check if the email address is deliverable. +- `test_environment` (bool): Whether the function is being run in a test environment. +- `allow_smtputf8` (bool): Whether to allow non-ASCII characters in the email address. +- `allow_empty_local` (bool): Whether to allow email addresses with an empty local part. +- `allow_quoted_local` (bool): Whether to allow email addresses with a quoted local part. +- `allow_display_name` (bool): Whether to allow email addresses with a display name. +- `allow_domain_literal` (bool): Whether to allow email addresses with a domain literal. +- `globally_deliverable` (bool): Whether the email address should be globally deliverable. +- `timeout` (int): The timeout for the validation in seconds. +- `dns_type` (str): The type of DNS to use for the validation. Can be `'dns'` or `'timeout'`. + +## Functions + +### `validate_email_address(email: str, **kwargs: dict) -> dict` +Validates an email address using the provided configuration and returns a dictionary with the results. + +## Example Output + +The script outputs the validation results in a sorted order, along with the time taken for the validation process. Each result includes: +- The email address. +- The validation status. +- Additional metadata based on the configuration used. + +## License +This module is licensed under the MIT License. """ import pprint import time diff --git a/images/devsetgo_lib_logo.svg b/images/devsetgo_lib_logo.svg new file mode 100644 index 00000000..da3f527a --- /dev/null +++ b/images/devsetgo_lib_logo.svg @@ -0,0 +1,28 @@ + + diff --git a/images/devsetgo_lib_logo_white_bg.svg b/images/devsetgo_lib_logo_white_bg.svg new file mode 100644 index 00000000..7155e057 --- /dev/null +++ b/images/devsetgo_lib_logo_white_bg.svg @@ -0,0 +1,31 @@ + + diff --git a/images/dsg_logo_1.png b/images/dsg_logo_1.png new file mode 100644 index 00000000..717e5220 Binary files /dev/null and b/images/dsg_logo_1.png differ diff --git a/images/dsg_logo_2.png b/images/dsg_logo_2.png new file mode 100644 index 00000000..ed9b59b9 Binary files /dev/null and b/images/dsg_logo_2.png differ diff --git a/images/dsg_logo_3.png b/images/dsg_logo_3.png new file mode 100644 index 00000000..521c9512 Binary files /dev/null and b/images/dsg_logo_3.png differ diff --git a/makefile b/makefile index 6d5a1aa8..8a0cd4b9 100644 --- a/makefile +++ b/makefile @@ -36,7 +36,9 @@ bump: ## Bump the version of the project cleanup: isort ruff autoflake ## Run isort, ruff, autoflake create-docs: ## Build and deploy the project's documentation + python3 scripts/changelog.py + python3 scripts/update_docs.py cp /workspaces/$(REPONAME)/README.md /workspaces/$(REPONAME)/docs/index.md cp /workspaces/$(REPONAME)/CONTRIBUTING.md /workspaces/$(REPONAME)/docs/contribute.md cp /workspaces/$(REPONAME)/CHANGELOG.md /workspaces/$(REPONAME)/docs/release-notes.md @@ -44,7 +46,9 @@ create-docs: ## Build and deploy the project's documentation mkdocs gh-deploy create-docs-local: ## Build and deploy the project's documentation + python3 scripts/changelog.py + python3 scripts/update_docs.py cp /workspaces/$(REPONAME)/README.md /workspaces/$(REPONAME)/docs/index.md cp /workspaces/$(REPONAME)/CONTRIBUTING.md /workspaces/$(REPONAME)/docs/contribute.md cp /workspaces/$(REPONAME)/CHANGELOG.md /workspaces/$(REPONAME)/docs/release-notes.md diff --git a/mkdocs.yml b/mkdocs.yml index 0ff338c9..3063f90e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -31,12 +31,18 @@ nav: - Async Database Setup: 'database/async_database_setup.md' - Database Operations: 'database/database_operations.md' - - Recipes: - - FastAPI: 'recipes/fastapi.md' - - Async Database: 'recipes/asyncDatabase.md' - - Logging: 'recipes/loggingExample.md' - - Patterns: 'recipes/patterns.md' - - EmailValidation: 'recipes/emailValidation.md' + - Examples: + - csv_example: 'examples/csv_example.md' + - log_example: 'examples/log_example.md' + - cal_example: 'examples/cal_example.md' + - pattern_example: 'examples/pattern_example.md' + - validate_emails: 'examples/validate_emails.md' + - csv_example_with_timer: 'examples/csv_example_with_timer.md' + - fastapi_example: 'examples/fastapi_example.md' + - json_example: 'examples/json_example.md' + - text_example: 'examples/text_example.md' + - file_monitor: 'examples/file_monitor.md' + - About: - About: 'about.md' - Contributing: 'contribute.md' diff --git a/report.xml b/report.xml index 92444390..35252c61 100644 --- a/report.xml +++ b/report.xml @@ -1 +1 @@ -