Releases: thiswillbeyourgithub/wdoc
Release 3.3.0
What's new
This release focuses on adding DuckDuckGo web search capabilities and introducing binary embeddings support for more efficient vector storage.
✨ New Features
DuckDuckGo Web Search Integration
- [372fe57] Add DuckDuckGo search support with URL extraction and metadata
- [273195e] Support
wdoc wdb "your query"
shorthand for web search - [03bfe08] Add DuckDuckGo search tests and documentation
Binary Embeddings Support
- [c528bad] Add support for binary embeddings with 8x memory reduction
- [8f65197] Enable FAISS vectorstore compression by default
- [37ebd97] Create CompressedFAISS subclass with zlib compression
🐛 Bug Fixes
Core Functionality
- [0d72efd] Fix wrong decorator used for
load_one_doc
- [edcf671] Fix
ddg_region
type (str not int) - [66ab177] Fix type hints for
ddg_safesearch
andloading_failure
- [957936c] Use keyword arguments instead of fire when calling wdoc
Testing Environment
- [d3de58e] Fix piped input/output handling in pytest environment
- [42ff516] Prevent pipe usage in pytest environment
- [c78dc0b] Add pytest environment detection
🧪 Testing Improvements
- [1b09996] Fix the
run_all_test
script - [8ed1d0c] Add comprehensive DuckDuckGo search functionality tests
- [b184177] Split CLI tests into separate
test_cli.py
file - [9d7fe9c] Split parsing tests into separate
test_parsing.py
file - [12b012d] Move vector store tests to dedicated test file
📚 Documentation
- [d7d6b04] Explain how to run tests in README
- [dc15001] Clarify how to disable parallel processing
- [df4b79f] Document debug mode's effect on
loading_failure
default - [1832299] Add shell examples for DuckDuckGo usage
🔧 Enhancements
CLI/UX Improvements
- [7e994a6] Rename
parse_file
function toparse_doc
- [4aa247e] Re-ask for input when empty query provided in CLI
- [57d5d5f] Fix Fire's pager issue in CLI
Performance THISISANAMPERSAND Reliability
- [68d4c75] Bump LiteLLM to latest version for improved startup time
- [ab9c5e9] Add parallel processing option for Whisper audio splits
- [6b13044] Add loop counter and crash protection for recursive file processing
🔄 Version Update
- [6435133] Bump version from 3.2.5 → 3.3.0
Commits details since the last release
- [6435133] by @thiswillbeyourgithub, 36 minutes ago:
bump version 3.2.5 -> 3.3.0
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [1b09996] by @thiswillbeyourgithub, 24 hours ago:
test: fix the run_all_test script
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [d7d6b04] by @thiswillbeyourgithub, 24 hours ago:
doc: explain how to run the tests
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [62cc2ce] by @thiswillbeyourgithub, 24 hours ago:
fix: ddg test
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [0d72efd] by @thiswillbeyourgithub, 24 hours ago:
fix: wrong decorator used for load_one_doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [dc15001] by @thiswillbeyourgithub, 24 hours ago:
doc: clarify how to disable parallel processing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [e0453cb] by @thiswillbeyourgithub, 24 hours ago:
minor: mention a type hint
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [edcf671] by @thiswillbeyourgithub, 24 hours ago:
fix: ddg_region is actually a str not an int
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [df4b79f] by @thiswillbeyourgithub, 24 hours ago:
doc: mention that debug changes the default value for loading_failure
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [66ab177] by @thiswillbeyourgithub, 25 hours ago:
fix: type of ddg_safesearch and loading_failure should be Literal
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [98b0867] by @thiswillbeyourgithub, 25 hours ago:
doc: explain that loading_failure defaultto crash when parsing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [90eacb3] by @thiswillbeyourgithub, 25 hours ago:
test: ddg should use us region by default
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [c8b1944] by @thiswillbeyourgithub, 25 hours ago:
test: less severe check for pipes
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [6e12e5c] by @thiswillbeyourgithub, 25 hours ago:
test: remove one -n auto arg
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [d3de58e] by @thiswillbeyourgithub, 2 days ago:
fix: actually inside pytest we should not bypass piped input but only piped output
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/misc.py
- [5715bc4] by @thiswillbeyourgithub, 2 days ago:
test: add env variable to detect if being called by pytest
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
tests/conftest.py
- [42ff516] by @thiswillbeyourgithub, 2 days ago:
new: do not allow using pipe input or output in pytest environment
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [c78dc0b] by @thiswillbeyourgithub, 2 days ago:
new: detect when wdoc is called in pytest environment
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
wdoc/utils/env.py
wdoc/utils/misc.py
- [fca39c0] by @thiswillbeyourgithub, 2 days ago:
test: missing oneoff and failsafe when testing ddg
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [b2b4cf1] by @thiswillbeyourgithub, 2 days ago:
test: fix missing quotation sign for args
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [13409b1] by @thiswillbeyourgithub, 2 days ago:
test: fix a timeout not long enough
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [957936c] by @thiswillbeyourgithub, 2 days ago:
fix: use keyword aguments instead of fire when calling wdoc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b034337] by @thiswillbeyourgithub, 2 days ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b44d730] by @thiswillbeyourgithub, 2 days ago:
fix: replacing ddg_max_result
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [dfcaf3b] by @thiswillbeyourgithub, 2 days ago:
fix: wrong way to replace ddg_max_result to ddg_max_results
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [adc991a] by @thiswillbeyourgithub, 2 days ago:
actually no
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [9ab4cbf] by @thiswillbeyourgithub, 2 days ago:
fix: type hint of load_one_doc can be a list of string in case of error
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [5f7fcf4] by @thiswillbeyourgithub, 2 days ago:
typo: Nvidia instead of NVidia
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
tests/test_cli.py
wdoc/docs/examples.md
- [03bfe08] by @thiswillbeyourgithub, 2 days ago:
test: add test for ddg search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [48165fa] by @thiswillbeyourgithub, 2 days ago:
test: clearer echo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [08cac94] by @thiswillbeyourgithub, 2 days ago:
remove unused import
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [3aabd2d] by @thiswillbeyourgithub, 2 days ago:
style: format test_cli.py with linter
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [8ed1d0c] by @thiswillbeyourgithub, 2 days ago:
feat: add test for DuckDuckGo search functionality with NVIDIA query
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [a8e3e04] by @thiswillbeyourgithub, 2 days ago:
test: add test for DuckDuckGo search with NVIDIA query
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [1832299] by @thiswillbeyourgithub, 2 days ago:
doc: add shell example for using duckduckgo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/examples.md
- [e6c4641] by @thiswillbeyourgithub, 2 days ago:
typo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/examples.md
- [917ee51] by @...
Release 3.2.5
What's new
This release brings several improvements to command-line argument handling and filetype detection, along with key bug fixes and build process enhancements.
✨ Features
- CLI & Filetype Detection:
- Build Process:
- Integrated
sphinx-apidoc
into the ReadTheDocs build process via a pre-build job in.readthedocs.yaml
([cc86c7b]).
- Integrated
🐛 Fixes
- Corrected an issue with
sys.argv
handling that led to duplicated arguments ([e7cf185]). - Updated
litellm
dependency to resolve crashes experienced on Windows environments ([cfff0ac]), see #20.
🛠️ Improvements & Refactoring
- Filetype Detection Internals:
- Code Quality:
- Improved documentation by adding docstrings to custom exception classes ([8e6ca1a]).
Chores
- Version bumped to 3.2.5 ([82b7f81]).
Commits details since the last release
- [82b7f81] by @thiswillbeyourgithub, 19 minutes ago:
bump version 3.2.4 -> 3.2.5
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [e7cf185] by @thiswillbeyourgithub, 3 minutes ago:
fix: badly handled sys.argv was duplicating args
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [cfff0ac] by @thiswillbeyourgithub, 19 minutes ago:
fix: bump version of litellm because windows crashes
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [cc86c7b] by @thiswillbeyourgithub (aider), 2 days ago:
feat: add pre-build job to run sphinx-apidoc in .readthedocs.yaml
.readthedocs.yaml
- [ab76610] by @thiswillbeyourgithub, 2 days ago:
new: use the filetype detector to infer what to do in case of multiple implicit arguments from the cli
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [05966c6] by @thiswillbeyourgithub, 2 days ago:
enh: add debug prints to the filetype detector
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [520f4ce] by @thiswillbeyourgithub, 2 days ago:
new: use a specific exception when we can't infer the filetype
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [8e6ca1a] by @thiswillbeyourgithub, 2 days ago:
add docstring to some exceptions
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/errors.py
- [39af223] by @thiswillbeyourgithub, 2 days ago:
add an error for undetectable filetype
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/errors.py
- [b453748] by @thiswillbeyourgithub, 2 days ago:
new: put the filetype detection code in a separate function
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
Release 3.2.4
What's new
This release primarily focuses on significant documentation enhancements, crucial bug fixes for stability and build processes, and introduces updated dependencies and tokenization.
✨ New Features
- Upgraded default token estimation to use
gpt-4o-mini
tokenizer, replacinggpt-3.5-turbo
([6d41817]). - Integrated the latest
yt-dlp
for YouTube downloads ([ab207b4]). - Environment variable documentation is now automatically added to the
EnvDataclass
class__doc__
([ed9dd38]).
🐛 Bug Fixes
- Resolved a crash on ReadTheDocs caused by missing
yt-dlp
dependency ([f5068a3]). - Fixed an issue where accessing
env.__class__
on ReadTheDocs could cause a crash ([4e180f0]). - Corrected relative import paths in
wdoc
that were preventing Sphinx API documentation builds ([ade5930]). - Fixed issues with the Sphinx API command in the FAQ section of the README ([38008aa], [ff093a2]).
- Ensured collapsible bars in documentation function correctly ([3cef833]).
📚 Documentation & Refinements
- Extensive updates and fixes to Sphinx documentation generation and content:
- Addressed outdated Sphinx documentation files ([90bde99]).
- Improved API autodoc parameters for clearer documentation ([243de66]).
- Excluded private and special members from documentation ([7abedd4]).
- Added Sphinx command to FAQ in README ([1e6602e]) and removed private members from it ([11ae11b]).
- Updated copyright year to 2025 ([bd7e3c5]).
- Streamlined documentation structure and configuration:
- Removed unused make files (
Makefile
,make.bat
) for documentation ([07b0a7d]). - Removed unused argument for theme flyout display ([17bc5e6]).
- Removed unused templates path ([6bffa20]) and CSS ([712df08]).
- Removed duplicate README from the documentation source ([2b93162]).
- Added a documentation table to the main index ([1dfe2b3]).
- Removed unused make files (
⚙️ Build & Chores
- Bumped version to 3.2.4 ([ed7a9c7]).
Commits details since the last release
- [ed7a9c7] by @thiswillbeyourgithub, 20 seconds ago:
bump version 3.2.3 -> 3.2.4
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [f5068a3] by @thiswillbeyourgithub, 13 minutes ago:
fix: missing yt-dlp makes readthedock crash
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [17bc5e6] by @thiswillbeyourgithub, 19 minutes ago:
remove unused argument for theme flyout display
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [4e180f0] by @thiswillbeyourgithub, 22 minutes ago:
fix: class attribute of env is accessed by readthedocks and should not crash
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [243de66] by @thiswillbeyourgithub, 2 hours ago:
saner api autodoc parameters
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [ed9dd38] by @thiswillbeyourgithub, 3 hours ago:
new: add the environment variable documentation to the doc of the EnvDataclass class
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/env.py
- [07b0a7d] by @thiswillbeyourgithub, 3 hours ago:
doc: remove unused make files for doc
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
docs/Makefile
docs/make.bat
- [7abedd4] by @thiswillbeyourgithub, 4 hours ago:
doc: dont include private nor special
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [38008aa] by @thiswillbeyourgithub, 2 hours ago:
fix: sphinx api command of faq
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [11ae11b] by @thiswillbeyourgithub, 4 hours ago:
remove private from sphinx command
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [90bde99] by @thiswillbeyourgithub, 4 hours ago:
fix outdated sphinx doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/wdoc.rst
docs/source/wdoc.utils.batch_file_loader.rst
docs/source/wdoc.utils.customs.compressed_embeddings_cache.rst
docs/source/wdoc.utils.customs.fix_llm_caching.rst
docs/source/wdoc.utils.customs.rst
docs/source/wdoc.utils.embeddings.rst
docs/source/wdoc.utils.env.rst
docs/source/wdoc.utils.errors.rst
docs/source/wdoc.utils.flags.rst
docs/source/wdoc.utils.import_tricks.rst
docs/source/wdoc.utils.interact.rst
docs/source/wdoc.utils.llm.rst
docs/source/wdoc.utils.loaders.rst
docs/source/wdoc.utils.logger.rst
docs/source/wdoc.utils.misc.rst
docs/source/wdoc.utils.prompts.rst
docs/source/wdoc.utils.retrievers.rst
docs/source/wdoc.utils.rst
docs/source/wdoc.utils.tasks.query.rst
docs/source/wdoc.utils.tasks.rst
docs/source/wdoc.utils.tasks.summarize.rst
docs/source/wdoc.utils.typechecker.rst
docs/source/wdoc.wdoc.rst
- [ff093a2] by @thiswillbeyourgithub, 4 hours ago:
fix: sphinx api command of faq
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [ade5930] by @thiswillbeyourgithub, 4 hours ago:
fix: relative wdoc imports were stopping sphinx api build
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/init.py
wdoc/main.py
wdoc/utils/init.py
wdoc/utils/batch_file_loader.py
wdoc/utils/customs/init.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
wdoc/utils/import_tricks.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/init.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py
wdoc/utils/typechecker.py
wdoc/wdoc.py
- [1e6602e] by @thiswillbeyourgithub, 5 hours ago:
doc: add to faq the sphinx command
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [bd7e3c5] by @thiswillbeyourgithub, 5 hours ago:
update copyright year to 2025
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [6bffa20] by @thiswillbeyourgithub, 6 hours ago:
remove unused templates path in doc
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [2b93162] by @thiswillbeyourgithub, 6 hours ago:
remove duplicate readme from the doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/index.rst
- [3cef833] by @thiswillbeyourgithub, 6 hours ago:
fix collapsible bar
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/conf.py
- [712df08] by @thiswillbeyourgithub, 6 hours ago:
remove unused css from the doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/_static/custom.css
docs/source/conf.py
- [1dfe2b3] by @thiswillbeyourgithub, 6 hours ago:
documentation table
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
docs/source/index.rst
- [6d41817] by @thiswillbeyourgithub, 25 hours ago:
new: use gpt-4o-mini tokenizer by default to estimate tokens
previously we used the ageing gpt-3.5-turbo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/misc.py
- [ab207b4] by @thiswillbeyourgithub, 25 hours ago:
new: use the latest yt-dl install from yt-dlp
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
Release 3.2.3
What's new
This release primarily focuses on enhancing context management for embedding models, improving debugging utilities, and updating documentation for better clarity. It also includes several important bug fixes and feature additions.
✨ Features
- Introduced a new environment variable
WDOC_MAX_EMBED_CONTEXT
to allow capping the context size for embedding models ([d9e200f8]
)- Documentation for this new variable has been added (
[a2408fd0]
)
- Documentation for this new variable has been added (
- Enhanced debugging by ensuring debug prints are always active when
md_printer
is used. This helps in retrieving LLM answers from logs if they weren't saved to a file ([69db1916]
) - Added the current date to summary metadata and headers to help reduce potential LLM hallucinations (
[64ca4665]
)
🐛 Fixes
- Text Splitting & Context Handling:
- Addressed an issue where large language models have more context than embedding models by setting a
max_tokens
limit for the text splitter ([dac6802d]
) - Fixed an edge case where the
wdoc max chunk
setting could be ignored ([196b3a00]
) - Corrected an old variable name within the text splitting logic (
[767bc754]
)
- Addressed an issue where large language models have more context than embedding models by setting a
- Updated the default model to
gemini 2.5 preview
to reflect its renaming on OpenRouter ([22978609]
) - Improved the mechanism for ignoring initial "breathing" or placeholder lines in summaries (
[4dbcf158]
)
📚 Documentation
- Clarity and Enhancements:
- Clarified the usage of
save
andload
functionalities ([9d9642d4]
) and specifically advised against using them simultaneously ([5270c350]
) - Made multiple clarifications to the README for better understanding (
[9284ff54]
,[cb4cb519]
,[f677e5a2]
,[39e0da55]
) - Updated Ollama examples to recommend
snowflake-arctic-embed2
instead ofbge-m3
([d045702b]
) - Added documentation for the
WDOC_MAX_EMBED_CONTEXT
environment variable ([a2408fd0]
)
- Clarified the usage of
- Removed a documentation file (
summary_rag.md
) that was not yet ready for release ([6d20c220]
)
⚙️ Chore & Maintenance
- Version bumped to
3.2.3
(following an earlier bump to3.2.2
[[71ac503c]
]) ([f62a2322]
) - README Updates:
- Updated TODO items (
[8f2cbfd7]
,[5d090421]
) - Added a PyPI badge for better project visibility (
[60ef4112]
)
- Updated TODO items (
Commits details since the last release
- [f62a232] by @thiswillbeyourgithub, 46 seconds ago:
bump version 3.2.2 -> 3.2.3
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [6d20c22] by @thiswillbeyourgithub, 76 seconds ago:
doc: removed file not yet ready
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
summary_rag.md
- [71ac503] by @thiswillbeyourgithub, 4 minutes ago:
bump version 3.2.1 -> 3.2.2
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [8f2cbfd] by @thiswillbeyourgithub, 3 minutes ago:
todo
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [69db191] by @thiswillbeyourgithub, 40 minutes ago:
new: now debug print is used anyway when md_printer is used
this is to make you able to go to the logs to fetch and answer form the
LLM if you have forgotten to store it to a file
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
wdoc/wdoc.py
- [a2408fd] by @thiswillbeyourgithub (aider), 66 minutes ago:
docs: Add documentation for WDOC_MAX_EMBED_CONTEXT variable
wdoc/docs/help.md
- [d9e200f] by @thiswillbeyourgithub, 66 minutes ago:
feat: add new env var to cap the context size for embedding models
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/misc.py
- [196b3a0] by @thiswillbeyourgithub, 72 minutes ago:
fix: edge case where wdoc max chunk would be ignored
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [dac6802] by @thiswillbeyourgithub, 76 minutes ago:
fix: set a limit to max_tokens for the text splitter as large LLM have more context than embeddings models nowadays
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [767bc75] by @thiswillbeyourgithub, 80 minutes ago:
fix: forgot to rename an old variable name
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [2297860] by @thiswillbeyourgithub, 86 minutes ago:
fix: set default model to gemini 2.5 preview without date timestamp
openrouter renamed that model apparently
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
wdoc/utils/env.py
- [9d9642d] by @thiswillbeyourgithub, 22 hours ago:
doc: clarify save and load
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [5270c35] by @thiswillbeyourgithub, 22 hours ago:
doc: clarify that load and save shouldnt be used at the same time
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [d045702] by @thiswillbeyourgithub, 23 hours ago:
doc: use snowflake-arctic-embed2 instead of bge-m3 for ollama examples
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/examples.md
- [60ef411] by @thiswillbeyourgithub, 26 hours ago:
add a pypi badge
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [5d09042] by @thiswillbeyourgithub, 7 days ago:
update todo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [9284ff5] by @thiswillbeyourgithub, 7 days ago:
doc: clarify
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [cb4cb51] by @thiswillbeyourgithub, 7 days ago:
doc: clarify
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [f677e5a] by @thiswillbeyourgithub, 7 days ago:
doc: clarify
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [39e0da5] by @thiswillbeyourgithub, 7 days ago:
doc: clarify
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [64ca466] by @thiswillbeyourgithub (aider), 10 days ago:
feat: Add current date to summary metadata and header to reduce hallucinations
wdoc/wdoc.py
- [4dbcf15] by @thiswillbeyourgithub, 10 days ago:
enh: better ignoring of first line of summary if just breathing
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/tasks/summarize.py
Release 3.2.1
What's new
This small patch release primarily focuses on integrating OpenRouter for model pricing/metadata and refining cost calculations.
✨ Features
- Set default models to use OpenRouter ([915699c]).
- Fetch model prices and metadata automatically from OpenRouter, improving reliability ([7f840b7]).
🐛 Fixes & Enhancements
- Much improved price calculation and handling:
- Updated
litellm
dependency ([179b589]).
🧪 Tests
- API integration tests now fail faster if an underlying API call fails ([9a0c856]).
Commits details since the last release
- [03aeab2] by @thiswillbeyourgithub, 2 minutes ago:
bump version 3.2.0 -> 3.2.1
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [915699c] by @thiswillbeyourgithub, 6 minutes ago:
new: set the default models to use openrouter
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
wdoc/utils/env.py
- [c0b90d8] by @thiswillbeyourgithub, 64 minutes ago:
fix: reworked how pricing are computed to take internal thinking into account
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/llm.py
wdoc/utils/misc.py
wdoc/utils/tasks/summarize.py
wdoc/wdoc.py
- [a17b41c] by @thiswillbeyourgithub, 80 minutes ago:
enh: better way to get the model prices
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
wdoc/wdoc.py
- [9a0c856] by @thiswillbeyourgithub, 22 minutes ago:
test: crash early if one api crash fails
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [7f840b7] by @thiswillbeyourgithub, 2 hours ago:
feat: automatically fetch the price and metadata from openrouter instead of waiting for litellm
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
wdoc/wdoc.py
- [2b29a9d] by @thiswillbeyourgithub, 2 hours ago:
fix: error message on missing model price
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [179b589] by @thiswillbeyourgithub, 2 hours ago:
bump litellm version
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
Release 3.2.0
What's new
This release focuses on improving the command-line interface (especially handling piped input/output), enhancing language model interactions (switching defaults, better caching, Ollama support), and increasing overall stability through numerous bug fixes and testing improvements.
✨ Features
- Added arguments to set specific keyword arguments (
kwargs
) for language models (--model_kwargs
,--query_eval_model_kwargs
) ([1392553]). - Introduced
WDOC_LLM_REQUEST_TIMEOUT
environment variable for LLM request timeouts (default 600s), useful for Ollama ([ec3c0c5]). - Switched default models from Claude Sonnet/Haiku to Gemini 1.5 Pro/Flash ([82ef10d]).
- Unified LLM handling to primarily use
ChatLiteLLM
, removing directChatOpenAI
usage ([30a0f0c]). - Enabled cost tracking for queries, storing the cost in the output ([e7753af]).
- Added automatic download of
nltk punkt
tokenizer during post-installation ([44f5bf8]). - Overhauled Command Line Interface (CLI) argument parsing for
wdoc
andwdoc parse
usingfire
([7c51ed2], [2f4748d]). - Removed the
--pipe
argument, relying on automatic stdin/stdout detection ([b03e79a], [2e6c1dd], [838f164]). - Removed the separate
wdoc_parse_file
entry point; usewdoc parse
instead ([2e878d2]). - Added a new script
media_url_finder.py
([beaf8fa]).
🐛 Fixes
- LLM PLACEHOLDER Caching:
- Resolved issues with LLM caching, including invalidation when
kwargs
change and LangChain's SQLite cache ([cb785da], [3e3e753]). - Fixed edge cases in thinking block parsing for models like Gemini and updated tags (
<thinking>
-><think>
) ([e111bdb], [d0ae21a], [ca9245b], [99ed332]). - Corrected underflow errors in cost calculation due to low LLM prices ([3f18f5d], [95a1984]).
- Addressed issues specific to Ollama: API key requirement relaxation, price assumption (zero),
litellm
naming (ollama_chat
->ollama
), and context window estimation ([d2f92a3], [5784b25], [43c6340], [c3c15e1]). - Fixed handling of
testing/testing
models and associated parameters ([b995197], [91b5e67], [7cf840c], [9a7b95b]). - Fixed
query_retrievers
parsing ([02d7412]). - Pinned
litellm
version for stability ([1b17c78]).
- Resolved issues with LLM caching, including invalidation when
- CLI PLACEHOLDER Piping:
- Improved detection and handling of piped input/output ([2e6c1dd], [509626a], [db2fa0f]).
- Fixed crashes and hangs when using pipes, especially with long inputs or specific test commands ([f59f34b], [414de8d], [b95b125], [826e7aa], [b6f7fd7], [177be6b]).
- Corrected argument parsing issues affecting the
--help
command ([c909337]). - Ensured logs are not colorized and Markdown rendering is disabled when outputting to a pipe ([f1d63cd], [fe2665c]).
- Fixed issues where debug prints or warnings were incorrectly suppressed or handled ([64fcd60], [a7724ff]).
- General:
- Fixed various bugs in task execution, parameter handling, and attribute declarations ([27a8d35], [91d8df3], [a0eaf51], [a6effc0], [5dce2f3], [4623fcc], [b17f567], [8cc9190], [e91ed3b], [c3649ab]).
- Corrected import path in
__main__
([0ef5e4d]). - Suppressed excessive INFO logs from
faiss
([a17a8d1]). - Handled
BrokenPipeError
gracefully ([b40832b]).
🧪 Testing
- Improved test setup for caching, using separate directories and disabling cache where necessary ([9104f86], [89f4859], [085a87e], [6935fe7]).
- Added tests for OpenRouter/default models, piping functionality, summary/query tasks with testing models, and environment variable handling ([06e35b0], [bbb8371], [caae34c], [cb9d237], [eaafafd], [1f835eb]).
- Refactored pipe tests to use
subprocess
explicitly and fixed related issues (stderr redirection, pytest capture, shell usage) ([38a3571], [7f3249a], [573acf9]). (Note: Some pipe tests were later commented out ([45cf419])).
⚡ Enhancements
- Reworked logic for detecting and modifying model parameters based on the task ([564c4f9]).
- Improved
load_media
function to handle online media more robustly by finding and clicking appropriate buttons ([049c9cb], [67772f8], [c5828d3]). - Added checks to prevent exceeding total token limits during summarization ([9bdcabc]).
- Refined logging levels and Markdown printing logic ([edfec82], [4ca394c], [895a60f]).
📚 Documentation
- Updated examples for Ollama arguments, model usage (Gemma -> Qwen2), and general clarity ([0087117], [49437ec], [4083dda], [404bbe4]).
- Clarified behavior related to LLM caching and model
kwargs
in help documentation ([c3e0219], [3e3e753], [1392553], [7db844f]). - Updated README and help files reflecting changes in default models, CLI arguments, and entry points ([82ef10d], [b03e79a], [2e878d2], [a30bccf]).
⚙️ Build PLACEHOLDER Chore
- Bumped version to 3.2.0 ([7d69d79]).
- Added
nltk
to dependencies ([44f5bf8]). - Updated
.gitignore
([84aa559], [5374ee1], [39e4106], [a25e3d4]). - Renamed
embed_kwargs
toembed_model_kwargs
([431efcb]).
Commits details since the last release
- [7d69d79] by @thiswillbeyourgithub, 77 seconds ago:
bump version 3.1.0 -> 3.2.0
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [30a0f0c] by @thiswillbeyourgithub, 24 minutes ago:
new: stop using both ChatOpenAI and ChatLiteLLM
ChatLiteLLM seems to now work reliably
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
wdoc/docs/help.md
wdoc/utils/llm.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/query.py
- [9104f86] by @thiswillbeyourgithub, 41 minutes ago:
fix: in the pytest we should delete the cache dir regularly
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/conftest.py
- [e111bdb] by @thiswillbeyourgithub, 46 minutes ago:
fix: fix edge case for gemini models that only end their thinking block
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [84aa559] by @thiswillbeyourgithub, 78 minutes ago:
test: ignore cache dir
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
.gitignore
- [89f4859] by @thiswillbeyourgithub, 79 minutes ago:
test: use a separate user dir for the cache when running the tests
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
wdoc/utils/misc.py
- [3f18f5d] by @thiswillbeyourgithub, 79 minutes ago:
fix: underflow error in cost
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/tasks/summarize.py
wdoc/wdoc.py
- [27a8d35] by @thiswillbeyourgithub, 2 hours ago:
fix: latest cost attribute was not declared
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [c3c15e1] by @thiswillbeyourgithub, 2 hours ago:
enh: if ollama is used, lower the estimate of the context window
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/llm.py
- [91d8df3] by @thiswillbeyourgithub, 2 hours ago:
fix: wrong indentation in an if
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [a0eaf51] by @thiswillbeyourgithub, 2 hours ago:
fix: wrong deepcopy for eval llm
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [29c9c4e] by @thiswillbeyourgithub, 2 hours ago:
fix: test
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [cb785da] by @thiswillbeyourgithub, 2 hours ago:
fix: make the sqlite cache already patched for langchain s stupid cache
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/fix_llm_caching.py
- [3e3e753] by @thiswillbeyourgithub, 2 hours ago:
fix: try to make it so that changing the kwargs does not reuse the cache
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/llm.py
- [c3e0219] by @thiswillbeyourgithub, 2 hours ago:
doc: explain that changing the kwargs will not invalidate the cache
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [827d563] by @thiswillbeyourgithub, 2 hours ago:
test: improved test to also test caching
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [e7753af] by @thiswillbeyourgithub, 2 hours ago:
new: store the cost of the query in the output now
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [d0ae21a] by @thiswillbeyourgithub, 2 hours ago:
fix: reworked and improved how thinking_answer_parser works
some weak models could fail despite usable results
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [ca9245b] by @thiswillbeyourgithub, 3 hours ago:
fix: dont make the thinking block parser greedy
I'm sure some models can nest thoughts
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [99ed332] by @thiswillbeyourgithub, 3 hours ago:
fix: most models nowadays use not
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/tasks/query.py
- [085a87e] by @thiswillbeyourgithub, 3 hours ago:
test: disable the embedding cache
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
...
Release 3.1.0
What's new
This release primarily focuses on enhancing logging capabilities and fixing issues related to piping behavior.
Version bump to 3.1.0
([e93dcad6]
).
✨ New Features
- Logging:
- Always display the default log location (
[2fe2c431]
). - Set log level to debug for log files and critical when used in a pipe (
[130058a1]
).
- Always display the default log location (
🚀 Enhancements
- Logging:
- Improved log format (
[61465aff]
,[dc06ccfd]
). - Increased probability of early logger initialization (
[01f01ac7]
). - Clearer error messages from python-magic (
[c846dafa]
).
- Improved log format (
🐛 Fixes
- Piping:
- Resolved confusion between input and output during piping (
[e175b7d5]
). - Corrected initialization of
is_piped
variable ([e4532d30]
).
- Resolved confusion between input and output during piping (
- Logging & Environment:
- Fixed default handler issue in logger (
[43c859dd]
). - Prevented potential crash related to environment variable handling (
[d3b1e2bc]
).
- Fixed default handler issue in logger (
🧹 Minor Changes
- Removed unused imports (
[f3c05962]
). - Adjusted test imports structure (
[69738119]
). - Removed commented code (
[86b51102]
). - Removed unused
disable_md_printing
argument ([b3af430e]
).
✅ Testing
- Added test for exception handling (
[dfbfad54]
). - Added environment variable tests (
[0fba8a13]
).
Commits details since the last release
- [e93dcad] by @thiswillbeyourgithub, 10 minutes ago:
bump version 3.0.2 -> 3.1.0
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [e175b7d] by @thiswillbeyourgithub, 31 minutes ago:
fix: piping behavior was confusing input and output
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
wdoc/utils/env.py
wdoc/utils/loaders.py
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/wdoc.py
- [b3af430] by @thiswillbeyourgithub, 34 minutes ago:
forgot to remove the arg disable_md_printing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/wdoc.py
- [61465af] by @thiswillbeyourgithub, 36 minutes ago:
enh: better log format
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [2fe2c43] by @thiswillbeyourgithub, 37 minutes ago:
new: print the default log location always
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [43c859d] by @thiswillbeyourgithub, 37 minutes ago:
fix: default handler
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [e4532d3] by @thiswillbeyourgithub, 47 minutes ago:
fix: is_piped variable was wrong
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/misc.py
- [01f01ac] by @thiswillbeyourgithub, 66 minutes ago:
enh: increase chances of logger beint initialized asap
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/init.py
wdoc/main.py
- [dc06ccf] by @thiswillbeyourgithub, 89 minutes ago:
better log format
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [f3c0596] by @thiswillbeyourgithub, 2 hours ago:
remove unused imports
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [e637c2f] by @thiswillbeyourgithub, 2 hours ago:
new: the log level now is always at debug level for the logfile and using --debug only modifyed the stdout of user
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [86b5110] by @thiswillbeyourgithub, 2 hours ago:
minor: remove commented line
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [130058a] by @thiswillbeyourgithub, 2 hours ago:
new: if wdoc is used in a pipe, we set the log level to critical
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/logger.py
- [dfbfad5] by @thiswillbeyourgithub, 2 hours ago:
test: add test for exception handling
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [6973811] by @thiswillbeyourgithub, 2 hours ago:
minor: move the test imports higher up
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [0fba8a1] by @thiswillbeyourgithub, 2 hours ago:
test: add an unexpected env variable to test that it works
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [d3b1e2b] by @thiswillbeyourgithub, 2 hours ago:
fix: env variable handling could cause a crash
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [c846daf] by @thiswillbeyourgithub, 3 hours ago:
better error message from python-magic
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
Release 3.0.2
What's new
Fixes
- Error Message Stability
- Fixed a crash caused by error messages in batch file loader
- Files affected:
wdoc/utils/batch_file_loader.py
wdoc/utils/loaders.py
- Files affected:
- Commit hash: [4af7dc6]
- Author: @thiswillbeyourgithub
- Fixed a crash caused by error messages in batch file loader
Version Bump
- Version Update
- Updated version from
3.0.1
to3.0.2
for better stability and minor enhancements- Files affected for version bump:
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- Files affected for version bump:
- Commit hash: [504b5c9]
- Author: @thiswillbeyourgithub
- Updated version from
Note
These updates aim to enhance overall functionality and prevent errors from causing interrupts, ensuring a smoother user experience. The version bump signifies an incremental improvement with significant internal fixes.
Commits details since the last release
- [504b5c9] by @thiswillbeyourgithub, 6 seconds ago:
bump version 3.0.1 -> 3.0.2
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [4af7dc6] by @thiswillbeyourgithub, 10 seconds ago:
fix: error message was causing a crash
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
wdoc/utils/loaders.py
Release 3.0.1
What's new
Version 3.0.1 - April 18, 2025
Chores Housekeeping
- Version Bump:
- Bumped version from 3.0.0 to 3.0.1.
- Commit: [3341823] by @thiswillbeyourgithub
- Bumped version from 3.0.0 to 3.0.1.
Bug Fixes
- Error Message Fix:
- Resolved issue where error message was causing a crash.
- Location:
wdoc/utils/loaders.py
- Commit: [20b5ccd] by @thiswillbeyourgithub
- Location:
- Resolved issue where error message was causing a crash.
Documentation
- Companion Tool Mention:
- Updated README to mention that a companion tool might be needed.
- Location:
README.md
- Commit: [75bc42c] by @thiswillbeyourgithub
- Location:
- Updated README to mention that a companion tool might be needed.
Testing
- Test Script Modification:
- Changed script to use
rm
instead oftrash
.- Location:
tests/run_all_tests.sh
- Commit: [75a21ee] by @thiswillbeyourgithub
- Location:
- Changed script to use
Commits details since the last release
- [3341823] by @thiswillbeyourgithub, 6 seconds ago:
bump version 3.0.0 -> 3.0.1
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [20b5ccd] by @thiswillbeyourgithub, 44 seconds ago:
fix: error message was causing a crash
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [75bc42c] by @thiswillbeyourgithub, 16 hours ago:
doc: mention the companion tool might be needed
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [75a21ee] by @thiswillbeyourgithub, 18 hours ago:
test: use rm instead of trash
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
Release 3.0.0
What's new
- Version Update 3.0.0
- Environment Variable Handling
- Logging Improvements
Enhancements
- Documentation and Scripts
- Code Optimization
Commits details since the last release
- [51bbc55] by @thiswillbeyourgithub, 29 minutes ago:
bump version 2.9.0 -> 3.0.0
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [94963ef] by @thiswillbeyourgithub, 21 minutes ago:
better test script
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [247ef4d] by @thiswillbeyourgithub, 30 minutes ago:
doc: update todo list
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [8f516e4] by @thiswillbeyourgithub, 45 minutes ago:
fix: wrongly setting env vars to True instead of "true"
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/wdoc.py
- [3491b6a] by @thiswillbeyourgithub, 47 minutes ago:
fix: main was still using flags instead of env
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [c06ec64] by @thiswillbeyourgithub, 62 minutes ago:
new: compulsively check for unexpected values in env var
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [f216546] by @thiswillbeyourgithub, 67 minutes ago:
replace a print by a logger.warning
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [f133503] by @thiswillbeyourgithub, 70 minutes ago:
remove weird handling of md_printing_disabled
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [138ef37] by @thiswillbeyourgithub, 71 minutes ago:
use loguru in main instead of print
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [d4fd1d7] by @thiswillbeyourgithub, 73 minutes ago:
new: stop using flags.py to store something that should be stored in env.py
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/init.py
wdoc/utils/batch_file_loader.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
wdoc/utils/flags.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/misc.py
wdoc/utils/tasks/query.py
wdoc/wdoc.py
- [a06b09f] by @thiswillbeyourgithub, 2 hours ago:
minor: explanatory comment
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [b9904af] by @thiswillbeyourgithub, 2 hours ago:
typo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [177e81a] by @thiswillbeyourgithub, 2 hours ago:
fix: unbounlocalerror incomprehenssible unless I reimport logger
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [69a6013] by @thiswillbeyourgithub, 2 hours ago:
minor: move cache dir declaration misc.py
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/wdoc.py
- [2e9b7f8] by @thiswillbeyourgithub, 2 hours ago:
swtich default log level depending on if is_debug is set
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/logger.py
- [31ade20] by @thiswillbeyourgithub, 2 hours ago:
feat: switch logging backend to loguru
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
wdoc/utils/batch_file_loader.py
wdoc/utils/embeddings.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py
wdoc/wdoc.py
- [d362034] by @thiswillbeyourgithub, 3 hours ago:
minor: pass the youtube playlist title metadata to docs
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [bd100cc] by @thiswillbeyourgithub, 3 hours ago:
test: add an api test that set debug to True
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [6ac351f] by @thiswillbeyourgithub, 3 hours ago:
fix: remove loaders_temp_dir_file contraption that was causing issue in some parallel edge cases
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
wdoc/utils/loaders.py
wdoc/utils/misc.py
- [c7e440d] by @thiswillbeyourgithub, 4 hours ago:
fix: error in edge case in summary
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/tasks/summarize.py
- [7087501] by @thiswillbeyourgithub, 4 hours ago:
test: better test by checking if the default install works too
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [2dda653] by @thiswillbeyourgithub, 4 hours ago:
update tests to use the new env handling
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [04d2b92] by @thiswillbeyourgithub, 4 hours ago:
remove experimental var test
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [e9fa7c1] by @thiswillbeyourgithub (aider), 4 hours ago:
docs: add docstrings to env.py
wdoc/utils/env.py
- [11cc311] by @thiswillbeyourgithub, 4 hours ago:
doc: add explanation for runtime modification of env variables
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [fdc237f] by @thiswillbeyourgithub, 4 hours ago:
update scripts for the env variable handling
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
scripts/AnkiFiltered/AnkiFilteredDeckCreator.py
- [4ce5d09] by @thiswillbeyourgithub, 4 hours ago:
finish updating how env variables are handled including for managing the faiss scoring function
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/embeddings.py
- [d28e7e0] by @thiswillbeyourgithub, 4 hours ago:
new: replace LANGFUSE env var by wdoc's LANGFUSE var twice instead of just at runtime, just in case
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/misc.py
- [02d2f84] by @thiswillbeyourgithub, 5 hours ago:
new: crash if WDOC_LANGFUSE keys are set in env but langfuse fails to import
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [b037236] by @thiswillbeyourgithub, 5 hours ago:
more updating of the env handling
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
wdoc/utils/import_tricks.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/misc.py
wdoc/utils/tasks/query.py
wdoc/utils/typechecker.py
wdoc/wdoc.py
- [0266909] by @thiswillbeyourgithub, 5 hours ago:
minor: remove unused import
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/interact.py
wdoc/utils/logger.py
- [3f6fcd6] by @thiswillbeyourgithub, 5 hours ago:
the private env var checker is now redundant so removed
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/flags.py
- [d15fcd3] by @thiswillbeyourgithub, 5 hours ago:
fix: better way to check if asyncio nested patch should be applied or not
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/init.py
- [fe49bd9] by @thiswillbeyourgithub, 5 hours ago:
new: completely changed how env variables are handled
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/errors.py
- [bb52773] by @thiswillbeyourgithub, 13 days ago:
new: rename 'default' argument of query_retrievers to 'basic' because its not the default
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
wdoc/docs/help.md
wdoc/wdoc.py
- [3c47a57] by @thiswillbeyourgithub, 13 days ago:
docs: remove obsolete line about hyde retriever
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [20d8499] by @thiswillbeyourgithub, 2 weeks ago:
update todo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [4c4b0d8] by @thiswillbeyourgithub, 2 weeks ago:
update todo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [cb1024d] by @thiswillbeyourgithub, 2 weeks ago:
new: increase the max ac...