Release 3.3.0
What's new
This release focuses on adding DuckDuckGo web search capabilities and introducing binary embeddings support for more efficient vector storage.
✨ New Features
DuckDuckGo Web Search Integration
- [372fe57] Add DuckDuckGo search support with URL extraction and metadata
- [273195e] Support
wdoc wdb "your query"
shorthand for web search - [03bfe08] Add DuckDuckGo search tests and documentation
Binary Embeddings Support
- [c528bad] Add support for binary embeddings with 8x memory reduction
- [8f65197] Enable FAISS vectorstore compression by default
- [37ebd97] Create CompressedFAISS subclass with zlib compression
🐛 Bug Fixes
Core Functionality
- [0d72efd] Fix wrong decorator used for
load_one_doc
- [edcf671] Fix
ddg_region
type (str not int) - [66ab177] Fix type hints for
ddg_safesearch
andloading_failure
- [957936c] Use keyword arguments instead of fire when calling wdoc
Testing Environment
- [d3de58e] Fix piped input/output handling in pytest environment
- [42ff516] Prevent pipe usage in pytest environment
- [c78dc0b] Add pytest environment detection
🧪 Testing Improvements
- [1b09996] Fix the
run_all_test
script - [8ed1d0c] Add comprehensive DuckDuckGo search functionality tests
- [b184177] Split CLI tests into separate
test_cli.py
file - [9d7fe9c] Split parsing tests into separate
test_parsing.py
file - [12b012d] Move vector store tests to dedicated test file
📚 Documentation
- [d7d6b04] Explain how to run tests in README
- [dc15001] Clarify how to disable parallel processing
- [df4b79f] Document debug mode's effect on
loading_failure
default - [1832299] Add shell examples for DuckDuckGo usage
🔧 Enhancements
CLI/UX Improvements
- [7e994a6] Rename
parse_file
function toparse_doc
- [4aa247e] Re-ask for input when empty query provided in CLI
- [57d5d5f] Fix Fire's pager issue in CLI
Performance THISISANAMPERSAND Reliability
- [68d4c75] Bump LiteLLM to latest version for improved startup time
- [ab9c5e9] Add parallel processing option for Whisper audio splits
- [6b13044] Add loop counter and crash protection for recursive file processing
🔄 Version Update
- [6435133] Bump version from 3.2.5 → 3.3.0
Commits details since the last release
- [6435133] by @thiswillbeyourgithub, 36 minutes ago:
bump version 3.2.5 -> 3.3.0
bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py
- [1b09996] by @thiswillbeyourgithub, 24 hours ago:
test: fix the run_all_test script
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [d7d6b04] by @thiswillbeyourgithub, 24 hours ago:
doc: explain how to run the tests
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [62cc2ce] by @thiswillbeyourgithub, 24 hours ago:
fix: ddg test
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [0d72efd] by @thiswillbeyourgithub, 24 hours ago:
fix: wrong decorator used for load_one_doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [dc15001] by @thiswillbeyourgithub, 24 hours ago:
doc: clarify how to disable parallel processing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [e0453cb] by @thiswillbeyourgithub, 24 hours ago:
minor: mention a type hint
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [edcf671] by @thiswillbeyourgithub, 24 hours ago:
fix: ddg_region is actually a str not an int
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [df4b79f] by @thiswillbeyourgithub, 24 hours ago:
doc: mention that debug changes the default value for loading_failure
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [66ab177] by @thiswillbeyourgithub, 25 hours ago:
fix: type of ddg_safesearch and loading_failure should be Literal
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [98b0867] by @thiswillbeyourgithub, 25 hours ago:
doc: explain that loading_failure defaultto crash when parsing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [90eacb3] by @thiswillbeyourgithub, 25 hours ago:
test: ddg should use us region by default
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [c8b1944] by @thiswillbeyourgithub, 25 hours ago:
test: less severe check for pipes
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [6e12e5c] by @thiswillbeyourgithub, 25 hours ago:
test: remove one -n auto arg
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [d3de58e] by @thiswillbeyourgithub, 2 days ago:
fix: actually inside pytest we should not bypass piped input but only piped output
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
wdoc/utils/misc.py
- [5715bc4] by @thiswillbeyourgithub, 2 days ago:
test: add env variable to detect if being called by pytest
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
tests/conftest.py
- [42ff516] by @thiswillbeyourgithub, 2 days ago:
new: do not allow using pipe input or output in pytest environment
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [c78dc0b] by @thiswillbeyourgithub, 2 days ago:
new: detect when wdoc is called in pytest environment
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
wdoc/utils/env.py
wdoc/utils/misc.py
- [fca39c0] by @thiswillbeyourgithub, 2 days ago:
test: missing oneoff and failsafe when testing ddg
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [b2b4cf1] by @thiswillbeyourgithub, 2 days ago:
test: fix missing quotation sign for args
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [13409b1] by @thiswillbeyourgithub, 2 days ago:
test: fix a timeout not long enough
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [957936c] by @thiswillbeyourgithub, 2 days ago:
fix: use keyword aguments instead of fire when calling wdoc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b034337] by @thiswillbeyourgithub, 2 days ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b44d730] by @thiswillbeyourgithub, 2 days ago:
fix: replacing ddg_max_result
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [dfcaf3b] by @thiswillbeyourgithub, 2 days ago:
fix: wrong way to replace ddg_max_result to ddg_max_results
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [adc991a] by @thiswillbeyourgithub, 2 days ago:
actually no
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [9ab4cbf] by @thiswillbeyourgithub, 2 days ago:
fix: type hint of load_one_doc can be a list of string in case of error
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [5f7fcf4] by @thiswillbeyourgithub, 2 days ago:
typo: Nvidia instead of NVidia
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
tests/test_cli.py
wdoc/docs/examples.md
- [03bfe08] by @thiswillbeyourgithub, 2 days ago:
test: add test for ddg search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [48165fa] by @thiswillbeyourgithub, 2 days ago:
test: clearer echo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [08cac94] by @thiswillbeyourgithub, 2 days ago:
remove unused import
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_cli.py
- [3aabd2d] by @thiswillbeyourgithub, 2 days ago:
style: format test_cli.py with linter
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [8ed1d0c] by @thiswillbeyourgithub, 2 days ago:
feat: add test for DuckDuckGo search functionality with NVIDIA query
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [a8e3e04] by @thiswillbeyourgithub, 2 days ago:
test: add test for DuckDuckGo search with NVIDIA query
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
- [1832299] by @thiswillbeyourgithub, 2 days ago:
doc: add shell example for using duckduckgo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/examples.md
- [e6c4641] by @thiswillbeyourgithub, 2 days ago:
typo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/examples.md
- [917ee51] by @thiswillbeyourgithub, 2 days ago:
minor: test
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [04c3fdd] by @thiswillbeyourgithub, 2 days ago:
fix: I butchered the extra deps again didnt I
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [84dd100] by @thiswillbeyourgithub, 2 days ago:
doc: add echo to the test script
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [7f3a7bf] by @thiswillbeyourgithub, 2 days ago:
docs: add Args and Returns sections to parse_recursive_* functions
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4)
aider@aider.chat
wdoc/utils/batch_file_loader.py
- [c443608] by @thiswillbeyourgithub, 2 days ago:
doc: add docstrings for recursive parsers
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [5bb889f] by @thiswillbeyourgithub, 2 days ago:
enh: better way to do deterministic sorting
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [6f9672c] by @thiswillbeyourgithub, 2 days ago:
doc: add more docstring
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [c0623a7] by @thiswillbeyourgithub, 2 days ago:
fix: typo in setup.py
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [5d42574] by @thiswillbeyourgithub, 2 days ago:
doc: add docstring for batch_load_doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [5d73036] by @thiswillbeyourgithub, 2 days ago:
minor: remove unused imports
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [56dbb2f] by @thiswillbeyourgithub, 2 days ago:
test: make sure that all test files are executed
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [53d8b6d] by @thiswillbeyourgithub, 2 days ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [54efa41] by @thiswillbeyourgithub, 2 days ago:
fix: I butchered the extra deps again didnt I
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [8757c2a] by @thiswillbeyourgithub, 2 days ago:
doc: add precision for how auto can detect recursive filetypes
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [e1bcdd6] by @thiswillbeyourgithub, 3 days ago:
doc: sort filetype and add missing powerpoint doc
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [fadc4b8] by @thiswillbeyourgithub, 3 days ago:
doc: mention ddg in help.md
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [5497b81] by @thiswillbeyourgithub, 3 days ago:
doc: sort the filetype
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [2fec902] by @thiswillbeyourgithub, 3 days ago:
todo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [b4be9cd] by @thiswillbeyourgithub, 3 days ago:
doc: update the readme for the online search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [fecef51] by @thiswillbeyourgithub, 3 days ago:
doc: add default value for ddg args
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [a3aa26f] by @thiswillbeyourgithub, 3 days ago:
set default region for ddg to ''
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [1ff281d] by @thiswillbeyourgithub, 3 days ago:
doc: reformulation
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [3d8ce31] by @thiswillbeyourgithub, 3 days ago:
fix: test setup
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [8c44d89] by @thiswillbeyourgithub, 3 days ago:
typo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [64d920d] by @thiswillbeyourgithub, 3 days ago:
fix: filter out arguments expected by recursing parser but not by loaders
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [27aa480] by @thiswillbeyourgithub, 3 days ago:
enh: better way to call the appropriate recursive parser
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [73cf9d0] by @thiswillbeyourgithub, 3 days ago:
minor: always replace ddg_max_result to ddg_max_results
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [fc5c539] by @thiswillbeyourgithub, 3 days ago:
new: ignore warnings from httpx
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [273195e] by @thiswillbeyourgithub, 3 days ago:
feat: add support for 'wdoc wdb "your query"' to do web search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [1443468] by @thiswillbeyourgithub, 3 days ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [e67d786] by @thiswillbeyourgithub, 3 days ago:
fix: make sure query and path match the other in ddg search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [03b45ca] by @thiswillbeyourgithub, 3 days ago:
fix: add check to make sure that filetype ddg is only for query task
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [46e0de4] by @thiswillbeyourgithub, 3 days ago:
fix: forgot to close a file in interact.py
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/interact.py
- [998a111] by @thiswillbeyourgithub, 3 days ago:
remove ddg file that was used to figure out ddg search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
ddg_test.py
- [5b22491] by @thiswillbeyourgithub, 3 days ago:
add test file that helped create ddg search
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
ddg_test.py
- [6f14495] by @thiswillbeyourgithub, 3 days ago:
fix: subitem_link must be part of kwargs of load one doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [6ad4b25] by @thiswillbeyourgithub, 3 days ago:
fix: remove extra metadata from ddg
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/batch_file_loader.py
- [313f6a0] by @thiswillbeyourgithub, 3 days ago:
fix: missing subitem_link from allowed keys
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [a895464] by @thiswillbeyourgithub, 3 days ago:
test: allow parallel test run
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [ebfaefc] by @thiswillbeyourgithub, 3 days ago:
fix: extra require for pdftotext and fasttext were apparently broken
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
setup.py
tests/run_all_tests.sh
- [372fe57] by @thiswillbeyourgithub, 3 days ago:
feat: add DuckDuckGo search support with URL extraction and metadata
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
setup.py
wdoc/utils/batch_file_loader.py
wdoc/utils/misc.py
- [4e92974] by @thiswillbeyourgithub, 3 days ago:
fix: typo in run_all_tests
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [d292048] by @thiswillbeyourgithub, 3 days ago:
better default for pytest
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [b184177] by @thiswillbeyourgithub, 4 days ago:
Based on the changes you've made, I'll generate a concise commit message that captures the essence of the refactoring:
refactor: split CLI tests into separate test_cli.py file
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_cli.py
tests/test_parsing.py
tests/test_wdoc.py
- [9d7fe9c] by @thiswillbeyourgithub, 4 days ago:
refactor: Split parsing tests into separate test_parsing.py file
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_parsing.py
tests/test_wdoc.py
- [12b012d] by @thiswillbeyourgithub, 4 days ago:
feat: move BinaryFAISS and CompressedFAISS tests to tests/test_vectorstores.py
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_vectorstores.py
tests/test_wdoc.py
- [5f4b001] by @thiswillbeyourgithub, 4 days ago:
test: add test to verify different distances in binary vs regular FAISS
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [240b7cf] by @thiswillbeyourgithub, 4 days ago:
test: Add sanity check for BinaryFAISS semantic relationship preservation
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [f1678bb] by @thiswillbeyourgithub, 5 days ago:
fix: add tolerance to binary vector conversion for consistent embeddings
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [970ee7a] by @thiswillbeyourgithub, 5 days ago:
fix: ensure consistent binary embedding conversion for same query
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
wdoc/utils/customs/binary_faiss_vectorstore.py
- [fdae3b6] by @thiswillbeyourgithub, 5 days ago:
fix typehint
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [b34cd2f] by @thiswillbeyourgithub, 8 days ago:
fix: Replace mistral embeddings with OpenAI embeddings in test cases
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [33912f1] by @thiswillbeyourgithub, 8 days ago:
refactor: Improve binary embedding conversion with global zero threshold
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
wdoc/utils/customs/binary_faiss_vectorstore.py
- [7438238] by @thiswillbeyourgithub, 9 days ago:
refactor: improve binary vector conversion using per-vector mean thresholding
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [eb1961c] by @thiswillbeyourgithub, 9 days ago:
refactor: improve binary vector conversion with zero-centered thresholding
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [f52d11b] by @thiswillbeyourgithub, 9 days ago:
refactor: improve binary vector conversion using median thresholding
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [5029fc1] by @thiswillbeyourgithub, 9 days ago:
refactor: fix binary FAISS embedding conversion and dimension handling
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [baf90f7] by @thiswillbeyourgithub, 9 days ago:
fix: resolve binary embedding dimension mismatch in MMR search
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [0b51ecb] by @thiswillbeyourgithub, 9 days ago:
fix: correct binary embedding conversion in max_marginal_relevance_search
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [cebaaf9] by @thiswillbeyourgithub, 9 days ago:
fix: catch exception in pytest
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [de7ee1a] by @thiswillbeyourgithub, 9 days ago:
fix: harmonize error message for binary faiss
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [fcad24f] by @thiswillbeyourgithub, 9 days ago:
fix: catch exceptions
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [2fe5680] by @thiswillbeyourgithub, 9 days ago:
refactor: Move edge case tests from separate function to test_binary_faiss_functionality
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [bc068a3] by @thiswillbeyourgithub, 9 days ago:
feat: add comprehensive edge case tests for BinaryFAISS vectorstore
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [a6b3dfc] by @thiswillbeyourgithub, 9 days ago:
test: Add comprehensive edge case tests for BinaryFAISS vectorstore
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [6212501] by @thiswillbeyourgithub, 9 days ago:
fix: loading a binary index should use runtimeerror not typeerror
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [8e89b5f] by @thiswillbeyourgithub, 9 days ago:
feat: add test for BinaryFAISS semantic relationship preservation
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [68aa7c9] by @thiswillbeyourgithub, 9 days ago:
test: add comprehensive test for CompressedFAISS functionality
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [161e020] by @thiswillbeyourgithub, 9 days ago:
forgot to commit something to binary faiss
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [cafb9b3] by @thiswillbeyourgithub, 9 days ago:
fix: standardize the question used in pytest
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [b4fc7a8] by @thiswillbeyourgithub, 9 days ago:
fix: fiddle with the caching of embeddings during pytest
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [44a3921] by @thiswillbeyourgithub, 9 days ago:
remove the verbose flag for the pytest
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [4dec6d3] by @thiswillbeyourgithub, 9 days ago:
bind litellm version to 1.72.4
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [972db56] by @thiswillbeyourgithub, 9 days ago:
fix: add trio as a req to avoid issue with python 3.11
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [d790b98] by @thiswillbeyourgithub, 10 days ago:
fix: better tests
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [b2afc84] by @thiswillbeyourgithub, 10 days ago:
test: fix outdated embedding model for ollama
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [18174d4] by @thiswillbeyourgithub, 10 days ago:
new: in pytest don't exitfirst
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/run_all_tests.sh
- [e675e90] by @thiswillbeyourgithub, 10 days ago:
refactor: update _vec_to_binary to handle 1D vector conversion correctly
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
wdoc/utils/customs/binary_faiss_vectorstore.py
- [110fc77] by @thiswillbeyourgithub, 10 days ago:
fix: embedding function was getting overriden when using binary faiss
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [a578aae] by @thiswillbeyourgithub, 10 days ago:
pin the versions of langchain etc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [f373f4a] by @thiswillbeyourgithub, 10 days ago:
fix: custom embeddings were broken
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [6719045] by @thiswillbeyourgithub, 10 days ago:
fix: pin version of langfuse
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [47913f5] by @thiswillbeyourgithub, 10 days ago:
fix: save and load binary index
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [39b95ac] by @thiswillbeyourgithub, 10 days ago:
add comment for langfuse version
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [8a19510] by @thiswillbeyourgithub, 10 days ago:
minor: use uppercase when checking env variables
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [647f624] by @thiswillbeyourgithub, 10 days ago:
new: import ChatLiteLLM from langchain_litellm instead of langchain_community
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
wdoc/utils/llm.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/query.py
- [94218aa] by @thiswillbeyourgithub, 10 days ago:
minor: bump latest langfuse version
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [d32edba] by @thiswillbeyourgithub, 10 days ago:
minor: add start check for incompatible faiss env value
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [7e493e5] by @thiswillbeyourgithub, 10 days ago:
fix: for the test, because we enable binary we have to disable the custom scoring function
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [be72d53] by @thiswillbeyourgithub, 10 days ago:
fix: check_docs_tkn_length 's identifier arg should be anything that can be stringified so removing type hint
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [c46da81] by @thiswillbeyourgithub, 10 days ago:
minor: better exception handling for exception when loading docx
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [61b2ee1] by @thiswillbeyourgithub, 10 days ago:
fix: add User-Agent header to resolve 403 error in requests download
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [063d693] by @thiswillbeyourgithub, 10 days ago:
refactor: replace subprocess with requests and add SHA512 hash verification for DOCX test
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) aider@aider.chat
tests/test_wdoc.py
- [24990bc] by @thiswillbeyourgithub, 2 weeks ago:
new: clarify that the reference python version is 3.12.11
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
setup.py
tests/run_all_tests.sh
- [68d4c75] by @thiswillbeyourgithub, 3 weeks ago:
perf: bump litellm to latest as they improved startup time by a lot
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [7f429ab] by @thiswillbeyourgithub, 4 weeks ago:
todo: update
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [4aa247e] by @thiswillbeyourgithub, 4 weeks ago:
new: re ask in the cli if empty query
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/interact.py
- [5e4c963] by @thiswillbeyourgithub, 4 weeks ago:
enh: reworked how the loop between cli and query is implemented
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [8fe7a95] by @thiswillbeyourgithub, 4 weeks ago:
fix: add check for cli while piped
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/interact.py
- [d600b59] by @thiswillbeyourgithub, 4 weeks ago:
add check for empty queries
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [191de44] by @thiswillbeyourgithub, 4 weeks ago:
fix: add check for non empty strings to embed
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/litellm_embeddings.py
- [25c3de2] by @thiswillbeyourgithub, 4 weeks ago:
rename env variable for binary and compression
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/embeddings.py
wdoc/utils/env.py
- [b703588] by @thiswillbeyourgithub (aider), 4 weeks ago:
feat: add fallback mechanism for loading compressed FAISS vectorstore
wdoc/docs/help.md
wdoc/utils/customs/binary_faiss_vectorstore.py
- [8f65197] by @thiswillbeyourgithub, 4 weeks ago:
feat: add compression for faiss vectorstores by default
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
wdoc/docs/help.md
wdoc/utils/embeddings.py
wdoc/utils/env.py
- [e60b329] by @thiswillbeyourgithub (aider), 4 weeks ago:
style: format code with linter
wdoc/utils/customs/binary_faiss_vectorstore.py
- [37ebd97] by @thiswillbeyourgithub (aider), 4 weeks ago:
refactor: create CompressedFAISS subclass with compression methods
wdoc/utils/customs/binary_faiss_vectorstore.py
- [c18133a] by @thiswillbeyourgithub (aider), 4 weeks ago:
feat: add zlib compression for pickle data in save_local and load_local methods
wdoc/utils/customs/binary_faiss_vectorstore.py
- [b02a07e] by @thiswillbeyourgithub, 4 weeks ago:
remove beartype in binary faiss vectorstore
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [ddeeb00] by @thiswillbeyourgithub (aider), 4 weeks ago:
feat: use Hamming score function by default for binary embeddings
wdoc/utils/customs/binary_faiss_vectorstore.py
- [a477277] by @thiswillbeyourgithub, 4 weeks ago:
new: now set the faiss score function to True by default
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/env.py
- [673cc6f] by @thiswillbeyourgithub, 4 weeks ago:
fix: add check to catch "no answer" exception
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [8901894] by @thiswillbeyourgithub (aider), 4 weeks ago:
fix: correct binary embedding dimension calculation for FAISS index
wdoc/utils/customs/binary_faiss_vectorstore.py
- [50b3b79] by @thiswillbeyourgithub, 4 weeks ago:
add docstring at the top
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [e806494] by @thiswillbeyourgithub, 4 weeks ago:
fix: always convert to binarythe same way
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [cac38a3] by @thiswillbeyourgithub, 4 weeks ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [9f6a780] by @thiswillbeyourgithub, 4 weeks ago:
fix: typehint of vec tobinary
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [bc14635] by @thiswillbeyourgithub, 4 weeks ago:
fix: classmethod
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [64448a5] by @thiswillbeyourgithub, 4 weeks ago:
remove a check for binary embeddings
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [2fdef8d] by @thiswillbeyourgithub, 4 weeks ago:
fix: type hint for np array
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [8735d5e] by @thiswillbeyourgithub (aider), 4 weeks ago:
feat: add async embedding function for binary vectorstore
wdoc/utils/customs/binary_faiss_vectorstore.py
- [ce78aef] by @thiswillbeyourgithub, 4 weeks ago:
doc: mention that binary embeddings needs dimensions multiple of 8 and to not change the binary usage in between runs
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [e11fe23] by @thiswillbeyourgithub, 4 weeks ago:
fix: progress on binary embeddings
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/customs/binary_faiss_vectorstore.py
- [3242d4d] by @thiswillbeyourgithub, 4 weeks ago:
fix: don't try to bypass the embeddings cache in case of errors when no cache is used anyway
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/embeddings.py
- [ad8ebeb] by @thiswillbeyourgithub, 4 weeks ago:
doc: mention binary embeddings as a feature
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [bebf6f4] by @thiswillbeyourgithub, 4 weeks ago:
test by default should use binary embeddings
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
tests/test_wdoc.py
- [c528bad] by @thiswillbeyourgithub, 4 weeks ago:
feat: add support for binary embeddings
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/customs/binary_faiss_vectorstore.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
- [d338d25] by @thiswillbeyourgithub, 4 weeks ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/embeddings.py
- [7b9846d] by @thiswillbeyourgithub, 4 weeks ago:
minor
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [a7eb6c4] by @thiswillbeyourgithub, 4 weeks ago:
fix: wrong cache dir created for faiss
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/embeddings.py
- [e125e87] by @thiswillbeyourgithub, 4 weeks ago:
doc: clarify why we replaced the scoring function of langchain's faiss
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [faeefe2] by @thiswillbeyourgithub, 5 weeks ago:
minor: remove .md from license
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
LICENSE
- [d17f1f2] by @thiswillbeyourgithub, 5 weeks ago:
update completions
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
shell_completions/wdoc_completion.bash
shell_completions/wdoc_completion.fish
shell_completions/wdoc_completion.zsh
- [14a3d6c] by @thiswillbeyourgithub, 5 weeks ago:
fix: call for --completion
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b813cf6] by @thiswillbeyourgithub, 5 weeks ago:
fix: -- --completion was crashing the cli
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [ca4082a] by @thiswillbeyourgithub, 5 weeks ago:
todo
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
README.md
- [a822d71] by @thiswillbeyourgithub, 6 weeks ago:
minor: use pathlib a bit more for path argument
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [6bca7ff] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add fallback mechanism for whisper transcription using requests
wdoc/utils/loaders.py
- [0d79b66] by @thiswillbeyourgithub, 6 weeks ago:
enh: add one more check for private mode api key removal
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [80343b5] by @thiswillbeyourgithub, 6 weeks ago:
new: when setting private mode, don't overwrite API keys that start with WDOC_
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [770ff3d] by @thiswillbeyourgithub, 6 weeks ago:
new: allow using whisper in private mode if using a specific backend
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [2f047bf] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add custom whisper endpoint and API key environment variables
wdoc/docs/help.md
wdoc/utils/env.py
wdoc/utils/loaders.py
- [f34a176] by @thiswillbeyourgithub, 6 weeks ago:
fix: env var typo
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/env.py
- [45d3355] by @thiswillbeyourgithub, 6 weeks ago:
black
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
setup.py
- [6917687] by @thiswillbeyourgithub, 6 weeks ago:
fix: finding subtitles for youtube videos
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [8070f2d] by @thiswillbeyourgithub, 6 weeks ago:
fix: don't include empty sub files
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [7e994a6] by @thiswillbeyourgithub, 6 weeks ago:
new: rename the function parse_file to parse_doc
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
MANIFEST.in
README.md
docs/source/index.rst
docs/source/parse_doc_help.md
docs/source/parse_doc_help.rst
docs/source/parse_file_help.md
docs/source/single_page_doc.rst
tests/test_wdoc.py
wdoc/main.py
wdoc/docs/help.md
wdoc/docs/parse_doc_help.md
wdoc/utils/logger.py
wdoc/wdoc.py
- [853f224] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: simplify yt-dlp installation to use pip pre-release
setup.py
- [86d7988] by @thiswillbeyourgithub, 6 weeks ago:
apply black
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [bacd7c4] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: restructure parse_file to handle out_file after format processing
wdoc/wdoc.py
- [4dd801d] by @thiswillbeyourgithub, 6 weeks ago:
new: now audio_unsilence defaults to True
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
wdoc/utils/loaders.py
- [94b0755] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: Improve argument formatting in error messages for better readability
wdoc/utils/loaders.py
- [c98d645] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: moveformat_args_with_types
function before its usage
wdoc/utils/loaders.py
- [ebcbf90] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add type hints to loader function's valid arguments error message
wdoc/utils/loaders.py
- [e4eadba] by @thiswillbeyourgithub, 6 weeks ago:
rename runtime_arguments to wdoc_global_arguments
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [7092284] by @thiswillbeyourgithub, 6 weeks ago:
fix: wrong list of runtime args
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [ed59a86] by @thiswillbeyourgithub, 6 weeks ago:
rename error to MissingDocDictArguments
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/errors.py
wdoc/utils/loaders.py
- [7eedf18] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add validation for unexpected loader function arguments
wdoc/utils/loaders.py
- [9fb3efc] by @thiswillbeyourgithub, 6 weeks ago:
enh: better error message for docdict error
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [fdf921b] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: enhance error messages with optional argument types in loaders
wdoc/utils/loaders.py
- [57d5d5f] by @thiswillbeyourgithub, 6 weeks ago:
new: add fix for fire's annoying pager issue
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/main.py
- [b274a00] by @thiswillbeyourgithub, 6 weeks ago:
enh: remove the path arguments of parse_file because its a docdict
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/parse_file_help.md
wdoc/wdoc.py
- [7595570] by @thiswillbeyourgithub, 6 weeks ago:
doc: clarify doc on docdict arguments
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/help.md
- [d13a5f6] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add --out_file option to parse_file for file output with binary file check
wdoc/main.py
wdoc/docs/parse_file_help.md
wdoc/wdoc.py
- [f694039] by @thiswillbeyourgithub, 6 weeks ago:
feat: add validation to the args of cli parser
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/docs/parse_file_help.md
wdoc/wdoc.py
- [6b13044] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add loop counter and crash protection for recursive file type processing
wdoc/utils/batch_file_loader.py
- [0c247e0] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: improve type hint formatting to show complete type information
wdoc/utils/loaders.py
- [c0d0d9a] by @thiswillbeyourgithub, 6 weeks ago:
enh: better load doc crash error for missing arguments
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [59b53bc] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: replaceload_one_doc_wrapped
with decoratorwrapper_load_one_doc
wdoc/utils/batch_file_loader.py
wdoc/utils/loaders.py
- [4b9af96] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: enhance error messages with argument type hints in load_one_doc
wdoc/utils/loaders.py
- [54a55b9] by @thiswillbeyourgithub (aider), 6 weeks ago:
fix: improve error handling for missing runtime and user arguments
wdoc/utils/loaders.py
- [7541b97] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: Improve argument handling in load_one_doc with runtime and user arg separation
wdoc/utils/loaders.py
- [e1688ba] by @thiswillbeyourgithub, 6 weeks ago:
enh: better trial of ModelName attributes to try to get the model max token
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [fde9c4f] by @thiswillbeyourgithub, 6 weeks ago:
black
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [40f486b] by @thiswillbeyourgithub, 6 weeks ago:
enh: better exception handling
wdoc/utils/loaders.py
wdoc/utils/misc.py
- [9b88aee] by @thiswillbeyourgithub, 6 weeks ago:
fix: cli parser max token checker was wrong
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
- [c5912ff] by @thiswillbeyourgithub (aider), 6 weeks ago:
refactor: Implement dynamic loader function mapping with signature inspection
wdoc/utils/loaders.py
- [7095282] by @thiswillbeyourgithub, 6 weeks ago:
enh: use logger exception instead of warning for exception handling
Signed-off-by: thiswillbeyourgithub
26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [c1b0378] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add DocLoadMissingArguments exception class for document loader errors
wdoc/utils/errors.py
wdoc/utils/loaders.py
- [ab9c5e9] by @thiswillbeyourgithub (aider), 6 weeks ago:
feat: add parallel processing option for whisper audio splits with joblib
wdoc/docs/help.md
wdoc/utils/env.py
wdoc/utils/loaders.py
- [9af14ae] by @thiswillbeyourgithub (aider), 6 weeks ago:
fix: Resolve variable shadowing of built-in sum function
wdoc/wdoc.py
- [e21b366] by @thiswillbeyourgithub, 6 weeks ago:
fix: compute and return the sum of total tokens (no matter if thinking or whatever)
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/wdoc.py
- [581efc5] by @thiswillbeyourgithub, 6 weeks ago:
enh: add explanatory error message to fix missing libsox.so
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [24fffa6] by @thiswillbeyourgithub, 6 weeks ago:
enh: add a warning if audio silence did not change the duration
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [493e5f5] by @thiswillbeyourgithub, 7 weeks ago:
feat: allow larger document size by default when using the parser function instead of other tasks
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/misc.py
wdoc/wdoc.py
- [136b428] by @thiswillbeyourgithub, 7 weeks ago:
fix: dont ask for french subtitles by default in youtube
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [0c53967] by @thiswillbeyourgithub, 7 weeks ago:
yt: better yet subtitles parsing
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py
- [ef6b74b] by @thiswillbeyourgithub (aider), 7 weeks ago:
refactor: extract VTT content processing logic into separate function
wdoc/utils/loaders.py
- [20f0498] by @thiswillbeyourgithub, 7 weeks ago:
enh: better formatting for vtt as text (yt-dlp)
Signed-off-by: thiswillbeyourgithub 26625900+thiswillbeyourgithub@users.noreply.github.com
wdoc/utils/loaders.py