Skip to content

Dev -> Main Merge #572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 391 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 147 commits
Commits
Show all changes
391 commits
Select commit Hold shift + click to select a range
a77eb7d
Update RSS_Ranking.md
rmusser01 Apr 26, 2025
d1d82eb
failed at adding a titlebar to chatbook
rmusser01 Apr 27, 2025
cd8fa52
Well, chat endpoint should work now...
rmusser01 Apr 28, 2025
db0234b
test fix
rmusser01 Apr 28, 2025
9c280f7
Update LLM_API_Calls.py
rmusser01 Apr 28, 2025
5940337
fucking python
rmusser01 Apr 28, 2025
387ef20
Sync Engine - Client
rmusser01 Apr 29, 2025
4e31e55
Sync engine + tests
rmusser01 Apr 29, 2025
6b4d5af
removing old sync
rmusser01 Apr 29, 2025
eec8159
docs
rmusser01 May 3, 2025
3110fe6
sql
rmusser01 May 3, 2025
faeb6ff
SQL Temp
rmusser01 May 3, 2025
d50f5a5
holy fuck cluster headaches
rmusser01 May 3, 2025
936ba55
squashing bugs
rmusser01 May 3, 2025
a0b9bc2
wew
rmusser01 May 3, 2025
8faf0ea
fml
rmusser01 May 3, 2025
9d05d20
f
rmusser01 May 3, 2025
cb02107
Update test_sync_server.py
rmusser01 May 3, 2025
d4aad95
holy fuck. only 8? hours later, it all fucking works.
rmusser01 May 3, 2025
a143407
cleanup/moving
rmusser01 May 3, 2025
88fca28
before
rmusser01 May 3, 2025
36600de
f
rmusser01 May 3, 2025
b4339c1
a little bit closer. also, fuck sqlite and FTS and why didn't I just …
rmusser01 May 3, 2025
d7bc8fd
3/46 failing, not corruption issues this time...
rmusser01 May 3, 2025
d576436
Update test_sqlite_db.py
rmusser01 May 3, 2025
cff3265
this is some bullshit
rmusser01 May 3, 2025
992d1d3
holy fuck.
rmusser01 May 3, 2025
7ef6cab
Update Media_DB_v2.py
rmusser01 May 4, 2025
5f46f94
docstrings for every DB function
rmusser01 May 4, 2025
a947cf6
holy fuck it still works.
rmusser01 May 4, 2025
d11e4be
and it launches
rmusser01 May 4, 2025
f955eb8
split chatbook into its own repo
rmusser01 May 4, 2025
09ce6fb
DB paths
rmusser01 May 4, 2025
88324be
f
rmusser01 May 4, 2025
6d57267
6/24 Endpoints
rmusser01 May 4, 2025
9ae996b
17 passing for endpoints, 49 passing for DB2
rmusser01 May 4, 2025
da0a9ad
Update media.py
rmusser01 May 4, 2025
d8461f9
another down
rmusser01 May 4, 2025
2bd7b03
another down
rmusser01 May 5, 2025
0a7192f
And one test loosened but passing
rmusser01 May 5, 2025
dba519f
starting media processing tests
rmusser01 May 5, 2025
820026f
Update test_media_processing.py
rmusser01 May 5, 2025
edfea8e
Completely removing MediaDB v1
rmusser01 May 5, 2025
3fc80cb
refactor
rmusser01 May 7, 2025
70535cb
Ingestion back to working
rmusser01 May 7, 2025
26c07c7
media_versions fully passing
rmusser01 May 7, 2025
197e1f2
Media processing tests all passing
rmusser01 May 7, 2025
6e01084
Update test_media_versions.py
rmusser01 May 7, 2025
0abcf69
Delete config.txt
rmusser01 May 7, 2025
39d0801
eh
rmusser01 May 7, 2025
bec1063
and all tests pass
rmusser01 May 8, 2025
0b7a62f
eh
rmusser01 May 8, 2025
96e60ae
removed json file refe
rmusser01 May 9, 2025
eaaed82
json
rmusser01 May 9, 2025
b77fb2c
some stuff
rmusser01 May 9, 2025
6d90e55
Update Chunk_Lib.py
rmusser01 May 9, 2025
12d92e0
Update python-tldw.yml
rmusser01 May 9, 2025
3fa11bb
and there we go
rmusser01 May 9, 2025
c779fe9
fixes
rmusser01 May 9, 2025
c0f9f39
yay and we got basic processing through the API in single-user-mode v…
rmusser01 May 9, 2025
23baf71
quick fix for the html API tester
rmusser01 May 9, 2025
7e58218
Update api_tester.html
rmusser01 May 9, 2025
c4ffb91
chat
rmusser01 May 9, 2025
6196c37
tests & search endpoint & stuff
rmusser01 May 10, 2025
e5bc859
Create TTS-README.md
rmusser01 May 10, 2025
48d19db
RAG API
rmusser01 May 11, 2025
e5d8d68
cleanup
rmusser01 May 11, 2025
56409c6
Update Media_DB_v2.py
rmusser01 May 12, 2025
12d30ca
Chats/Characters/Notes
rmusser01 May 12, 2025
05dc2fb
ChaChaNotes
rmusser01 May 12, 2025
7142ce2
chacha
rmusser01 May 13, 2025
b62dc1b
testing
rmusser01 May 13, 2025
de8a73e
Update python-poc-tldw.yml
rmusser01 May 13, 2025
41e392e
Update python-poc-tldw.yml
rmusser01 May 13, 2025
16085ab
Update python-poc-tldw.yml
rmusser01 May 13, 2025
62e9c84
more chacha concurrency issues
rmusser01 May 13, 2025
58d6e67
Fixes
rmusser01 May 13, 2025
08cbfb4
wew.
rmusser01 May 13, 2025
8d045ed
f
rmusser01 May 13, 2025
27dd815
f
rmusser01 May 13, 2025
b2f6fcb
f
rmusser01 May 13, 2025
ce6c71e
f
rmusser01 May 13, 2025
aa8a9de
f
rmusser01 May 13, 2025
350af62
f
rmusser01 May 13, 2025
9df30ec
f
rmusser01 May 13, 2025
55570a7
Update ChaChaNotes_DB.py
rmusser01 May 13, 2025
764c90e
f
rmusser01 May 14, 2025
f74cc44
39/40
rmusser01 May 14, 2025
015a673
f
rmusser01 May 14, 2025
f121b76
f
rmusser01 May 14, 2025
dfd0459
ff
rmusser01 May 14, 2025
90fc183
fff
rmusser01 May 14, 2025
7e4a75f
F
rmusser01 May 14, 2025
4e59926
Update ChaChaNotes_DB.py
rmusser01 May 15, 2025
08db1c0
Update ChaChaNotes_DB.py
rmusser01 May 15, 2025
34827ca
before
rmusser01 May 15, 2025
041fe3d
well, put gemini next to sonnet as another pile of shit
rmusser01 May 15, 2025
4128a45
well fuck me
rmusser01 May 15, 2025
ddf5177
Update ChaChaNotes_DB.py
rmusser01 May 15, 2025
caf3f7a
wow, o1 wins
rmusser01 May 15, 2025
edeb975
llm-gen'd docs
rmusser01 May 15, 2025
23533e1
rag library
rmusser01 May 16, 2025
acee69d
f
rmusser01 May 16, 2025
d03c805
start refactoring chat + image supprt
rmusser01 May 16, 2025
9dd915b
Update LICENSE.txt
rmusser01 May 16, 2025
ea6e367
why yes, I am that retarded
rmusser01 May 16, 2025
a148dec
chat
rmusser01 May 17, 2025
3d79cba
Update Chat_Functions.py
rmusser01 May 17, 2025
f0dcf2c
chat refactor
rmusser01 May 17, 2025
f002137
chat templates
rmusser01 May 17, 2025
0d00ddf
chat tests
rmusser01 May 17, 2025
d95ac8f
Update test_chat_endpoint.py
rmusser01 May 17, 2025
56d9fe6
fixes
rmusser01 May 17, 2025
4408b35
bugfixes
rmusser01 May 17, 2025
14a7083
Update LLM_API_Calls.py
rmusser01 May 17, 2025
40e5748
Update test_chat_functions.py
rmusser01 May 17, 2025
8d3d354
scaffolding for logprobs/etc later
rmusser01 May 17, 2025
5a6cdd0
config filepath
rmusser01 May 17, 2025
66de1d6
f
rmusser01 May 17, 2025
1ab6d54
tests
rmusser01 May 17, 2025
92a0c77
Update test_chat_endpoint.py
rmusser01 May 17, 2025
4380a32
wew
rmusser01 May 17, 2025
c34f3c1
hf
rmusser01 May 17, 2025
64ce34f
Update config.py
rmusser01 May 17, 2025
7625571
f
rmusser01 May 17, 2025
577e723
Update prompt_template_manager.py
rmusser01 May 17, 2025
e5ed536
character 'aware'
rmusser01 May 17, 2025
17784e6
chat params
rmusser01 May 17, 2025
b485c4c
weet
rmusser01 May 17, 2025
9dee12b
f
rmusser01 May 17, 2025
4f58112
chunking
rmusser01 May 17, 2025
8d5275c
chunking
rmusser01 May 17, 2025
9db2388
chat
rmusser01 May 17, 2025
b2c7779
Update Book_Processing_Lib.py
rmusser01 May 17, 2025
cb5e0bf
chunking
rmusser01 May 17, 2025
76dc36e
chunking endpoint + tests
rmusser01 May 18, 2025
bcce529
j
rmusser01 May 18, 2025
a0ba12a
f
rmusser01 May 18, 2025
b8f1f1a
Update Chunk_Lib.py
rmusser01 May 18, 2025
aacd310
character chat
rmusser01 May 18, 2025
b881a7c
Update Character_Chat_Lib.py
rmusser01 May 18, 2025
019bfca
PoC fixes
rmusser01 May 18, 2025
fe6c30b
PoC
rmusser01 May 18, 2025
9c581b6
PoC version fixups
rmusser01 May 18, 2025
d095f0b
PoC version now works like before
rmusser01 May 18, 2025
a3807d5
PoC
rmusser01 May 18, 2025
b325315
Update python-tldw.yml
rmusser01 May 18, 2025
e1ea47a
Update python-tldw.yml
rmusser01 May 18, 2025
d5d9048
installer scripts
rmusser01 May 18, 2025
476cedc
tests
rmusser01 May 18, 2025
583494a
Update LLM_API_Calls.py
rmusser01 May 18, 2025
656fa6f
Update requirements.txt
rmusser01 May 18, 2025
eb65c19
Update requirements.txt
rmusser01 May 18, 2025
8f4f414
Update RAG_Library_2.py
rmusser01 May 18, 2025
4cb8325
uhhh
rmusser01 May 18, 2025
275f284
fixup
rmusser01 May 18, 2025
9bf7aaa
f
rmusser01 May 18, 2025
3d096cf
Update config.txt
rmusser01 May 18, 2025
74950ae
Update python-tldw.yml
rmusser01 May 18, 2025
7905d9f
Update python-tldw.yml
rmusser01 May 18, 2025
ee8e2c1
ugh
rmusser01 May 18, 2025
4606b1a
tests
rmusser01 May 18, 2025
0ce9a52
fixup
rmusser01 May 18, 2025
6b6f59c
Update characters.py
rmusser01 May 18, 2025
bea67a8
Update python-tldw.yml
rmusser01 May 18, 2025
88802c1
Update Embeddings_Create.py
rmusser01 May 18, 2025
0ce97a8
I love fixing tests
rmusser01 May 18, 2025
8332780
Update python-tldw.yml
rmusser01 May 18, 2025
7a25381
Update main.py
rmusser01 May 18, 2025
5d001ff
Update python-tldw.yml
rmusser01 May 18, 2025
8dce03d
Notes + chacha
rmusser01 May 18, 2025
e6d3d86
chat functionality + broken chat
rmusser01 May 18, 2025
713f35b
tests
rmusser01 May 18, 2025
5fd0944
progress
rmusser01 May 18, 2025
7c19811
tests
rmusser01 May 18, 2025
81a2e01
tests
rmusser01 May 18, 2025
06ef314
f
rmusser01 May 19, 2025
b949931
f
rmusser01 May 19, 2025
4db2320
Update LLM_API_Calls.py
rmusser01 May 19, 2025
398815c
f
rmusser01 May 19, 2025
db1fa2d
Update test_chat_completions_integration.py
rmusser01 May 19, 2025
a788ffc
test
rmusser01 May 19, 2025
ef532eb
fixups
rmusser01 May 19, 2025
0762f94
fixed chunking
rmusser01 May 19, 2025
8afe0f6
wew. test_media_processing done
rmusser01 May 19, 2025
86562da
fixed
rmusser01 May 19, 2025
310e86c
docs
rmusser01 May 19, 2025
7133059
docs
rmusser01 May 19, 2025
388fac4
fix
rmusser01 May 20, 2025
7165c81
Create .env
rmusser01 May 20, 2025
ae5d68c
Update Embeddings_Create.py
rmusser01 May 20, 2025
bd4b474
Moving on from teh PoC
rmusser01 May 20, 2025
539f739
Update python-tldw.yml
rmusser01 May 20, 2025
e034e91
f
rmusser01 May 20, 2025
7e99ec3
Update python-tldw.yml
rmusser01 May 20, 2025
f95b6a0
mediawiki
rmusser01 May 20, 2025
9fe2901
arxiv, semantic scholar, mediawiki
rmusser01 May 20, 2025
1e9562b
Update requirements.txt
rmusser01 May 20, 2025
fef903c
reqs
rmusser01 May 20, 2025
e7465e4
search endpoint
rmusser01 May 20, 2025
ee8ab95
Update config.py
rmusser01 May 20, 2025
270c63d
Update config.py
rmusser01 May 20, 2025
ed22087
Changed `Database` to `MediaDatabase`
rmusser01 May 20, 2025
d9a1f24
F
rmusser01 May 20, 2025
c786795
wew, now pycharms breaking
rmusser01 May 20, 2025
61c76c1
wew
rmusser01 May 20, 2025
d1cbbfd
wew
rmusser01 May 20, 2025
f90fa5e
uh
rmusser01 May 20, 2025
c0cbf69
Rename LICENSE.txt to LICENSE.md
rmusser01 May 20, 2025
3e9a0b1
Rename LICENSE.md to LICENSE.txt
rmusser01 May 20, 2025
6a554e9
notes
rmusser01 May 21, 2025
f4caecf
Create __init__.py
rmusser01 May 21, 2025
01ed5cb
Merge branch 'dev' of https://github.com/rmusser01/tldw into dev
rmusser01 May 21, 2025
4a64c71
chat
rmusser01 May 21, 2025
5e39b05
tests
rmusser01 May 21, 2025
49603e7
tests
rmusser01 May 21, 2025
caed4e8
fucking hate bullshit test
rmusser01 May 21, 2025
a46126a
really fucking hate tests
rmusser01 May 21, 2025
1b8abff
wew
rmusser01 May 21, 2025
95f7945
f
rmusser01 May 21, 2025
a9cdd23
Update test_prompts_api.py
rmusser01 May 21, 2025
54ab23c
Fix multiple test failures in Prompt Management
google-labs-jules[bot] May 22, 2025
2bcc1e6
tests
rmusser01 May 22, 2025
647b6c4
Merge pull request #612 from rmusser01/fix/prompt-management-tests - …
rmusser01 May 22, 2025
10cc81d
Update Prompts_DB.py
rmusser01 May 22, 2025
249160f
docs
rmusser01 May 23, 2025
22e60e2
uh
rmusser01 May 23, 2025
62f8f42
tests
rmusser01 May 23, 2025
2082613
prompts
rmusser01 May 24, 2025
c1b4f44
prompts
rmusser01 May 24, 2025
4431a79
prompts endpoint fixed
rmusser01 May 24, 2025
3ffb41e
Update test_prompts_api.py
rmusser01 May 24, 2025
4569570
woops
rmusser01 May 24, 2025
15e3587
chat fixes
rmusser01 May 24, 2025
7efba0c
Update requirements.txt
rmusser01 May 24, 2025
c285ff0
chroma+embeddings
rmusser01 May 24, 2025
8949af6
Update main.py
rmusser01 May 24, 2025
e5e29e3
Update characters.py
rmusser01 May 24, 2025
3ef1524
tests
rmusser01 May 24, 2025
63da63b
Update test_character_functionality_db.py
rmusser01 May 24, 2025
63f387e
Update ChaChaNotes_DB.py
rmusser01 May 24, 2025
504430c
I love writing tests
rmusser01 May 24, 2025
e885df7
characters, endpoint tests still broken
rmusser01 May 24, 2025
631c387
Update test_characters_endpoint.py
rmusser01 May 24, 2025
98b7e5b
inference backend
rmusser01 May 25, 2025
163707b
local LLMs have full chat option support
rmusser01 May 25, 2025
55f5c9f
docs
rmusser01 May 25, 2025
9dc0a5b
Update llamacpp.py
rmusser01 May 25, 2025
5317880
docs
rmusser01 May 28, 2025
ab0fa1b
re-add helper scripts...
rmusser01 May 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 5 additions & 0 deletions .github/codeql/codeql-config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
paths:
- 'PoC_Version'
paths-ignore:
- '**/*.*'
- '**/PoC_Version/**'
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python application
name: Python poc-tldw

on:
push:
Expand Down Expand Up @@ -36,41 +36,42 @@ jobs:
pwd
ls
ls ./Config_Files
cd ./Tests/ChromaDB/
cd ./PoC_Version/Tests/ChromaDB/
pytest ./test_chromadb.py

- name: Test RAG lib functions with pytest
run: |
pwd
cd ./Tests/RAG
cd ./PoC_Version/Tests/RAG
pytest test_RAG_Library_2.py
pytest test_enhanced_rag_pipeline.py

- name: Test RAG Notes functions with pytest
run: |
pwd
cd ./Tests/RAG_QA_Chat
cd ./PoC_Version/Tests/RAG_QA_Chat
pytest test_notes_search.py

- name: Test SQLite lib functions with pytest
run: |
pwd
cd ./Tests/SQLite_DB
cd ./PoC_Version/Tests/SQLite_DB
pytest .

- name: Test Utils lib functions with pytest
run: |
pwd
cd ./Tests/Utils
cd ./PoC_Version/Tests/Utils
pytest test_utils.py

- name: Test Character Chat Library functions with pytest
run: |
pwd
cd ./Tests/Character_Chat
cd ./PoC_Version/Tests/Character_Chat
pytest test_character_chat.py

- name: Test tldw runs
run: |
pwd
cd ./PoC_Version
python summarize.py -h
88 changes: 88 additions & 0 deletions .github/workflows/python-tldw.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python tldw

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

permissions:
contents: read

jobs:
build-and-run-tests:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python 3.12
uses: actions/setup-python@v3
with:
python-version: "3.12"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest pytest-mock
sudo apt install portaudio19-dev python3-all-dev

if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
pip check
# - name: Test ChromaDB lib functions with pytest
# run: |
# pwd
# ls
# ls ./Config_Files
# cd ./Tests/ChromaDB/
# pytest ./test_chromadb.py
#
# - name: Test RAG lib functions with pytest
# run: |
# pwd
# cd ./Tests/RAG
# pytest test_RAG_Library_2.py
# pytest test_enhanced_rag_pipeline.py
#
# - name: Test RAG Notes functions with pytest
# run: |
# pwd
# cd ./Tests/RAG_QA_Chat
# pytest test_notes_search.py
#
# - name: Test SQLite lib functions with pytest
# run: |
# pwd
#
# pytest -s tldw_Server_API/tests/
#
# - name: Test Utils lib functions with pytest
# run: |
# pwd
# cd ./Tests/Utils
# pytest test_utils.py
#
# - name: Test Character Chat Library functions with pytest
# run: |
# pwd
# cd ./Tests/Character_Chat
# pytest test_character_chat.py
#
- name: Test Media Versioning Functions/Endpoints with pytest
run: |
pwd
cd ./tldw_Server_API/tests/Media_Ingestion_Modification/
pytest -s test_media_versions.py

- name: Test Media Processing Endpoint functions with pytest
run: |
pwd
cd ./tldw_Server_API/tests/Media_Ingestion_Modification/
pytest -s test_media_processing.py

- name: Test tldw runs
run: |
pwd
uvicorn tldw_Server_API.app.main:app
Binary file modified .gitignore
Binary file not shown.
Loading