Skip to content

Commit bd9e678

Browse files
Adds default tokenizer to old projects (#76)
* Adds default tokenizer to old projects * ref update --------- Co-authored-by: anmarhindi <anmar.hindi@kern.ai>
1 parent eeb7b36 commit bd9e678

File tree

2 files changed

+13
-1
lines changed

2 files changed

+13
-1
lines changed

upgrade_logic/business_objects/gateway.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,18 @@
1313

1414
def gateway_1_16_0() -> bool:
1515
__gateway_1_16_0_add_cognition_project_folder_defaults()
16+
__gateway_1_16_0_add_cognition_project_tokenizer_defaults()
17+
return True
18+
19+
20+
def __gateway_1_16_0_add_cognition_project_tokenizer_defaults() -> bool:
21+
query = """
22+
UPDATE cognition.project
23+
SET tokenizer = 'en_core_web_sm'
24+
WHERE tokenizer IS NULL
25+
"""
26+
general.execute(query)
27+
general.commit()
1628
return True
1729

1830

0 commit comments

Comments
 (0)