-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Milestone
Description
This would extend current philo ids to 10 32 bit integers instead of 9. This shouldn't be difficult to implement, but requires many small changes across the Python code base. Too risky for 5.0. Delay to 5.1
Advantages:
- easier highlighting
- Tokens stored in the index can be multi-word tokens (e.g named-entities)
Downsides:
- 4 bytes of extra disk space per token
- Tiny impact on disk reads (probably not measurable on SSD/NVME drives)
- Potential for bugs if we miss sections of the code that assume 9 32 bit ints for philo ids.
Metadata
Metadata
Assignees
Labels
No labels