Redesign for webtools — move to vite/react/ts (#25)

* Drop flask front-end

* First batch of vite/react/ts front-end with flask server

* Updates for docs and readability

* Cleanup

* Update assets + readme

* Refactoring webtools + integrating flask-socketio

* Bump readme

* Bump

* dev reqs & concurrent pids bash in npm

* Change handling of edit mode from modal to inline

* cleanup

* Labeller help hover

* Missed doc notes

* Train model tab updates

- Cleaned up flask code
- Removed train tab interrupt button (limitations)
- Added package requirements precheck on app startup
- Added time elapsed on training screen

* Accidental none check against thread, preventing re-runs

* Asset screenshot

* Move time tracker on trainer to zustrand state

"Time elapsed" tracking needs to live within managed state, and not ui state, due to the training being asynchronous and user can use the app while the model is training in the background

* Bump

* Add seed input to training

* Add seed input to training (server)

* Sets up cache reset on model loader

Addresses one of the comments by @strangetom on #25 (comment) regarding cache resetting by the @ lru_cache decorator on the model loading. This ensures completed training cycles done via the web app (which impacts the parser) updates the parser optimistically to handle the cache resetting

* Bump up third-party package versions against vite and mantine

* WIP - add gridsearch options, revert to logger.info, and refine console output from websockets

1. Work-in-progress new feature for gridsearch
2. Rever to logger.info for websocket output, to be aligned with develop branch
4. Refine the console output, should have originally used <pre> tags to persist  reserved str \n, \t, etc

Note: This commit refers to a function set_redirect_log_stream, that will be introduced in #35, since it is required to set a logging stream handle correctly to pipe results to websockets

* Screenshots

* Remove remnants

* Cleanup, readability, etc

* Remove interrupt and keepModels based on maintainer feedback

* Pre-commit format fixes, part I

* Pre-commit fixes, part II (adds biomejs for webtools linter/formatter/etc)

Tacks on to the existing pre-commit hooks in the repo with BiomeJS. BiomeJS is specifically for the web side (typescript/js/css). Local configuration modeled after preconfigs at https://github.com/biomejs/pre-commit. Commit includes all file format fixes necessary. Anticipated that config will be modified as necessary if new web contributors participate in future commits.

* Restore training.sqlite3 to previous commit

Accidentally included new training.sqlite3 in e01430b

* Address bug feedback, round I

- [x] Parser: The amount flags are not shown in the results table
- [x] Parser: order labels in token tooltip as per old webapp
- [x] Parser: Reorder rows in result table as per old webapp
- [x] General: Can we avoid use of google fonts? Everything else (after running npm install) is entirely local.
- [x] Parser: show % sign in tooltips for scores for each label
- [x] Trainer: Output include debug info app.sockets, which isn't relevant to training.
- [x] Trainer: html and tsv files saved in webtools directory instead of parent
- [x] General: on the parser and labeller pages, the contrast between text and colours is a bit low (particularly for anything on a red background). This is probably best fixed by increasing the font size on those pages, since it's a little small anyway.
- [x] Trainer: default model location incorrect (this was changed recently, is should be in ingredient_parser/en/data/)
- [x] Trainer: split value does not allow more than one decimal place. There shouldn't really be any limit of the number of decimal places (but in practice, 3 might be a more reasonable limit than 1).
- [x] Trainer: saveur dataset is selected by default, but doesn't exist.
- [x] Parser: Missing separate_names option

* Address bugs feedback, round II

- [x] Trainer: Inputs for seed, split do not focus on click so cannot edit without tabbing to the inputs or finding another way to focus them. This might be a wider problem than just those.
- [x] Trainer: unable to start another training run after completing a previous one if confusion matrix is generated by the first run. There's a RuntimeError related to Tk.

* Address bugs feedback, round III

- [x] Labeller: Unable to search unicode fractions
- [x] Labeller: Unable to quickly select a label for a token using first letter (as per <select> element)
- [x] Labeller: Bug when searching (coarse, dried returns no results but should return sentence id 4718)

* Address bugs feedback, round IV

- [x] Trainer: The split value seems to be capped at 0.9. That should probably changed 0.999
- [x] Labeller: There's a weird bug when changing the label for a token - an addition PUNC token gets inserted. This seems to effect any sentence containing a comma when you change the label of the first token.

* Address bugs feedback, round V

- [x] Labeller: Is it possible to be able to tab between tokens in a sentence to change to the next one?

* Address bugs feedback, round VI

- [x] Trainer: Gridsearch incomplete

---------

Co-authored-by: tom <tpstrange@gmail.com>

Author: mcioffi

.pre-commit-config.yaml

@@ -5,26 +5,52 @@ repos:
     rev: v5.0.0
     hooks:
       - id: check-added-large-files
+        name: (base:repo) debug-statements
         args: ['--maxkb=5000']
         stages: [pre-commit]
       - id: check-ast
+        name: (base:repo) check-ast
         stages: [pre-commit]
       - id: check-toml
+        name: (base:repo) check-toml
         stages: [pre-commit]
       - id: check-yaml
+        name: (base:repo) check-yaml
         stages: [pre-commit]
       - id: debug-statements
+        name: (base:repo) debug-statements
         stages: [pre-commit]
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.11.6
     hooks:
       - id: ruff
+        name: (base:repo) ruff
         args: ["--fix"]
         stages: [pre-commit]
       - id: ruff-format
+        name: (base:repo) ruff-format
         stages: [pre-commit]
   - repo: https://github.com/sphinx-contrib/sphinx-lint
     rev: v1.0.0
     hooks:
       - id: sphinx-lint
+        name: (docs:repo) sphinx-lint
         stages: [pre-commit]
+  - repo: local
+    hooks:
+      - id: biome-check-ts-js
+        name: (webtools:repo) biome check typescript/javascript
+        entry: npx @biomejs/biome check --write ./webtools/src --config-path=./webtools/biome.json --files-ignore-unknown=true --no-errors-on-unmatched
+        language: node
+        types: [text]
+        files: "\\.(jsx?|tsx?|c(js|ts)|m(js|ts)|d\\.(ts|cts|mts)|jsonc?)$"
+        exclude: ^docs/
+        require_serial: true
+      - id: biome-check-css
+        name: (webtools:repo) biome check css
+        entry: npx @biomejs/biome check --write ./webtools/src --config-path=./webtools/biome.json --files-ignore-unknown=true --no-errors-on-unmatched
+        language: node
+        types: [text]
+        files: "\\.(css?)$"
+        exclude: ^docs/
+        require_serial: true

README.md

@@ -59,7 +59,9 @@ Word-level results:
 
 ## Development
 
-The development dependencies are in the ```requirements-dev.txt``` file. Details on the training process can be found in the [Model Guide](https://ingredient-parser.readthedocs.io/en/latest/guide/index.html) documentation.
+**Basic**
+
+The development dependencies are in the `requirements-dev.txt` file. Details on the training process can be found in the [Model Guide](https://ingredient-parser.readthedocs.io/en/latest/guide/index.html) documentation.
 
 Before committing anything, install [pre-commit](https://pre-commit.com/) and run
 ```
@@ -68,16 +70,34 @@ pre-commit install
 
 to install the pre-commit hooks.
 
-Please target the **develop** branch for pull requests. The main branch is used for stable releases and hotfixes only.
+**Web app**
 
-There is a simple web app for testing the parser with ingredient sentences and showing the parsed output. To run the web app, run the command
+There is a simple web app for testing the parser, browsing the database, labelling entries, and running the basic model.
+
+To run the web app, follow the prior steps on `requirements-dev.txt`, and separately install [Node](https://nodejs.org/en/download), the JS runtime. Once globally installed on your machine, download your packages in the webtools directory with:
+
+```bash
+$ npm install
+```
+
+After package installs, run dev to start.
 
 ```bash
-$ flask --app webapp run
+$ npm run dev
 ```
 
-![Screen shot of web app](docs/source/_static/app-screenshot.png)
+*tl;dr — inside package.json, `flask` runs the base server, `sockets` runs the web sockets server, and `watch` runs vite build for the react/typescripe bundle — separate flask instances are required to accommodate better web socket behavior*
+
+![Screen shot of web parser](docs/source/_static/app.parser.screenshot.png)
+![Screen shot of web labeller](docs/source/_static/app.labeller.screenshot.png)
+![Screen shot of web trainer](docs/source/_static/app.trainer.screenshot.png)
+
+**Documentation**
+
+The dependencies for building the documentation are in the `requirements-doc.txt` file.
 
-This requires the development dependencies to be installed.
+**Contribution**
 
-The dependencies for building the documentation are in the ```requirements-doc.txt``` file.
+Please target the **develop** branch for pull requests. The main branch is used for stable releases and hotfixes only.
+
+There is a simple web app for testing the parser with ingredient sentences and showing the parsed output. To run the web app, run the command

docs/source/_static/app-screenshot.png renamed to docs/source/_static/app.archived.screenshot.png

@@ -128,16 +128,15 @@ Depending on the language, you may need a language specific :func:`tokenize <ing
 
 .. tip::
 
-    A web app has been developed to aid in the adding and labelling of training sentences. Run the command
+    A web app has been developed to aid in the adding and labelling of training sentences. If intending to add another language, you will need to adjust some of the web app code. Regardless, you can run these commands under the webtools directory to install the packages and run the web app.
 
     .. code:: bash
 
-        $ flask --app labeller run
+        $ npm install
+        $ npm run dev
 
     Then navigate to http://localhost:5000 in your browser.
 
-    You may need to tweak this to work with the correct database.
-
 1. Normalise
 ~~~~~~~~~~~~