Adjusted SCID definition to the technically correct definition and ma…

.github/workflows/deploy-to-gh-pages.yml

@@ -13,6 +13,9 @@ on:
   ## C: Triggered on Wiki changes:
   # gollum
 
+  ## D: Triggered by a workflow call:
+  workflow_call:
+
 jobs:
 
   ##############################

.github/workflows/import-external-glossaries.yml

@@ -13,6 +13,9 @@ on:
   ## C: Triggered on Wiki changes:
   # gollum
 
+  ## D: Triggered by a workflow call:
+  workflow_call:
+
 jobs:
   deploy:
     name: Import external glossaries

.github/workflows/import-metadata-google-sheet.yml

@@ -2,7 +2,7 @@
 
 name: Import metadata from Google Sheets
 on:
-  # A: Triggered on push to the main branch:
+  ## A: Triggered on push to the main branch:
   # push:
   #   branches:
   #     - main
@@ -13,6 +13,9 @@ on:
   ## C: Triggered on Wiki changes:
   # gollum
 
+  ## D: Triggered by a workflow call:
+  workflow_call:
+
 jobs:
   deploy:
     name: Import metadata from Google Sheets

.github/workflows/import-wiki.yml

@@ -13,6 +13,9 @@ on:
   ## C: Triggered on Wiki changes:
   # gollum
 
+  ## D: Triggered by a workflow call:
+  workflow_call:
+
 jobs:
 
   ##############################

.github/workflows/main.yml

@@ -0,0 +1,40 @@
+name: Import all and Deploy to GitHub Pages
+
+on:
+  # A: Triggered on push to the main branch:
+  # push:
+  #   branches:
+  #     - main
+
+  # B: Triggered on manual workflow dispatch:
+  workflow_dispatch:
+
+  ## C: Triggered on Wiki changes:
+  # gollum
+jobs:
+  call-action1:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Import Wiki
+        uses: ./.github/workflows/import-wiki.yml
+
+  call-action2:
+    needs: call-action1
+    runs-on: ubuntu-latest
+    steps:
+      - name: Import metadata from Google Sheet
+        uses: ./.github/workflows/import-metadata-google-sheet.yml
+
+  call-action3:
+    needs: call-action2
+    runs-on: ubuntu-latest
+    steps:
+      - name: Import external glossaries
+        uses: ./.github/workflows/import-external-glossaries.yml
+
+  call-action4:
+    needs: call-action3
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy to GitHub Pages
+        uses: ./.github/workflows/deploy-to-gh-pages.yml

.gitignore

@@ -20,11 +20,11 @@ yarn-debug.log*
 yarn-error.log*
 
 # logs
-search-index-typesense/search-index-entries/*
-search-index-typesense/logs/*
-search-index-typesense/sitemaps/*
-search-index-typesense/overrides/overridesID.json
-search-index-typesense/singleUrlsFromWotTermsGoogleSheet/*
+_search-index-typesense/search-index-entries/*
+_search-index-typesense/logs/*
+_search-index-typesense/sitemaps/*
+_search-index-typesense/overrides/overridesID.json
+_search-index-typesense/singleUrlsFromWotTermsGoogleSheet/*
 .history
 # media
 # *.mp4

README.md

@@ -1,33 +1,26 @@
 # What 
 ## WOT-terms 
-It's documentation and functionality to explore this documentation. We aim to document everything related to **KERI, ACDC and CESR** and alike (umbrella term is *KERI Suite*). 
+WOT is "WebofTrust". It's the home of all KERI-related open-source code. The umbrella term is *KERI Suite*. It covers KERI core, ACDC, CESR, OOBI, and all other parts.
 
 ### What is this?
-In [How we did]() we document 'the making of' this site that consists of 
-- data; which we call WOT-terminology or **WOT-terms**
-- functionality; which we call **KERISSE**
+This repo manages the content and functionality of all concepts and terminology behind the KERI Suite. It offers a glossary with definitions and links to the terminology source management tools. It also feeds the dictionary tool Kerific.
+
+## Documentation subsystems and their technologies
+- **This repo handles concepts and terminology** resulting in a glossary run as a Docusaurus site (GitHub pages)
+- The [KERI Suite documentation](https://github.com/WebOfTrust/keridoc/), which is also a Docusaurus site (GitHub pages)
+- The [KERI Suite Search Engine](https://github.com/WebOfTrust/kerisse/), which is Typesense-based GitHub pages; custom-made Bootstrap.
+- The [Kerific SSI dictionary](https://github.com/WebOfTrust/kerific): Self-Sovereign Identity dictionary *including* KERI terms (browser extension for Brave/Edge/Chrome).
 
 ## WOT-terms
 Is the **data** part of this site and consists of:
-- whitepapers
-- code of the KERI Suite
 - Glossary
-- Q-and-A's
 - Concepts behind the KERI Suite
-- blogs
-- etc
-
-### KERISSE
-Is the **functionality** part of this site and consists of:
-- Typesense [Search Engine](https://weboftrust.github.io/WOT-terms/?searchModalStatus=open)
-- Artificial Intelligence chatBot (in lower right corner)
-- [Docusaurus static site](https://weboftrust.github.io/WOT-terms/) functionality
 
 ### Why
-To let autonomic identifiers florish.
+Clear criteria for term definitions around KERI are needed.
 
 ### For who
-WOT-terms is pimarilly aimed at KERI Suite developers (to be)
+The WOT-terms repo is primarily aimed at people trying to understand the inner workings of the KERI Suite and how the suite relates to other identifier systems in the SSI field.
 
 ### Policy
 
@@ -39,7 +32,7 @@ We have technical meetings twice a week on Monday and Thursdays.
 
 The Edu Meetings held every other Thursday (First meeting was on 2022 July 28), stopped April 2023.
 
-The meeting agenda of the Technical Meeting may be found [here](https://hackmd.io/2C8ch1meS6ad3F3g2aEewg?both). The zoom link is also at the top of the page.
+The meeting agenda of the Technical Meeting may be found [here](https://hackmd.io/2C8ch1meS6ad3F3g2aEewg). The Zoom link is also at the top of that page.
 
 ## Related Meetings
 
@@ -53,14 +46,9 @@ There also is the [Concepts and terminology](https://wiki.trustoverip.org/pages/
 
 ## Install this project
 
-This repo is a documentation website with a Typesense search engine and a scraper engine that collects documentation.
+This repo is a GitHub pages website based on Docusaurus that manages terminology through a source management tool, which currently is the [wiki of WOT-terms](https://github.com/WebOfTrust/WOT-terms/wiki).
 
-This website is built in Docusaurus ( https://docusaurus.io ). The Typesense search engine is integrated in Docusaurus. Also the scraper engine code is inside the Docusaurus code.
-
-This repo has three parts:
-* **Docusaurus documentation** site
-* Typesense **search engine**
-* Chatbot (under construction)
+This website is built in Docusaurus ( https://docusaurus.io ).
 
 ### Run website
 
@@ -75,4 +63,15 @@ What should you do to install this project:
 * Now you can run a local version of the **Docusaurus website**:
 `$ npx docusaurus start`. The search engine also works now.
 
-[How we did](https://weboftrust.github.io/WOT-terms/docs/category/how-we-did) describes the follow-up steps.
+{ Kor, change link pls [How we did](https://weboftrust.github.io/WOT-terms/docs/category/how-we-did) } describes the follow-up steps.
+
+## Origin
+
+Mid 2024 we've split up old WOT-terms repo into:
+
+- [kerisse](https://github.com/WebOfTrust/kerisse) : search engine (Typesense-based [github.io site](https://weboftrust.github.io/kerisse/))
+- new [WOT-terms](https://github.com/WebOfTrust/WOT-terms): glossary, solely focussed on terminology, filtering, and connect to source management (Docusaurus-based [github.io site](https://weboftrust.github.io/WOT-terms/?level=2))
+- [keridoc:](https://github.com/WebOfTrust/keridoc) KERI documentation site (Docusaurus-based: [github.io site](https://weboftrust.github.io/keridoc/?level=2))
+
+Apart from this, we still have:
+- [kerific](https://github.com/WebOfTrust/kerific): SSI-terminology dictionary including KERI terms (browser extension for Brave/Edge/Chrome).

search-index-typesense/config/config-search-index-entries-manual/README.md → _search-index-typesense/config/config-search-index-entries-manual/README.md

@@ -1,3 +1,7 @@
-# Instructions
+# Information
 
-These files can be in .json or .jsonl format. They are directly copied into the directory that holds files that are imported into Typesense.
+These files can be in .json or .jsonl format. They are directly copied into the directory that holds files that are imported into Typesense.
+
+You are free to name the files.
+
+See handmade.json` as an example.

_search-index-typesense/config/config-sitemaps-exlude-urls/README.md

@@ -0,0 +1,14 @@
+# Information
+
+Create an array with strings that should be ignored.
+
+Writing `newsroom` results in every URL that contains the string `newsroom` being ignored.
+
+```
+[
+    "gleif.org/en/meta",
+    "gleif.org/en/contact",
+    "gleif.org/en/newsroom",
+    "gleif.org/en/lei-data/email-notifications-on-technical-updates"
+]
+```

_search-index-typesense/config/config-sitemaps-manual/README.md

@@ -0,0 +1,7 @@
+# Information
+
+In this directory you can place handcrafted sitemaps. They will be copied to the sitemaps working dir and included as a source for the scraper bots.
+
+You are free to name the files.
+
+See slack-keri-archive.xml` as an example: this is a one time sitemap since this archive will not change anymmore.

search-index-typesense/config/config_sitemaps_create.sh → _search-index-typesense/config/config_sitemaps_create.sh

@@ -11,9 +11,10 @@ source .env
 
 
 ########################################
-# General websites - createSitemap.mjs
+# General websites
 ########################################
 
+# `sitemap-generator-cli` should be installed globally via `npm install -g sitemap-generator-cli` to run the `sitemap-generator` command.
 sitemap-generator https://keria.readthedocs.io --verbose -f ./search-index-typesense/sitemaps/sitemap-keria.readthedocs.io.xml
 
 sitemap-generator https://keripy.readthedocs.io --verbose -f ./search-index-typesense/sitemaps/sitemap-keripy.readthedocs.io.xml

_search-index-typesense/isLibraryInstalled.mjs

@@ -0,0 +1,83 @@
+/**
+ * @file This file checks if the required libraries are installed.
+ * If not, it logs an error message to the console and to a log file.
+ * The libraries that are checked are ImageMagick, jq, and curl.
+ * This script uses the child_process module to execute shell commands and the util module to promisify the exec function.
+ * The logger module is used to log messages to the console and to a log file.
+ * @author Kor Dwarshuis
+ * @version 1.0.0
+ * @since 2024-05-21
+ */
+
+import { exec } from 'child_process';
+import util from 'util';
+import logger from './modules/logger.mjs';
+
+const execPromisified = util.promisify(exec);
+
+async function isLibraryInstalled(libraryName, command = libraryName, expectedOutput = libraryName) {
+  console.log("Testing if " + libraryName + " is installed");
+  try {
+    const { stdout } = await execPromisified(`${command} --version`);
+    return stdout.includes(expectedOutput);
+  } catch (error) {
+    return false;
+  }
+}
+
+async function isNpmPackageInstalled(packageName) {
+  try {
+    const { stdout } = await execPromisified(`npm list -g ${packageName}`);
+    return !stdout.includes('empty');
+  } catch (error) {
+    return false;
+  }
+}
+
+const messages = {
+  error: ' is not installed',
+  success: ' is installed'
+};
+
+// Usage:
+if (!await isLibraryInstalled('ImageMagick', 'convert', 'ImageMagick')) {
+  console.log('ImageMagick' + messages.error);
+  logger.setLogFile('error.log');
+  logger.log('ImageMagick' + messages.error);
+
+} else {
+  console.log('ImageMagick' + messages.success);
+  logger.setLogFile('success.log');
+  logger.log('ImageMagick' + messages.success);
+}
+
+if (!await isLibraryInstalled('jq')) {
+  console.log('jq' + messages.error);
+  logger.setLogFile('error.log');
+  logger.log('jq' + messages.error);
+
+} else {
+  console.log('jq' + messages.success);
+  logger.setLogFile('success.log');
+  logger.log('jq' + messages.success);
+}
+
+if (!await isLibraryInstalled('curl')) {
+  console.log('curl' + messages.error);
+  logger.setLogFile('error.log');
+  logger.log('curl' + messages.error);
+} else {
+  console.log('curl' + messages.success);
+  logger.setLogFile('success.log');
+  logger.log('curl' + messages.success);
+}
+
+if (!await isNpmPackageInstalled('sitemap-generator-cli')) {
+  console.log('sitemap-generator-cli' + messages.error);
+  logger.setLogFile('error.log');
+  logger.log('sitemap-generator-cli' + messages.error);
+} else {
+  console.log('sitemap-generator-cli' + messages.success);
+  logger.setLogFile('success.log');
+  logger.log('sitemap-generator-cli' + messages.success);
+}