📈 Improving TerminusDB (software) efficiency

[alexander-mart]

Let's talk about comprehensive optimization of TerminusDB project

I am starting this topic and may edit it as comments are added and new facts are discovered.

Levels of optimization (by wikipedia related article) ^[1]

• Design level
• Algorithms and data structures
• Source code level
• Build level
• Compile level
• Assembly level
• Run time
• Platform dependent and independent optimizations

Directions to optimize TerminusDB project

• TerminusDB as database
  • Data query speed
  • Data write speed
  • Data disk space usage
• TerminusDB as software bundle
  • Installation disk space usage
  • RAM usage
  • CPU/GPU usage
  • Per platform-optimization
• TerminusDB dev expeirience
  • Build speed

What else?

What can we do?

• Formulate specification, construct benchmarks and publish results

Benchmarks

• https://github.com/terminusdb/terminusdb/blob/main/src/test/python_tests/benchmarks.py

Documentation pages

• Only one search result of "speed":
  • https://terminusdb.org/docs/terminusdb-explanation/#why-choose-terminus-db

Related issues:

• Decrase docker image size of terminusdb-server (764Mb now!) #19 #2192
• Decrase docker image size of this module (1.3Gb now!) terminusdb-change-request-api#19
  • Use node:18-alpine instead of node:18 to decrase docker image size terminusdb-change-request-api#21

Issues search queries

• Issues including "speed"
• Issues including "benchmark"

Related topics:

• Embedding database into applications #237
• Outcome of ingest speed improvement discussion #994
• How does percistence work? What is written to disk and when? #1865

References:

• [1] https://en.wikipedia.org/wiki/Program_optimization