feature: deferred loading and requirement pruning

[leondz]

Support construction-time loading of optional modules. Includes

• many generators now using this pattern
• pattern for loading modules at run-time and failing if absent
• optional requirements moved to pyproject.toml options and pruned from requirements.txt
• _load_deps and _clear_deps pattern, used in generator constructor and _load_client / _clear_client
• tests to check that optional deps are in the right place and not in the wrong place

Todo / in scope:

• skipping tests requiring absent optional imports
• take a stance on whether to skip or stop on missing module, per plugin type (likely skip for probe, stop for main generator, beyond that tba)
• Consider moving load/clear dep function defs up and out for all classes (where - garak._plugins? How - assign func member? create mixin?)
• validation: exception throws correctly / doesn't throw if dependency present
  • cohere
  • langchain
  • litellm
  • mistral
  • nemoguardrails
  • nemollm
  • octo
  • ollama
  • optimum (huggingface.OptimumPipeline) - exception fires, pkg install seems borked
  • replicate

Not done:

• load/clear deps for plugin types other than generators
• gh actions for testing optional components
• deeper validation

Out of scope:

• handling of versioning outside of pyproject.toml

Resolves #101

[jmartin-tech]

Consider adding to allows for pip install garak[plugin_all]:

plugin_all = garak[
    plugin_cohere,
    plugin_langchain,
    plugin_llava,
    plugin_litellm,
    plugin_mistralai,
    plugin_nemoguardrails,
    plugin_nemollm,
    plugin_octoai,
    plugin_ollama,
    plugin_replicate,
]

There might be a better name to use for this group.

[jmartin-tech]

Should we keep requirements.txt as a consolidated lists of all possible requirements and require usage of pyproject.toml to get a limited dependencies install?

Another option to me would be simply remove requirements.txt and require all installs be based on using pyproject.toml until we implement a lock file.

[leondz]

Right. After migrating the gh workflows to use pyproject.toml, requirements.txt becomes superfluous - I'm leaning toward the latter.

Removal does mean we lose a cross check in test_reqs, but that's OK. Could do with a test to ensure all packages (including optional) specity a version.