Faster and more efficient ways to switch backends/directives?

[karthink]

From @LazerJesus (#177)

---

with this gptel becomes even more important to my daily work. i love it!
i got a couple of questions, not sure if here is the best spot, so let me know if i should start a new issue about any of these.

managing multiple backends

at the moment i toggle models by keybinding custom expressions. like this

  (defun set-gpt4-universal ()
    (interactive)
    (setq gptel-model "gpt-4-1106-preview")
    (setq chatgpt-shell-chatgpt-model-version "gpt-4-1106-preview"))

  (defun gptel-with-argument ()
    (interactive)
    (setq current-prefix-arg '(4))
    (call-interactively #'gptel))

  (general-menu "ai" "a"
    "r" '(gptel-send :wk "gptel region")
    "R" '(gptel-send-with-argument :wk "gptel region prefixed")
    "a" '(gptel-abort :wk "abort")
    "C-<return>" '(gptel-send-with-argument  :wk "gptel send prefixed")
    "3" '(set-gpt3-universal  :wk "gpt 3.5 turbo")
    "4" '(set-gpt4-universal  :wk "gpt 4"))

but with multiple backends that is becoming tedious. is there a better way?

whats the fastest way to provide one-off directives

i often highlight text regions and want to write just a few words as directive.
using the gpt-system buffer (reached via h h from the gptel minibuffer) is about 8 key hits for me.
open minibuffer, h h, closing minibuffer, clearing text, c-c c-c.
is there a faster way?

managing directives / directives per model?

different models have different strengths and weaknesses, therefore i use them for different reasons.
for example, i use gpt4 for coding and mixtral for summarizing large bodies of text.
that requires different directives. is it possible to have directives per model?
how would i do that?

[karthink]

There are many ways to speed up switching between models/backends/directives -- gptel's defaults try to be simple and discoverable, not fast.

Ranked easy to involved, you could try:

Keyboard macros

using the gpt-system buffer (reached via h h from the gptel minibuffer) is about 8 key hits for me: open minibuffer, h h, closing minibuffer, clearing text, c-c c-c.

Most sequences of keys can be automated easily. Assuming your leader key is SPC, you could do

(keymap-global-set "<f6>" "SPC u SPC a r h h d")

to jump to the directive editor. SPC u is the prefix arg, SPC a r your keybinding for gptel-send, h h to open the editor, and d (assuming evil-mode) to clear the pre-selected region.

You can set up keyboard macros for other things too, like switching to gpt-4:

(keymap-global-set "SPC a 4" "SPC u SPC a r - m c h a <return> 4 <return>")

Call gptel functions directly

The transient menu is available at M-x gptel-menu, you can bind to it directly. The directives menu can be accessed directly at M-x gptel-system-prompt. You can also include these in the above keyboard macros as "<<gptel-menu>>", etc.

Helper functions

but with multiple backends that is becoming tedious. is there a better way?

How is what you're doing (to switch between gpt-4 and gpt-3.5) tedious? It's just one keybinding?

To be safe, you should be setting the backend along with the model.

for example, i use gpt4 for coding and mixtral for summarizing large bodies of text. that requires different directives. is it possible to have directives per model? how would i do that?

If you know these combinations ahead of time, you can do something like this:

(defvar my/gptel-presets
  ;; PRESET NAME         MODEL           BACKEND NAME   DIRECTIVE      
  '(("gpt4+coding"       "gpt-4"         "ChatGPT"      "My coding directive for gpt-4")
    ("mixtral+summarize" "Mixtral..."    "anyscale"     "My directive for summarizing with Mixtral")
    ("gpt-3.5+general"   "gpt-3.5-turbo" "ChatGPT"      "My directive for gpt-3.5"))
  "List of preset combinations for gptel.")

(defun my/gptel-switch-preset (preset)
  (interactive (list (completing-read
                      "Select preset: "
                      (mapcar #'car my/gptel-presets)
                      nil t)))
  (let ((combination (assoc preset my/gptel-presets)))
    (setq gptel-model (nth 1 combination)
          gptel-backend (cdr (assoc (nth 2 combination) gptel--known-backends))
          gptel--system-message (nth 3 combination))
    (message "Selected gptel preset: %s" preset)))

NOTE: This can break in the future as it accesses internal gptel variables (gptel--system-message, gptel--known-backends).

Custom gptel queries with gptel-request

whats the fastest way to provide one-off directives

For custom workflows, gptel provides gptel-request. See the wiki for examples.

Here's a function that asks you for a directive, sends the current region (or text before point if there's no selected region) and inserts the response below the region:

(defvar my/gptel-directive-history nil
  "Store history of chosen gptel directives.")

;; Optional: persist history across Emacs sessions
(add-to-list 'savehist-additional-variables 'my/gptel-directive-history)

(defun my/gptel-send-with-directive (directive)
  (interactive (list (completing-read
                      "Directive for region: "
                      (mapcar #'cdr gptel-directives)
                      nil nil nil my/gptel-directive-history)))
  (gptel-request nil :system directive))

You can tweak it to do what you want, see the documentation of gptel-request.

[nineluj]

I'm curious to see if anyone has created a way of switching out the default prompt on a mode-by-mode basis. For example, I have a prompt to write me commands while I'm in eshell-mode.

I created a new directive using this code:

(defun nlj/get-relative-file-path (filename)
  "Get the full path of FILENAME relative to the current file.
FILENAME is the name of the file to look up.
Returns the absolute path of the file relative to the directory of
either the current loaded file or current buffer."
  (expand-file-name filename
                    (file-name-directory (or load-file-name buffer-file-name))))

(defun nlj/llm-get-directives ()
  "Reads all the directive files in the directives directory, and creates an alist of
   the form (DIRECTIVE . CONTENT). Each directive name is the file name without the extension."
  (let ((dir (nlj/get-relative-file-path "directives")))  ; this just expands the relative path to absolute path
    (mapcar (lambda (file)
              `(,(intern (file-name-base file))
                .
                ,(f-read-text file)))
            (f-files dir
                     (lambda (file)
                       (string= "txt" (f-ext file)))))))

(use-package! gptel
  :config
  ;; <redacted>
  (dolist (entry (nlj/llm-get-directives))
    (add-to-list 'gptel-directives entry t)))

and then to use it, I call C-u M-x gptel-send, hit s to change the system message and then the corresponding key to switch to my shell directive. But this adds a decent amount of additional work, especially considering that this is the only directive that I ever use in eshell-mode. And I'm also only ever using this directive in eshell mode as well.

The approaches that I can currently think of are:

1. Adding a function to some hook that runs when the eshell-mode buffer is created, which overrides gptel-directives with a buffer local value so that the eshell directive is the only directive available.
2. Creating a new function llm-shell-complete that uses gptel-request with the desired directive (based on wiki/Defining-custom-gptel-commands and the example posted above). Call this instead of C-u M-x gptel-send. But then I lose access to the convenient transient menu, so I won't be able to easily add things to the context unless I re-implement that.

I think that (1) might be a better option. More generally it would be nice to have a major mode to directive map, but this might introduce too much maintenance overhead on the user side.

Let me know your thoughts!

[karthink]

1. Adding a function to some hook that runs when the eshell-mode buffer is
   created, which overrides gptel-directives with a buffer local value so that
   the eshell directive is the only directive available.

You can set gptel--system-message in the hook directly and not bother with gptel-directives.

With presets (#542) all of this should be more convenient, although still not automatic.

[nineluj]

Cool, thanks for the tip!

[karthink]

With the addition of presets, there is now a built-in way to switch to another model, backend, system message, set of tools (etc) all at once.