http requests have no default timeout

[rodionlim]

@olimorris I noticed this very weird thing with the copilot adaptor where there are periods where server can take a long time (>600s) to respond to a request. The server does not have a timeout, and the request is held for the entire duration, causing the chat buffer to be in an unmodifiable state for the entire duration. From a user experience perspective, the chat just appears to be frozen.

It seems like there is a http adaptor with the following defaults:

local raw = {
    "--retry",
    "3",
    "--retry-delay",
    "1",
    "--keepalive-time",
    "60",
    "--connect-timeout",
    "10",
  }

I was wondering whether it would make sense to set a default --max-time to short circuit the request beyond a certain sensible default time period. In this instance, neither connect timeout nor keepalives were an issue. Not surprisingly, gD didn't show any issues and I closed the neovim session altogether, only managed to figure out what went wrong because I had MITM opened.

MITM logs

23:09:40 HTTPS POST …individual.githubcopilot.com /chat/completions 200  application/json 15.6k 27.8s
>>23:10:28 HTTPS POST …individual.githubcopilot.com /chat/completions 504 text/html 53.6k  612s

[olimorris]

Pretty sure plenary has a high default timeout

[rodionlim]

I checked plenary's source, it does not pass in any default values. If the server does not terminate, then chat buffer remains read-only till server returns. We can extend the current codecompanion config to pass in a max time parameter via adapters.raw.

From the curl documentation which plenary uses, we can pass in this parameter multiple times and only the last one will be used. Would it make sense to set a default timeout that aligns with the keep-alive timeout of 60 seconds, and allow the user to overwrite it if they require a longer timeout?

adapters = {
      copilot = function()
        return require("codecompanion.adapters").extend("copilot", {
          raw = {
            "--max-time",
            "60",
          },
        })
      end,
    },