Clarification on Retry Strategy

[dheshanm]

From com.aallam.openai.client.OpenAIConfig , I see that RetryStrategy in OpenAIConfig is used for rate limit retry configuration.

/**
 * Specifies the retry strategy
 *
 * @param maxRetries the maximum amount of retries to perform for a request
 * @param base retry base value
 * @param maxDelay max retry delay
 */
public class RetryStrategy(
    public val maxRetries: Int = 3,
    public val base: Double = 2.0,
    public val maxDelay: Duration = 60.seconds,
)

Here, I am not sure if I understand what each parameter does.

Here, maxRetries, refers to the number of time a same request can be retired, if rate limited. and max Delay is how long for a http connection takes for it to be considered rate-limited? And what does the 'base' value represent?

Additionally, OpenAIConfig takes in a timeout parameter as below:

/**
 * OpenAI client configuration.
 *
 * @param token OpenAI Token
 * @param logger http client logger
 * @param logLevel http client logging level
 * @param timeout http client timeout
 * @param headers extra http headers
 * @param organization OpenAI organization ID
 * @param host OpenAI host configuration
 * @param proxy HTTP proxy url
 * @param host OpenAI host configuration.
 * @param retry rate limit retry configuration
 */
public class OpenAIConfig(
    public val token: String,
    public val logLevel: LogLevel = LogLevel.Headers,
    public val logger: Logger = Logger.Simple,
    public val timeout: Timeout = Timeout(socket = 30.seconds),
    public val organization: String? = null,
    public val headers: Map<String, String> = emptyMap(),
    public val host: OpenAIHost = OpenAIHost.OpenAI,
    public val proxy: ProxyConfig? = null,
    public val retry: RetryStrategy = RetryStrategy(),
)

I'm building an OpenAI client for Android, and I want the end users to configure their timeout and retry strategy, as I find 30s default to cause a lot of timeouts for the users. So I wanted to allow users to increase the timeout, will it conflict with the retry strategy?

If I have a retry timeout of 120s and a default RetryStrategy with 60s maxDelay, Will it cause long http connection to me considered rate-limited and attempt to retry? I would like a little clarification on this matter.

Sorry for the long post.

[aallam]

Hello, the retry delay is equal to the base raised to the power of the retry count. It has a maximum value of maxDelay and is limited by the maxRetries parameter.

Indeed, there is a distinction between the delay between retries and the response timeout. The delay between retries refers to the time interval after a failed call before attempting another one, while the response timeout is the maximum duration allowed for waiting to receive a response.