Clarification of flush()

[tim-janik]

Clarification of ->flush()

I have some questiongs regarding the specification of flush(),
and would like to see some clarification of the spec or at least improve
my understanding of it. The relevant bits from the spec are:

# "plugin.h"
// Call start processing before processing.
// [audio-thread & active_state & !processing_state]
bool (*start_processing)(const struct clap_plugin *plugin);
// Call stop processing before sending the plugin to sleep.
// [audio-thread & active_state & processing_state]
void (*stop_processing)(const struct clap_plugin *plugin);

# "params.h"
// Flushes a set of parameter changes.
// This method must not be called concurrently to clap_plugin->process().
// This method must not be used if the plugin is processing.
//
// [active && !processing : audio-thread]
// [!active : main-thread]
void (*flush)(const clap_plugin_t        *plugin,
              const clap_input_events_t  *in,
              const clap_output_events_t *out);
[...];
// Request the host to call clap_plugin_params->fush().
// This is useful if the plugin has parameters value changes to report to the host but the plugin
// is not processing.
//
// eg. the plugin has a USB socket to some hardware controllers and receives a parameter change
// while it is not processing.
//
// This must not be called on the [audio-thread].
//
// [thread-safe]
void (*request_flush)(const clap_host_t *host);

The questions:

A) Is active_state from plugin.h and active from params.h meant to be the same thing?
I suppose the latter should be renamed to active_state then?

B) Is processing_state from plugin.h and processing from params.h meant to be the same thing,
or does !processing here mean "not be called concurrently to clap_plugin->process()" ?

C) Shouldn't there be a way to flush parameters from the main thread, after plugin->activate()
and before the audio-thread calls plugin->start_processing()? E.g. to load parameters from saved project state
before the plugin starts its first processing?
Or what is the recommended way/time to load initial plugin state?

D) Once plugin->start_processing() has been called and the plugin is in active_state & processing_state,
how should parameter updates be carried out?

1. Via flush() (in audio-thread) inbetween process() calls (i.e. !processing means
   "not be called concurrently to clap_plugin->process()")?
2. Or via process() only (i.e. !processing means "between start_processing and stop_processing")?
   If this is the case, what is the purpose of request_flush() calls from the plugin at this point?

E) AFAICS for request_flush, "This must not be called on the [audio-thread]" and "thread-safe]" is
contradicting. May a plugin call request_flush only on the main-thread?
Then it should say [main-thread].
Or may a plugin call request_flush from any thread?
Then "This must not be called on the [audio-thread]" should be removed.

I'd particularly like to see input from @abique on these questions, sorry if they sound a bit redundant.

[nakst]

I'm also having trouble working this all out. Here a copy of my post in Surge's Discord,

How exactly are you meant to report parameter changes from your plugin's UI to the host?
My current understand is this:
[A] If the plugin is processing, when the host calls your plugin's clap_plugin_t.process (on the audio thread), you should respond by doing clap_process_t.process->out_events->try_push for each of the parameters that have changed.
[B] If the plugin is not processing, on the main thread the plugin should call clap_host_params_t.request_flush and then on an arbitrary thread the host will (later) call clap_plugin_params_t.flush.
Here's my concern:
The processing state of the plugin is reported to the plugin by the host in calls to clap_plugin_t.start_processing and clap_plugin_t.stop_processing on the audio thread. Thus there is then a race condition to which of [A] or [B] should be done.

[robbert-vdh]

You should just always call clap_host_params::request_flush(). If the plugin is currently processing audio then that is a no-op.

So to communicate a parameter change to the host, you just

1. Push it to an MPMC queue (SPSC would also be fine as long as the API invariants are upheld)
2. Call clap_host_params::request_flush()
3. Write all events from that queue to the host's output event queue at either the end of the next process function or in clap_plugin_params::flush(), whichever the host ends up calling.

[robbert-vdh]

a) Yes.
b) Yes.
c) I'd load the state before activating the plugin. But perhaps [active && !processing : audio-thread] should be changed to [active && !processing : thread-safe]? There's no reason why that would cause any problems.
d) Via processing() only. clap_plugin_params::flush() is essentially just a shortcut to calling process() without audio/for 0 frames, so it wouldn't make any sense to call it while the plugin is processing audio. And the documented invariants also forbid this (!process).
e) That indeed looks like a typo. Calling clap_host_params::request_flush() from the audio thread doesn't make any sense, since that would mean you're requesting a flush while in the middle of a process() call, where you could just output your parameter change events directly.

[tim-janik]

Clap 1.0.3 addresses some of the issues mentioned in this discussion, but not the request_flush one. Here's a comment addressing 1.0.3:

// This function is always safe to use and must not be called on the [audio-thread].
// [thread-safe,!audio-thread]
void (*request_flush)(const clap_host_t *host);

When it's thread-safe, it should be callable from any and all threads.
If it's !audio-thread, it should say "main-thread" (other threads are not mentioned/described in the clap spec).
"always safe" contradicts "not ... [audio-thread]".

Can we just say the plugin may call this at any time and from any thread?
The host must implement this in a thread-safe and efficient manner anyways...

[robbert-vdh]

It's more that calling this from the audio thread doesn't make any sense. It won't cause any issues, but why would you ever request a flush while you're inside of the process function?

[signaldust]

When you only spec [thread-safe] it's fine for the host to implement thread-safety using things like mutexes. If you also specced it [audio-thread] then the implementation would have to be real-time safe (ie. probably wait-free). So I don't see a contradiction here: [thread-safe] says it must work when called concurrently and [!audio-thread] says it need not be real-time safe.

Also, since audio thread is supposed to be getting regular process calls anyway, from the API point of view there is no reason why you would ever want to call request_flush from the audio thread... since the point of request_flush (as far as I understood the previous discussion here #103) is simply to notify the host that there are events pending, so it should flush unless it's going to be calling process anyway.

In fact.. I feel like perhaps the ideal "fix" to the specification would be to rename request_flush to events_pending or similar to emphasize that it's really just a notification for the host to call one of the methods that allows the plugin to drain it's outgoing event queue.

[tim-janik]

You are right (and @robbert-vdh), request_flush doesn't make sense in the audio-thread because plugin execution in the audio-thread is always part of flush() or process(), both of which have event output queues.
@abique explained that to me, and I think the point it crucial enough to go into the spec (at least I needed it to fully understand the CLAP audio-thread concept), It's submitted here: #131

[tim-janik]

I'd like to close this (mark as answered) but the reply from @signaldust cannot be marked as such.
So here is the relevant snippet from #131 that should clear things up:

   // Request a parameter flush. Note that this is not useful to call from an
   // [audio-thread], because a plugin executing within any [audio-thread] is either:
   //   1. within process() (which may include clap_plugin_thread_pool->exec)
   //   2. within flush()
   //
   // The host will then schedule a call to either:
   // - clap_plugin.process()
   // - clap_plugin_params->flush()
   //
   // This function is always safe to use and must not be called on the [audio-thread].
   // [thread-safe,!audio-thread]
   void (*request_flush)(const clap_host_t *host);