PICARD-2101: Enhance script autocompletion with dynamic variable parsing and plugin support

[knguyen1]

Summary

• This is a…
  • Bug fix
  • Feature addition
  • Refactoring
  • Minor / simple change (like a typo)
  • Other
• Describe this change in 1-2 sentences: Enhanced script autocompletion with dynamic variable parsing, plugin variable support, and context-aware completion for improved user experience.

Problem

• JIRA ticket (optional): PICARD-2101

The current script autocompletion system has several limitations:

• Missing custom variables: When users create variables with $set(), those variables don't appear in autocompletion
• No dynamic discovery: The system doesn't learn from existing scripts or current script content
• Limited context awareness: No smart suggestions based on what the user is currently working on
• No plugin extensibility: Plugins cannot register additional variables for completion

Solution

Implemented three major enhancements to the script completion system:

1. Dynamic Script Parsing

• Real-time parsing of currently edited script to extract $set(variable_name, ...) calls
• Uses existing ScriptParser for safe parsing with AST traversal
• Caches parsed variables and updates on script changes
• Includes regex fallback for resilience during live edits

2. Plugin Variable Extension

• New extension point ext_point_script_variables for plugins to register variables
• register_script_variable(name, documentation) function for plugin integration
• Plugin variables appear in completion with %variable_name% format
• Full integration with existing completion ranking and sorting

3. Context-Aware Completion

• Smart completion based on cursor position and context
• Function parameter completion: Shows tag names when cursor is inside $set(, $get(, etc.
• Variable usage completion: Shows relevant variables when typing %
• Smart filtering: Prioritizes recently used or contextually relevant variables
• Handles complex nested structures and edge cases

Technical Implementation:

• Extended ScriptCompleter with dynamic variable extraction methods
• Added comprehensive test coverage for all new functionality
• Maintains backward compatibility with existing completion behavior
• Follows established extension point patterns used throughout Picard

Action

Additional actions required:

• Update Picard documentation (please include a reference to this PR)
• Other (please specify below)

[Sophist-UK]

Thanks for putting so much effort in coding the tests cf. coding the solution.

[rdswift]

Will it properly interpret cases where the variable name is passed indirectly to the $set() command? For example:

$noop( Create a separate tag for each artist listed for the
       track as 'artist_1', 'artist_2', etc. )
$foreach(%artists%,$set(artist_%_loop_count%,%_loop_value%))

Will this include the resulting artist_1, artist_2, etc. variables in the autocomplete list?

[rdswift]

Something for future consideration is to allow translated description strings for variables added (registered) by a plugin. I suppose that could be handled within the new plugin system, so the translated description is passed when registering the variable.

[knguyen1]

Will this include the resulting artist_1, artist_2, etc. variables in the autocomplete list?

Out of scope of original JIRA requirement. Current AST parsing + regex fallback only covers statically named variables.

If you'd like me to proceed with dynamic var's, have to vote on it. I think it can be done with variable_extractor.py changes.

EDIT: Way too complex to solve--I will not be doing it. Besides, I think your example $foreach(%artists%,$set(artist_%_loop_count%,%_loop_value%)) relies on runtime context (the %artists% value) that the script parser does not have at time of parsing. One can probably write a parser for known formats like $foreach(1;2;3,$set(test_%_loop_count%,%_loop_value%)) though.

[zas]

LGTM

Tested, work as expected for me.

[phw]

This is a really great addition. Both auto completion of user defined or plugin defined variables, but also the great idea of triggering variable automcompletion for functions like $set etc.

Only found one bug in the UI that I'd like to see resolved before merging: Once any input results in a parsing error and the red error bar is being shown, the error bar is not hidden again once the error is resolved.

[phw]

EDIT: Way too complex to solve--I will not be doing it. Besides, I think your example $foreach(%artists%,$set(artist_%_loop_count%,%_loop_value%)) relies on runtime context (the %artists% value) that the script parser does not have at time of parsing. One can probably write a parser for known formats like $foreach(1;2;3,$set(test_%_loop_count%,%_loop_value%)) though.

Agreed that this is too complex to solve, and won't capture any case that requires context. Let's focus on statically declared variable names.

[knguyen1]

@phw That's out of scope of this PR's changes; it touches the file picard/ui/options/scripting.py. Regardless here is the change needed: fix(scripting_options): bug red error bar + add debounce

Also, I found it strange that the red error bar is unforgiving against pauses in typing. So added a debounce of 350ms.

[phw]

Thanks for the fixes, and good idea with the debouncing. Just tested, works great.

[phw]

That's out of scope of this PR's changes; it touches the file picard/ui/options/scripting.py. Regardless here is the change needed: fix(scripting_options): bug red error bar + add debounce

You are actually right, same issue is on master branch. So that's a regression on master already (works fine in the 2.x versions). I had assumed it would be broken in this branch only. But good to have it fixed regardless.