Add namespaces to nf-lang (#6176)

---------

Signed-off-by: Ben Sherman <bentshermann@gmail.com>

Author: bentsherman

docs/channel.md

@@ -2,81 +2,63 @@
 
 # Channels
 
-Nextflow is based on the dataflow programming model in which processes communicate through channels.
+In Nextflow, **channels** are the key data structures that facilitate the dataflow dependencies between each step (i.e. {ref}`process <process-page>`) in a pipeline.
 
-A channel has two major properties:
+There are two kinds of channels, *queue channels* and *value channels*. Channels are created using *channel factories* and transformed using *channel operators*.
 
-1. Sending a message is an *asynchronous* (i.e. non-blocking) operation, which means the sender doesn't have to wait for the receiving process.
-2. Receiving a message is a *synchronous* (i.e. blocking) operation, which means the receiving process must wait until a message has arrived.
-
-(channel-types)=
+(channel-type-queue)=
 
-## Channel types
+## Queue channels
 
-In Nextflow there are two kinds of channels: *queue channels* and *value channels*.
+A *queue channel* is a channel that *emits* an asynchronous sequence of values.
 
-(channel-type-queue)=
+A queue channel can be created by channel factories (e.g., {ref}`channel.of <channel-of>` and {ref}`channel.fromPath <channel-path>`), operators (e.g., {ref}`operator-map` and  {ref}`operator-filter`), and processes (see {ref}`Process outputs <process-output>`).
 
-### Queue channel
+The values in a queue channel cannot be accessed directly -- they can only be accessed by passing the channel as input to an operator or process. For example:
 
-A *queue channel* is a non-blocking unidirectional FIFO queue connecting a *producer* process (i.e. outputting a value)
-to a consumer process or an operator.
+```nextflow
+channel.of(1, 2, 3).view { v -> "queue channel emits ${v}" }
+```
 
-A queue channel can be created by factory methods ({ref}`channel-of`, {ref}`channel-path`, etc), operators ({ref}`operator-map`, {ref}`operator-flatmap`, etc), and processes (see {ref}`Process outputs <process-output>`).
+```console
+queue channel emits 1
+queue channel emits 2
+queue channel emits 3
+```
 
 (channel-type-value)=
 
-### Value channel
+## Value channels
 
-A *value channel* can be bound (i.e. assigned) with one and only one value, and can be consumed any number of times by
-a process or an operator.
+A *value channel* is a channel that is *bound* to an asynchronous value.
 
-A value channel can be created with the {ref}`channel-value` factory method or by any operator that produces a single value
-({ref}`operator-first`, {ref}`operator-collect`, {ref}`operator-reduce`, etc). Additionally, a process will emit value
-channels if it is invoked with all value channels, including simple values which are implicitly wrapped in a value channel.
+A value channel can be created with the {ref}`channel.value <channel-value>` factory, certain operators (e.g., {ref}`operator-collect` and {ref}`operator-reduce`), and processes (under {ref}`certain conditions <process-out-singleton>`).
 
-For example:
+The value in a value channel cannot be accessed directly -- it can only be accessed by passing the channel as input to an operator or process. For example:
 
 ```nextflow
-process echo {
-  input:
-  val x
-
-  output:
-  path 'x.txt'
-
-  script:
-  """
-  echo $x > x.txt
-  """
-}
-
-workflow {
-  result = echo(1)
-  result.view { file -> "Result: ${file}" }
-}
+channel.value(1).view { v -> "value channel is ${v}" }
 ```
 
-In the above example, since the `echo` process is invoked with a simple value instead of a channel, the input is implicitly
-wrapped in a value channel, and the output is also emitted as a value channel.
-
-See also: {ref}`process-multiple-input-channels`.
+```console
+value channel is 1
+```
 
 ## Channel factories
 
-Channel factories are functions that can create channels.
+Channel factories are functions that create channels from regular values.
 
-For example, the `channel.of()` factory can be used to create a channel from an arbitrary list of arguments:
+The `channel.fromPath()` factory creates a channel from a file name or glob pattern, similar to the `files()` function:
 
 ```nextflow
-channel.of(1, 2, 3).view()
+channel.fromPath('input/*.txt').view()
 ```
 
 See {ref}`channel-factory` for the full list of channel factories.
 
 ## Operators
 
-Channel operators, or _operators_ for short, are functions that consume and produce channels. Because channels are asynchronous, operators are necessary to manipulate the values in a channel, aside from using a process. As a result, operators are useful for implementing the _glue logic_ between processes.
+Channel operators, or *operators* for short, are functions that consume and produce channels. Because channels are asynchronous, operators are necessary to manipulate the values in a channel. Operators are particularly useful for implementing glue logic between processes.
 
 Commonly used operators include:
 

docs/process.md

@@ -776,9 +776,9 @@ The process `echo` is executed two times because the `x` channel emits only two
 2 and b
 ```
 
-A different semantic is applied when using a {ref}`value channel <channel-type-value>`. This kind of channel is created by the {ref}`channel.value <channel-value>` factory method or implicitly when a process is invoked with an argument that is not a channel. By definition, a value channel is bound to a single value and it can be read an unlimited number of times without consuming its content. Therefore, when mixing a value channel with one or more (queue) channels, it does not affect the process termination because the underlying value is applied repeatedly.
+When a {ref}`value channel <channel-type-value>` is supplied as a process input alongside a queue channel, the process is executed for each value in the queue channel, and the value channel is re-used for each execution.
 
-To better understand this behavior, compare the previous example with the following one:
+For example, compare the previous example with the following:
 
 ```nextflow
 process echo {
@@ -811,7 +811,7 @@ The above example executes the `echo` process three times because `x` is a value
 In general, multiple input channels should be used to process *combinations* of different inputs, using the `each` qualifier or value channels. Having multiple queue channels as inputs is equivalent to using the {ref}`operator-merge` operator, which is not recommended as it may lead to {ref}`non-deterministic process inputs <cache-nondeterministic-inputs>`.
 :::
 
-See also: {ref}`channel-types`.
+See also: {ref}`process-out-singleton`.
 
 (process-output)=
 
@@ -1160,6 +1160,22 @@ In this example, the process is normally expected to produce an `output.txt` fil
 While this option can be used with any process output, it cannot be applied to individual elements of a [tuple](#output-tuples-tuple) output. The entire tuple must be optional or not optional.
 :::
 
+(process-out-singleton)=
+
+### Singleton outputs
+
+When a process is only supplied with value channels, regular values, or no inputs, it returns outputs as value channels. For example:
+
+```{literalinclude} snippets/process-out-singleton.nf
+:language: nextflow
+```
+
+In the above example, the `echo` process is invoked with a regular value that is wrapped in a value channel. As a result, `echo` returns a value channel and `greet` is executed three times.
+
+If the call to `echo` was changed to `echo( channel.of('hello') )`, the process would instead return a queue channel, and `greet` would be executed only once.
+
+See also: {ref}`process-multiple-input-channels`.
+
 (process-when)=
 
 ## When

docs/reference/stdlib-namespaces.md

@@ -119,6 +119,20 @@ The `channel` namespace contains the built-in channel factories. See {ref}`chann
 
 (stdlib-namespaces-nextflow)=
 
+## `log`
+
+The `log` namepsace contains functions for logging messages to the console.
+
+`error( message: String )`
+: Log an error message to the console.
+: This function does not terminate the pipeline -- use the global `error()` function instead.
+
+`info( message: String )`
+: Log an info message to the console.
+
+`warn( message: String )`
+: Log a warning message to the console.
+
 ## `nextflow`
 
 The `nextflow` namespace contains information about the current Nextflow runtime.

docs/snippets/process-out-singleton.nf

@@ -0,0 +1,29 @@
+process echo {
+  input:
+  val greeting
+
+  output:
+  val greeting
+
+  exec:
+  true
+}
+
+process greet {
+  input:
+  val greeting
+  val name
+
+  output:
+  val "$greeting, $name!"
+
+  exec:
+  true
+}
+
+workflow {
+  names = channel.of( 'World', 'Mundo', 'Welt' )
+  greeting = echo('Hello')
+  result = greet(greeting, names)
+  result.view()
+}

docs/snippets/process-out-singleton.out

@@ -0,0 +1,3 @@
+Hello, World!
+Hello, Mundo!
+Hello, Welt!

modules/nf-lang/src/main/java/nextflow/script/control/VariableScopeChecker.java

@@ -187,7 +187,7 @@ private Variable findDslVariable(ClassNode cn, String name, ASTNode node) {
                 if( isDataflowMethod(mn) && name.equals(mn.getName()) ) {
                     return wrapMethodAsVariable(mn, name);
                 }
-                // built-in variables are methods annotated as @Constant
+                // built-in constants and namespaces are methods annotated as @Constant
                 var an = findAnnotation(mn, Constant.class);
                 if( !an.isPresent() )
                     continue;

modules/nf-lang/src/main/java/nextflow/script/control/VariableScopeVisitor.java

@@ -443,7 +443,7 @@ private boolean declareAssignedVariable(VariableExpression ve) {
         var variable = vsc.findVariableDeclaration(ve.getName(), ve);
         if( variable != null ) {
             if( isDslVariable(variable) )
-                vsc.addError("Built-in variable cannot be re-assigned", ve);
+                vsc.addError("Built-in constant or namespace cannot be re-assigned", ve);
             ve.setAccessedVariable(variable);
             return false;
         }
@@ -492,7 +492,7 @@ else if( node instanceof BinaryExpression be && be.getOperation().getType() == T
                 vsc.addWarning("Params should be declared at the top-level (i.e. outside the workflow)", target.getName(), target);
             // TODO: re-enable after workflow.onComplete bug is fixed
             // else
-            //     vsc.addError("Built-in variable cannot be mutated", target);
+            //     vsc.addError("Built-in constant or namespace cannot be mutated", target);
         }
         else if( variable != null ) {
             checkExternalWriteInAsyncClosure(target, variable);
@@ -529,13 +529,20 @@ public void visitMethodCallExpression(MethodCallExpression node) {
             declareAssignedVariable(target);
             return;
         }
+        if( node.getObjectExpression() instanceof VariableExpression ve )
+            checkClassNamespaces(ve);
         checkMethodCall(node);
         var ioc = inOperatorCall;
         inOperatorCall = isOperatorCall(node);
         super.visitMethodCallExpression(node);
         inOperatorCall = ioc;
     }
 
+    private void checkClassNamespaces(VariableExpression node) {
+        if( "Channel".equals(node.getName()) )
+            vsc.addWarning("The use of `Channel` to access channel factories is deprecated -- use `channel` instead", "CHannel", node);
+    }
+
     private static boolean isOperatorCall(MethodCallExpression node) {
         return node.getNodeMetaData(ASTNodeMarker.METHOD_TARGET) instanceof MethodNode mn
             && VariableScopeChecker.isOperator(mn);

modules/nf-lang/src/main/java/nextflow/script/dsl/DslScope.java

@@ -17,7 +17,7 @@
 
 /**
  * Marker interface for DSL scopes, which define the built-in
- * variables and functions for a particular context.
+ * constants and functions for a particular context.
  *
  * @author Ben Sherman <bentshermann@gmail.com>
  */

modules/nf-lang/src/main/java/nextflow/script/dsl/Namespace.java

@@ -0,0 +1,24 @@
+/*
+ * Copyright 2024-2025, Seqera Labs
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package nextflow.script.dsl;
+
+/**
+ * Marker interface for namespaces.
+ *
+ * @author Ben Sherman <bentshermann@gmail.com>
+ */
+public interface Namespace {
+}

modules/nf-lang/src/main/java/nextflow/script/dsl/ScriptDsl.java

@@ -21,16 +21,50 @@
 import java.util.Map;
 
 import groovy.lang.Closure;
-import nextflow.script.types.NextflowMetadata;
-import nextflow.script.types.WorkflowMetadata;
+import nextflow.script.namespaces.ChannelNamespace;
+import nextflow.script.namespaces.LogNamespace;
+import nextflow.script.namespaces.NextflowNamespace;
+import nextflow.script.namespaces.WorkflowNamespace;
 
 /**
- * The built-in constants and functions in a script.
+ * The built-in namespaces, constants, and functions in a script.
  *
  * @author Ben Sherman <bentshermann@gmail.com>
  */
 public interface ScriptDsl extends DslScope {
 
+    // namespaces
+
+    @Constant("channel")
+    @Description("""
+        The `channel` namespace contains the built-in channel factories.
+
+        [Read more](https://nextflow.io/docs/latest/reference/channel.html)
+    """)
+    ChannelNamespace getChannel();
+
+    @Constant("log")
+    @Description("""
+        The `log` namepsace contains functions for logging messages to the console.
+
+        [Read more](https://nextflow.io/docs/latest/reference/stdlib-namespaces.html#log)
+    """)
+    LogNamespace getLog();
+
+    @Constant("nextflow")
+    @Description("""
+        The `nextflow` namespace contains information about the current Nextflow runtime.
+    """)
+    NextflowNamespace getNextflow();
+
+    @Constant("workflow")
+    @Description("""
+        The `workflow` namespace contains information about the current workflow run.
+    """)
+    WorkflowNamespace getWorkflow();
+
+    // constants
+
     @Deprecated
     @Constant("baseDir")
     @Description("""
@@ -44,24 +78,12 @@ public interface ScriptDsl extends DslScope {
     """)
     Path getLaunchDir();
 
-    @Constant("log")
-    @Description("""
-        Logger which can be used to log messages to the console.
-    """)
-    Object getLog();
-
     @Constant("moduleDir")
     @Description("""
         The directory where a module script is located (equivalent to `projectDir` if used in the main script).
     """)
     Path getModuleDir();
 
-    @Constant("nextflow")
-    @Description("""
-        Map of Nextflow runtime information.
-    """)
-    NextflowMetadata getNextflow();
-
     @Constant("projectDir")
     @Description("""
         Alias of `workflow.projectDir`.
@@ -80,11 +102,7 @@ The directory where a module script is located (equivalent to `projectDir` if us
     """)
     Path getWorkDir();
 
-    @Constant("workflow")
-    @Description("""
-        Map of workflow runtime information.
-    """)
-    WorkflowMetadata getWorkflow();
+    // functions
 
     @Description("""
         Create a branch criteria to use with the `branch` operator.