Skip to content

Commit ff8c6d6

Browse files
[CONNECT] CommandPlugin et al.
1 parent e1537dd commit ff8c6d6

File tree

5 files changed

+91
-1
lines changed

5 files changed

+91
-1
lines changed

docs/connect/CommandPlugin.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
11
# CommandPlugin
22

3-
`CommandPlugin` is...FIXME
3+
`CommandPlugin` is an [abstraction](#contract) of [command processors](#implementations) in [Spark Connect](./index.md) that can [process a command](#process).
4+
5+
## Contract
6+
7+
### Process Command { #process }
8+
9+
```scala
10+
process(
11+
command: protobuf.Any,
12+
planner: SparkConnectPlanner): Option[Unit]
13+
```
14+
15+
Used when:
16+
17+
* `SparkConnectPlanner` is requested to [handleCommandPlugin](SparkConnectPlanner.md#handleCommandPlugin)
18+
19+
## Implementations
20+
21+
!!! note
22+
No built-in implementations available.

docs/connect/ExecuteHolder.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# ExecuteHolder
2+
3+
## Creating Instance
4+
5+
`ExecuteHolder` takes the following to be created:
6+
7+
* <span id="request"> `ExecutePlanRequest`
8+
* <span id="sessionHolder"> `SessionHolder`
9+
10+
`ExecuteHolder` is created when:
11+
12+
* `SparkConnectExecutionManager` is requested to [create an ExecuteHolder](SparkConnectExecutionManager.md#createExecuteHolder)

docs/connect/ExecuteThreadRunner.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# ExecuteThreadRunner
2+
3+
`ExecuteThreadRunner` is created alongside [ExecuteHolder](ExecuteHolder.md#runner).
4+
5+
## Creating Instance
6+
7+
`ExecuteThreadRunner` takes the following to be created:
8+
9+
* <span id="executeHolder"> `ExecuteHolder`
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# SparkConnectExecutionManager
2+
3+
## Creating Instance
4+
5+
`SparkConnectExecutionManager` takes no arguments to be created.
6+
7+
`SparkConnectExecutionManager` is created when:
8+
9+
* `SparkConnectService` is requested for the [executionManager](SparkConnectService.md#executionManager)

docs/connect/SparkConnectPlanner.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,3 +74,44 @@ transformPythonFunction(
7474
`transformPythonFunction` is used when:
7575

7676
* `SparkConnectPlanner` is requested to [transformPythonUDF](#transformPythonUDF) and [handleRegisterPythonUDF](#handleRegisterPythonUDF)
77+
78+
## Process Command { #process }
79+
80+
```scala
81+
process(
82+
command: proto.Command,
83+
responseObserver: StreamObserver[ExecutePlanResponse],
84+
executeHolder: ExecuteHolder): Unit
85+
```
86+
87+
`process` handles the input `Command` based on its type.
88+
89+
Command Type | Handler
90+
-|-
91+
`REGISTER_FUNCTION` | [handleRegisterUserDefinedFunction](#handleRegisterUserDefinedFunction)
92+
`REGISTER_TABLE_FUNCTION` | [handleRegisterUserDefinedTableFunction](#handleRegisterUserDefinedTableFunction)
93+
`WRITE_OPERATION` | [handleWriteOperation](#handleWriteOperation)
94+
`CREATE_DATAFRAME_VIEW` | [handleCreateViewCommand](#handleCreateViewCommand)
95+
`WRITE_OPERATION_V2` | [handleWriteOperationV2](#handleWriteOperationV2)
96+
`EXTENSION` | [handleCommandPlugin](#handleCommandPlugin)
97+
`SQL_COMMAND` | [handleSqlCommand](#handleSqlCommand)
98+
`WRITE_STREAM_OPERATION_START` | [handleWriteStreamOperationStart](#handleWriteStreamOperationStart)
99+
`STREAMING_QUERY_COMMAND` | [handleStreamingQueryCommand](#handleStreamingQueryCommand)
100+
`STREAMING_QUERY_MANAGER_COMMAND` | [handleStreamingQueryManagerCommand](#handleStreamingQueryManagerCommand)
101+
`GET_RESOURCES_COMMAND` | [handleGetResourcesCommand](#handleGetResourcesCommand)
102+
103+
---
104+
105+
`process` is used when:
106+
107+
* `ExecuteThreadRunner` is requested to [handle a command](ExecuteThreadRunner.md#handleCommand)
108+
109+
### handleCommandPlugin { #handleCommandPlugin }
110+
111+
```scala
112+
handleCommandPlugin(
113+
extension: ProtoAny,
114+
executeHolder: ExecuteHolder): Unit
115+
```
116+
117+
`handleCommandPlugin`...FIXME

0 commit comments

Comments
 (0)