Skip to content

[SPARK-52277] Upgrade Docker tags to 4.0.0 instead of 4.0.0-preview2 #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/integration_test_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
SPARK_REMOTE: "sc://localhost:15003"
services:
spark:
image: apache/spark:4.0.0-preview2
image: apache/spark:4.0.0
env:
SPARK_NO_DAEMONIZE: 1
ports:
Expand Down
6 changes: 3 additions & 3 deletions Examples/app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This is an example Swift application to show how to use Apache Spark Connect Swi
Prepare `Spark Connect Server` via running Docker image.

```
docker run --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
docker run --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
```

Build an application Docker image.
Expand All @@ -23,7 +23,7 @@ Run `app` docker image.

```
$ docker run --rm -e SPARK_REMOTE=sc://host.docker.internal:15002 apache/spark-connect-swift:app
Connected to Apache Spark 4.0.0-preview2 Server
Connected to Apache Spark 4.0.0 Server
EXECUTE: DROP TABLE IF EXISTS t
EXECUTE: CREATE TABLE IF NOT EXISTS t(a INT) USING ORC
EXECUTE: INSERT INTO t VALUES (1), (2), (3)
Expand Down Expand Up @@ -52,7 +52,7 @@ Run from source code.
```
$ swift run
...
Connected to Apache Spark 4.0.0-preview2 Server
Connected to Apache Spark 4.0.0 Server
EXECUTE: DROP TABLE IF EXISTS t
EXECUTE: CREATE TABLE IF NOT EXISTS t(a INT) USING ORC
EXECUTE: INSERT INTO t VALUES (1), (2), (3)
Expand Down
2 changes: 1 addition & 1 deletion Examples/pi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is an example Swift application to show how to use Apache Spark Connect Swi

Prepare `Spark Connect Server` via running Docker image.
```
docker run --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
docker run --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
```

Build an application Docker image.
Expand Down
4 changes: 2 additions & 2 deletions Examples/spark-sql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This is an example Swift application to show how to develop a Spark SQL REPL(Rea
Prepare `Spark Connect Server` via running Docker image.

```
docker run -it --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
docker run -it --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
```

Build an application Docker image.
Expand All @@ -23,7 +23,7 @@ Run `spark-sql` docker image.

```
$ docker run -it --rm -e SPARK_REMOTE=sc://host.docker.internal:15002 apache/spark-connect-swift:spark-sql
Connected to Apache Spark 4.0.0-preview2 Server
Connected to Apache Spark 4.0.0 Server
spark-sql (default)> SHOW DATABASES;
+---------+
|namespace|
Expand Down
2 changes: 1 addition & 1 deletion Examples/stream/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This is an example Swift stream processing application to show how to count word
## Run `Spark Connect Server`

```bash
docker run --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait -c spark.log.level=ERROR"
docker run --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait -c spark.log.level=ERROR"
```

## Run `Netcat` as a streaming input server
Expand Down
2 changes: 1 addition & 1 deletion Examples/web/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ index 2edcc8f..22313c8 100644
Prepare `Spark Connect Server` via running Docker image.

```
docker run --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
docker run --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait"
```

Build an application Docker image.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

This is an experimental Swift library to show how to connect to a remote Apache Spark Connect Server and run SQL statements to manipulate remote data.

So far, this library project is tracking the upstream changes like the [Apache Spark](https://spark.apache.org) 4.0.0 RC7 release and [Apache Arrow](https://arrow.apache.org) project's Swift-support.
So far, this library project is tracking the upstream changes of [Apache Arrow](https://arrow.apache.org) project's Swift-support.

## Resources

Expand All @@ -16,7 +16,7 @@ So far, this library project is tracking the upstream changes like the [Apache S

## Requirement

- [Apache Spark 4.0.0 RC7 (May 2025)](https://github.com/apache/spark/releases/tag/v4.0.0)
- [Apache Spark 4.0.0 (May 2025)](https://github.com/apache/spark/releases/tag/v4.0.0)
- [Swift 6.0 (2024) or 6.1 (2025)](https://swift.org)
- [gRPC Swift 2.2 (May 2025)](https://github.com/grpc/grpc-swift/releases/tag/2.2.1)
- [gRPC Swift Protobuf 1.3 (May 2025)](https://github.com/grpc/grpc-swift-protobuf/releases/tag/1.3.0)
Expand Down
4 changes: 2 additions & 2 deletions Sources/SparkConnect/Documentation.docc/Examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This document provides an overview of the example applications inside [Examples]
Start a Spark Connect Server:

```bash
docker run -it --rm -p 15002:15002 apache/spark:4.0.0-preview2 bash -c "/opt/spark/sbin/start-connect-server.sh --wait -c spark.log.level=ERROR"
docker run -it --rm -p 15002:15002 apache/spark:4.0.0 bash -c "/opt/spark/sbin/start-connect-server.sh --wait -c spark.log.level=ERROR"
```

## Basic Application Example
Expand Down Expand Up @@ -147,7 +147,7 @@ Welcome to the Swift world. Say hello!%

# Spark-powered endpoint
curl http://127.0.0.1:8080/hello
Hi, this is powered by the Apache Spark 4.0.0-preview2.%
Hi, this is powered by the Apache Spark 4.0.0.%
```

## Development Environment
Expand Down
Loading