Skip to content

Conversation

@andrii-kysylevskyi
Copy link

@andrii-kysylevskyi andrii-kysylevskyi commented Nov 2, 2025

Resolves issue 24924

Motivation

ScyllaDB is a drop-in, more performance-focused replacement for Apache Cassandra. It works out of the box with Pulsar simply by replacing the Cassandra connection string with a ScyllaDB one in the liquibase-cassandra extension (verified this locally)

There are already some articles on integrating Pulsar with Scylla here and here. So it might be worth solidifying this in Pulsar's docs and tests.

Modifications

  • A new ScyllaDBContainer class in tests/integration/containers
  • A new ScyllaDBSinkTester in integration/io/sinks (using the same Sink class as Cassandra)
  • New methods in PulsarSinksTest - testScyllaDBSink and testScyllaDBArchiveSink

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

mvn -DintegrationTests -pl tests/integration -Dtest=PulsarSinksTest#testScyllaDBSink,PulsarSinksTest#testScyllaDBArchiveSink -DtestRetryCount=0 \
 -DredirectTestOutputToFile=false test

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Doc PR available here

Matching PR in forked repository

PR in forked repository: andrii-kysylevskyi#1

@github-actions github-actions bot added the doc-required Your PR changes impact docs and you will update later. label Nov 2, 2025
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Please remove the changes to the top-level README.md file.

README.md Outdated
Comment on lines 56 to 66
## Database Integrations

Pulsar provides native sink connectors for NoSQL databases:

- **Apache Cassandra**: Official Cassandra sink connector for streaming key-value data
- **ScyllaDB**: Drop-in compatible with Cassandra sink connector. ScyllaDB offers high performance and low latency while maintaining full Cassandra protocol compatibility. See [ScyllaDB Integration Guide](SCYLLADB_INTEGRATION.md) for details.

**Learn more:**
- [Connect Pulsar to Cassandra/ScyllaDB](https://pulsar.apache.org/docs/4.1.x/io-quickstart/#connect-pulsar-to-cassandra)
- [Streaming Real-Time Chat Messages into ScyllaDB with Apache Pulsar](https://www.scylladb.com/2022/04/25/streaming-real-time-chat-messages-into-scylladb-with-apache-pulsar/)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The database integrations shouldn't be high-lighted in apache/pulsar README.
Please remove the changes to README.md.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid statements like "ScyllaDB offers high performance and low latency while maintaining full Cassandra protocol compatibility." that is marketing instead be more neutral like "ScyllaDB offers full Cassandra protocol compatibility." Users choosing ScyllaDB already have chosen it over Cassandra.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback! I have removed the change altogether, since it does not belong in this README as per @lhotari. I have also double-checked the rest of the codebase - don't think I have other ScyllaDB performance claims, so hope this works

@lhotari lhotari changed the title [feat][databases] Added ScyllaDB tests and README notes [feat][io] Added ScyllaDB tests Nov 3, 2025
@lhotari lhotari changed the title [feat][io] Added ScyllaDB tests [feat][io] Add ScyllaDB tests Nov 3, 2025
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-required Your PR changes impact docs and you will update later.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants