CASSANDRA-19978 Testcontainers integration #1780

ribaraka · 2024-07-15T18:26:00Z

In this PR, the CCM integration test runner has been replaced with the testcontainers-go library.

I utilized a main_test.go file to provide a centralized way to manage test setup and teardown logic. This approach initializes shared resources before any tests are run and cleans them up afterward, ensuring a consistent and efficient testing environment. The main_test.go file serves as the entry point for the integration tests and uses tags to omit building a cluster for unit tests.

Additionally, a bash script was added to dynamically change Cassandra's config file on the fly for a specific Cassandra version. I aimed to maintain consistency in the code so the configuration would look familiar.

Now, the GitHub Actions workflow uses the new testcontainers-go integration test runner instead of CCM. The Go version has been bumped up to 1.23 to ensure compatibility with the testcontainers-go package.

Added documentation on how to run an integration test runner locally.

Fixed flaky test. Closes #1795

I also took the liberty of removing all CCM and related components

mdelapenya · 2024-09-05T08:16:10Z

go.mod

 	github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
-	github.com/kr/pretty v0.1.0 // indirect
-	github.com/stretchr/testify v1.3.0 // indirect
+	github.com/testcontainers/testcontainers-go v0.32.0


You probably want to use the latest v0.33.0. just in case.

upgraded to v0.33.0

mdelapenya · 2024-09-05T08:16:59Z

main_test.go

+			Name: "cassandra",
+		},
+	}
+	cassandraNetwork, err := testcontainers.GenericNetwork(ctx, networkRequest)


GenericNetwork is deprecated. I'd use network.New instead. Please see https://golang.testcontainers.org/features/creating_networks/

I'm fixing the flacke integration tests, and the whole setup has been refactored, including the part you've suggested. I hope you will be able to review it as well, once it's pushed.

A commit fixing the flaky tests has already been pushed, and all integration replacements have been completed. Please take a look

mdelapenya · 2024-09-05T08:18:23Z

main_test.go

+	defer cassandraNetwork.Remove(ctx)
+
+	// Function to create a Cassandra container (node)
+	createCassandraContainer := func(number int) (string, error) {


I'm not a Cassandra expert, so you probably want to take a look and customise the existing cassandra Go module with your code: https://golang.testcontainers.org/modules/cassandra/ (even contributing to the module with updates would be great!)

The current version of gocql is 1.13, and the decision on which version to upgrade to hasn't been finalized yet. For now, I’m keeping things simple by not using the modules.

Initially, I started with the cass module, but I didn't find it handy. Managing an additional import also felt unnecessary since the generic structs provided most of the required functionality.

Once the Go version is decided, I plan to revise the Cassandra model and see what I can do to enhance the functionality that I might have been missing and make it handy 😄

mdelapenya · 2024-09-05T08:23:18Z

main_test.go

+			LifecycleHooks: []testcontainers.ContainerLifecycleHooks{{
+				PostStarts: []testcontainers.ContainerHook{
+					func(ctx context.Context, c testcontainers.Container) error {
+						// wait for cassandra config.yaml to initialize
+						time.Sleep(100 * time.Millisecond)
+
+						code, _, err := c.Exec(ctx, []string{"bash", "./update_container_cass_config.sh"})
+						if err != nil {
+							return err
+						}
+						if code != 0 {
+							return fmt.Errorf("script ./update_container_cass_config.sh exited with code %d", code)
+						}
+						return nil
+					},
+				},
+			}},


You can have a wait.ForAll strategy in which you add the wait for log below, and a wait.ForExec https://golang.testcontainers.org/features/wait/exec/

It's actually could be a nice way to handle it. Thank you! I'll try it.

Unfortunately, your advice didn’t resolve the issue. Waiting for any logs didn’t yield useful results, as I was attempting to change certain Cassandra configuration properties on the fly. These properties can also vary depending on the Cassandra version in use.

My solution using a hook also had flaws, as the hook couldn’t pause the bootstrapping process to allow the execution of custom code at the necessary point. As a result, my custom script couldn’t be executed in time.

To address this, I made some adjustments to the docker-entrypoint, which successfully handled the required tasks during the initialization process.

Thanks for your feedback, I'm a total newbie in Cassandra and you better than me know how ti start it 😊 In the current Cassandra module we do this: https://github.com/testcontainers/testcontainers-go/blob/main/modules/cassandra/cassandra.go#L97, although looking at this PR, it seems it's creating a rock-solid implementation for cassandra. Would you like to, at some point, contribute to the module? 🙏 Then the entire community would benefit from your changes. Of course, only if you want to.

mdelapenya · 2024-09-05T08:55:18Z

Hi @ribaraka, this is Manu, testcontainers-go maintainer! 👋 Thanks for adding the library to this project! My colleague told me about this PR so I took the chance to add a few suggestions in case you want to enhance the experience of using the library. Please let me know your thoughts about them, thanks!

ribaraka · 2024-09-05T14:45:23Z

Hi @mdelapenya!
Thank you so much for taking the time to review my PR and for your suggestions. They’re quite interesting, I’ll definitely use some of them. 😊

ribaraka · 2024-09-12T10:28:53Z

added fix for flaky tests to this PR.

Closes #1795

@Zariel, could you please review it?

ribaraka · 2024-09-23T10:11:36Z

@martin-sucha, after integrating the new test runner for the driver and fixing all tests marked as a flake, I ran coverage tests for all existing unit and integration tests, merged the results into one report, and the current test coverage stands at 76%.

I can send you the coverage report via Slack or attach it here. Alternatively, you can fetch this branch and generate it yourself. Here’s how I do it

// generate a report for all integration tests
go test -v -covermode=atomic -coverprofile=coverage_integration.out -tags "integration cassandra tc gocql_debug" -timeout=30m -gocql.timeout=60s -runssl -proto=4 -rf=3 -clusterSize=3 -autowait=2000ms -compressor=snappy -gocql.cversion=5.0.0  ./...

// generate a report for auth test
go test -v -covermode=atomic -coverprofile=coverage_auth.out -run=TestAuthentication -tags ""integration" gocql_debug" -timeout=2m -runauth -gocql.timeout=60s -runssl -proto=4 -rf=3 -clusterSize=1 -autowait=2000ms -runauth -compressor=snappy -gocql.cversion=5.0.0

// generate a report for all unit tests
 go test -v -covermode=atomic -coverprofile=coverage_unit.out -tags unit -race
 
 
 // once all reports have been generated, use the gocovmerge tool to run the merge command and retrieve the complete report for the entire driver.
  gocovmerge coverage_auth.out coverage_integration.out coverage_unit.out > coverage_merged.out

// view the merged coverage report
go tool cover -func=coverage_merged.out

mdelapenya

From Testcontainers standpoint, LGTM

As a follow-up, I'd like to work with you in using the cassandra tc-go module, or contributing to it to receive the enhancements in this PR.

…e driver

OleksiienkoMykyta

I would rename it from "CASSANDRA-19978" to "CASSGO-17". It's quite useful and looks like fixes the original issue. There are some issues with running testcontainers on Windows and macOS. According to the discussion in jira ticket, there is still no consensus about the usage of the testcontainers, and it looks stuck.
@joao-r-reis, should we focus on fixing run issues on Windows and macOS, or are you thinking about closing this? I think for most users would be easier to use testcontainers, but also we should keep drivers in sync, you have a point there.

joao-r-reis · 2025-03-18T13:56:37Z

I believe we had some discussons about this and we decided not to go for testcontainers because it would be significant departure from what other drivers do (including the java driver which is also ASF branded atm) and also because there's some limitations on testcontainers irt running multi node clusters. If anyone is interested there's a lot of context in the JIRA comment history.

I believe this should be closed for now.

ribaraka changed the title ~~Testcontainers integration~~ [WIP]Testcontainers integration Jul 15, 2024

ribaraka mentioned this pull request Jul 15, 2024

Rewrite integration test setup in Go #1686

Open

ribaraka force-pushed the testcontainers branch from 00a7f1b to 9e5c433 Compare July 31, 2024 09:13

ribaraka changed the title ~~[WIP]Testcontainers integration~~ Testcontainers integration Jul 31, 2024

ribaraka mentioned this pull request Aug 12, 2024

Fix flaky tests #1795

Open

ribaraka force-pushed the testcontainers branch 2 times, most recently from 7667070 to 4797f5a Compare September 4, 2024 16:14

mdelapenya reviewed Sep 5, 2024

View reviewed changes

ribaraka force-pushed the testcontainers branch 3 times, most recently from bc39d7f to c9e43bd Compare September 16, 2024 08:32

ribaraka mentioned this pull request Sep 19, 2024

CASSANDRA-19796 – cassandra-gocql-driver CI #1812

Closed

ribaraka requested a review from mdelapenya September 19, 2024 13:43

ribaraka force-pushed the testcontainers branch from 696af25 to 12ba1c5 Compare September 20, 2024 13:55

mdelapenya approved these changes Sep 23, 2024

View reviewed changes

joao-r-reis changed the title ~~Testcontainers integration~~ CASSANDRA-19978 Testcontainers integration Oct 4, 2024

ribaraka and others added 9 commits October 6, 2024 13:11

testcontainers-go integration

2bc28c1

auth test

472fb2b

ci bump to 1.23

e7bb5c7

fix flaky tests

2cae706

doc: running integration tests locally

a20211e

adding cass 5.0.0

0fb98c4

Remove the CCM test integration runner and related components from th…

8887c18

…e driver

minor fix

9fa9ab5

Increasing the timeout duration for cluster recovery.

737f415

ribaraka force-pushed the testcontainers branch from 12ba1c5 to 737f415 Compare October 6, 2024 10:12

test:connect all nodes

9c34737

ribaraka force-pushed the testcontainers branch from a9e7e8a to 9c34737 Compare October 15, 2024 19:49

OleksiienkoMykyta reviewed Mar 18, 2025

View reviewed changes

CASSANDRA-19978 Testcontainers integration #1780

Are you sure you want to change the base?

CASSANDRA-19978 Testcontainers integration #1780

Uh oh!

Conversation

ribaraka commented Jul 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ribaraka Sep 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ribaraka Sep 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdelapenya commented Sep 5, 2024

Uh oh!

ribaraka commented Sep 5, 2024

Uh oh!

ribaraka commented Sep 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ribaraka commented Sep 23, 2024

Uh oh!

mdelapenya left a comment

Choose a reason for hiding this comment

Uh oh!

OleksiienkoMykyta left a comment

Choose a reason for hiding this comment

Uh oh!

joao-r-reis commented Mar 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ribaraka commented Jul 15, 2024 •

edited

Loading

ribaraka Sep 5, 2024 •

edited

Loading

ribaraka Sep 5, 2024 •

edited

Loading

ribaraka commented Sep 12, 2024 •

edited

Loading