Skip to content

Docs optimization #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions docs/docs-src/core/antora.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
name: cdc-for-cassandra
title: DataStax CDC for Apache Cassandra
name: 'cdc-for-cassandra'
title: 'DataStax CDC for Apache Cassandra'
version: '2.2.9'
display_version: '2.2.9'
start_page: index.adoc
start_page: 'index.adoc'

nav:
- modules/ROOT/nav.adoc

asciidoc:
attributes:
cdc_cass_first: 'DataStax CDC for Apache Cassandra(R)'
Expand All @@ -13,5 +16,3 @@ asciidoc:
luna_version: '2.10'
pulsar_version: '2.10'
version: '2.2.9' # cdc-apache-cassandra latest {version}
nav:
- modules/ROOT/nav.adoc
6 changes: 3 additions & 3 deletions docs/docs-src/core/modules/ROOT/pages/backfill-cli.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ cd cdc-apache-cassandra
Result::
+
--
[source,bash]
[source,console]
----
BUILD SUCCESSFUL in 37s
17 actionable tasks: 15 executed, 2 up-to-date
Expand Down Expand Up @@ -175,7 +175,7 @@ Your data topic is now populated with the existing data from the Cassandra table
.Pulsar consumer subscription output:
[%collapsible]
====
[source,plain]
[source,console]
----
----- got message -----
2023-04-14T11:47:48,652-0400 [main] INFO org.apache.pulsar.client.impl.schema.AutoConsumeSchema - Configure topic schema \x00\x00\x00\x00\x00\x00\x00\x00 for topic persistent://public/default/data-ks1.table1 : {"key":{"name":"table1","schema":{"type":"record","name":"table1","namespace":"ks1","doc":"Table ks1.table1","fields":[{"name":"id","type":"string"}]},"type":"AVRO","timestamp":0,"properties":{}},"value":{"name":"table1","schema":{"type":"record","name":"table1","namespace":"ks1","doc":"Table ks1.table1","fields":[{"name":"val","type":["null","string"]}]},"type":"AVRO","timestamp":0,"properties":{}}}
Expand Down Expand Up @@ -236,7 +236,7 @@ When running the backfill CLI as a Pulsar-admin extension, all `--pulsar-*` para
The `--dsbulk-log-dir` is only available when running the backfill CLI as a standalone Java application.

The `--export-dsbulk-option` parameter passes extra parameters to DSBulk.
The relevant DSBulk settings configure the CSV connector and can be found https://github.com/datastax/dsbulk/blob/1.10.x/manual/settings.md#connector.csv[here^].
The relevant DSBulk settings configure the CSV connector and can be found https://github.com/datastax/dsbulk/blob/1.10.x/manual/settings.md#connector.csv[here].
Shortened option names are not supported.

In both the Pulsar-admin extension and the standalone Java application, C* related configurations are exclusively passed as command line arguments.
Expand Down
18 changes: 9 additions & 9 deletions docs/docs-src/core/modules/ROOT/pages/cdcExample.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Capture schema changes in your C* tables and pass them to Apache Pulsar(R) with DataStax Change Data Capture (CDC). This doc will guide you through installing, configuring, and using CDC with C* or DSE in a VM-based deployment.

This installation requires the following. Latest version artifacts are available https://github.com/datastax/cdc-apache-cassandra/releases/latest[here^]. Use image:https://img.shields.io/github/v/release/datastax/cdc-apache-cassandra?color=green&display_name=tag[link="https://github.com/datastax/cdc-apache-cassandra/releases/latest"^] for the latest version.
This installation requires the following. Latest version artifacts are available https://github.com/datastax/cdc-apache-cassandra/releases/latest[here]. Use image:https://img.shields.io/github/v/release/datastax/cdc-apache-cassandra?color=green&display_name=tag[link="https://github.com/datastax/cdc-apache-cassandra/releases/latest"] for the latest version.

* C* or DSE environment
** https://downloads.datastax.com/#enterprise[DSE 6.8.16+]
Expand All @@ -17,7 +17,7 @@ This installation requires the following. Latest version artifacts are available

== Installing and configuring

. Download the https://pulsar.apache.org/download/[Pulsar tarball] and set up a Pulsar cluster. This example uses Pulsar https://pulsar.apache.org/docs/en/standalone/[standalone mode], but you can also use our helpful https://github.com/datastax/pulsar-ansible[Ansible Scripts^].
. Download the https://pulsar.apache.org/download/[Pulsar tarball] and set up a Pulsar cluster. This example uses Pulsar https://pulsar.apache.org/docs/en/standalone/[standalone mode], but you can also use our helpful https://github.com/datastax/pulsar-ansible[Ansible Scripts].
+
[source,bash]
----
Expand All @@ -29,7 +29,7 @@ bin/pulsar standalone
====
We recommend using the latest CDC agent version (at least version `1.0.4`+) to support C* collection data types.
====
. Install C*/DSE with your preferred https://docs.datastax.com/en/install/6.8/install/installWhichOne.html[installation method^].
. Install C*/DSE with your preferred https://docs.datastax.com/en/install/6.8/install/installWhichOne.html[installation method].

. After installing C*/DSE, but before starting the C*/DSE service, set the `Cassandra-env.sh` configuration:
+
Expand Down Expand Up @@ -78,13 +78,13 @@ INFO [main] 2022-04-11 18:47:06,433 Agent.java:92 - CDC agent started

== Deploy Pulsar Connector

. Deploy the Pulsar Cassandra Source Connector (CSC) for each CDC-enabled C* table. In the below example, we're configuring the source connector with parameters passed in `--source-config`.
. Deploy the Pulsar Cassandra Source Connector (CSC) for each CDC-enabled C* table. In the below example, we're configuring the source connector with parameters passed in `--source-config`.
+
The `outputFormat` source configuration controls the format of messages on the data topic.
The `outputFormat` source configuration controls the format of messages on the data topic.
+
* `key-value-avro` - Default behavior. Key and value are encoded separately in AVRO format.
* `key-value-avro` - Default behavior. Key and value are encoded separately in AVRO format.
* `key-value-json` - Key and value are encoded *separately* in JSON format.
* `json` - Key and value are encoded *together* in a single JSON object. Key field is populated with a JSON string representing key fields.
* `json` - Key and value are encoded *together* in a single JSON object. Key field is populated with a JSON string representing key fields.
+
[tabs]
====
Expand Down Expand Up @@ -161,9 +161,9 @@ $ pulsar-admin source create \
--
====

. Verify the CSC connector is deployed.
. Verify the CSC connector is deployed.
+
[source, bash]
[source,bash]
----
pulsar-admin source list ["<csc_connector_name>"]
----
Expand Down
2 changes: 1 addition & 1 deletion docs/docs-src/core/modules/ROOT/pages/faqs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ We recommend active monitoring of the disk space of the C* nodes. If the Pulsar
When the disk space of the `cdc_raw` directory reaches your `cdc_total_space_in_mb` Cassandra setting (less than 4GB by default), writes to CDC-enabled tables will fail with a `CDCWriteException`.
The following warning message is included in Cassandra logs:

[source,language-bash]
[source,bash]
----
WARN [CoreThread-5] 2021-10-29 09:12:52,790 NoSpamLogger.java:98 - Rejecting Mutation containing CDC-enabled table. Free up space in /mnt/data/cdc_raw.
----
Expand Down
12 changes: 6 additions & 6 deletions docs/docs-src/core/modules/ROOT/pages/install.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The following files are available in the tar file:

. Extract the files from the tar with the following command:
+
[source,language-bash]
[source,bash]
----
tar xvf cassandra-source-agents-<version>.tar
----
Expand Down Expand Up @@ -75,7 +75,7 @@ Without the change agent to process the CDC data, the space used will grow until

Here is an example set of configurations for the `cassandra.yaml` file:

[source,language-bash]
[source,bash]
----
cdc_enabled: true
commitlog_sync_period_in_ms: 2000
Expand Down Expand Up @@ -107,7 +107,7 @@ The following files are available:

Extract the files from the tar with the following command:

[source,language-bash]
[source,bash]
----
tar xvf cassandra-source-connectors-<version>.tar
----
Expand Down Expand Up @@ -136,7 +136,7 @@ You have to specify the following parameters:

Here is an example:

[source,language-bash]
[source,bash]
----
pulsar-admin source create \
--name cassandra-source-1 \
Expand All @@ -159,7 +159,7 @@ pulsar-admin source create \
----

Then check your connector is in the running state with no errors:
[source,language-bash]
[source,bash]
----
pulsar-admin source status --name cassandra-source-1
----
Expand All @@ -175,7 +175,7 @@ For advanced configuration of the Cassandra driver in the {cdc_pulsar}, see xref

Once the change agent is installed and the connector is running, you can enable or disable CDC on table using the following commands:

[source,language-bash]
[source,bash]
----
CREATE TABLE foo (a int, b text, PRIMARY KEY(a)) WITH cdc=true;
ALTER TABLE foo WITH cdc=true;
Expand Down
2 changes: 1 addition & 1 deletion docs/docs-src/core/modules/ROOT/pages/monitor.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ The change agent is a JVM agent running in Apache Cassandra(R) nodes and provide

The {cdc_cass} framework reports stats for each connector. You can view the stats for a connector like this:

[source,language-bash]
[source,bash]
----
pulsar-admin source stats --name cassandra-source-1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
In your {cdc_pulsar} configuration file, you can directly pass settings to the DataStax Java driver by using the `datastax-java-driver` prefix.
For example:

[source,no-highlight]
[source,console]
----
datastax-java-driver.basic.request.consistency=ALL
----
Expand Down Expand Up @@ -48,7 +48,7 @@ For `datastax-java-driver.basic.contact-points`, you must provide the fully qual

By passing in the Java driver's setting, this option gives you more configuration flexibility because you can specify a different port for each host. For example:

[source,no-highlight]
[source,console]
----
datastax-java-driver.basic.contact-points = 127.0.0.1:9042, 127.0.0.2:9042
----
Expand Down
File renamed without changes.
File renamed without changes.
24 changes: 20 additions & 4 deletions docs/local-preview-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,20 @@ content:

antora:
extensions:
- '@antora/atlas-extension'
- '@antora/collector-extension'
- lib/tailwind-processor.js
- extensions/tailwind-processor.js
- id: unlisted-pages
enabled: true
require: lib/unlisted-pages-extension.js
require: extensions/unlisted-pages-extension.js
add_to_navigation: false
unlisted_pages_heading: Orphans

asciidoc:
extensions:
- '@asciidoctor/tabs'
- lib/remote-include-processor.js
- lib/svg-macro.js
- extensions/remote-include-processor.js
- extensions/svg-macro.js
- asciidoctor-kroki
- asciidoctor-external-callout
attributes:
Expand All @@ -55,8 +56,23 @@ asciidoc:
astra_db: 'Astra DB'
astra_stream: 'Astra Streaming'
astra_ui: 'Astra Portal'
astra_cli: 'Astra CLI'
astra-streaming-examples-repo: 'https://raw.githubusercontent.com/datastax/astra-streaming-examples/master'
luna-streaming-examples-repo: 'https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main'
support_url: 'https://support.datastax.com'
glossary-url: 'https://docs.datastax.com/en/glossary/docs/index.html#'
emoji-tada: "&#127881;"
emoji-rocket: "&#128640;"
emoji-smile: "&#128512"
dse: 'DataStax Enterprise (DSE)'
cassandra: 'Apache Cassandra(R)'
classic: 'classic'
classic_cap: 'Classic'
serverless: 'serverless'
serverless_cap: 'Serverless'
# Antora Atlas
primary-site-url: https://docs.datastax.com/en
primary-site-manifest-url: https://docs.datastax.com/en/site-manifest.json

urls:
latest_version_segment_strategy: redirect:from
Expand Down
8 changes: 5 additions & 3 deletions docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@
"url": "https://github.com/datastax/cdc-apache-cassandra"
},
"scripts": {
"build:local": "env FORCE_SHOW_EDIT_PAGE_LINK=true antora --clean --stacktrace local-preview-playbook.yml",
"tailwindcss": "tailwindcss build -c ./build/site/_/js/tailwind.config.js -i ./build/site/_/css/site.css -o ./build/site/_/css/site.css --minify"
"tailwindcss": "tailwindcss build -c ./build/site/_/js/tailwind.config.js -i ./build/site/_/css/site.css -o ./build/site/_/css/site.css --minify",
"build:local": "env FORCE_SHOW_EDIT_PAGE_LINK=true antora --clean --fetch --stacktrace local-preview-playbook.yml"
},
"dependencies": {
"@antora/atlas-extension": "^1.0.0-alpha.2",
"@antora/collector-extension": "^1.0.0-alpha.3",
"@asciidoctor/tabs": "^1.0.0-beta.6",
"antora": "~3.1",
"antora": "3.2.0-alpha.4",
"asciidoctor-external-callout": "~1.2.1",
"asciidoctor-kroki": "~0.18.1",
"csv-parser": "^3.0.0",
"lodash": "^4.17.21",
"npm-run-all": "^4.1.5",
"tailwindcss": "^3.3.5"
}
Expand Down
Loading