Skip to content

Commit af49c44

Browse files
lfranckelfrancke
committed
Merge branch 'main' into feat/configurable-tests
# Conflicts: # tests/test-definition.yaml
2 parents 1bfee40 + ae6266d commit af49c44

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+2749
-22640
lines changed

‎.github/ISSUE_TEMPLATE/new_version.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,25 +7,25 @@ assignees: ''
77

88
---
99

10-
**Which new version of Apache HDFS should we support?**
10+
## Which new version of Apache HDFS should we support?
1111

1212
Please specify the version, version range or version numbers to support, please also add these to the issue title
1313

14-
**Additional information**
14+
## Additional information
1515

1616
If possible, provide a link to release notes/changelog
1717

18-
**Changes required**
18+
## Changes required
1919

2020
Are there any upstream changes that we need to support?
2121
e.g. new features, changed features, deprecated features etc.
2222

23+
## Implementation checklist
2324

24-
25-
**Implementation checklist**
26-
27-
Please don't change anything in this list.
28-
Not all of these steps are necessary for all versions.
25+
<!--
26+
Please don't change anything in this list.
27+
Not all of these steps are necessary for all versions.
28+
-->
2929

3030
- [ ] Update the Docker image
3131
- [ ] Update documentation to include supported version(s)

‎.github/PULL_REQUEST_TEMPLATE/pre-release-getting-started-script.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Check and Update Getting Started Script
2+
3+
<!--
4+
Make sure to update the link in 'issues/.github/ISSUE_TEMPLATE/pre-release-getting-started-scripts.md'
5+
when you rename this file.
6+
-->
7+
8+
<!--
9+
Replace 'TRACKING_ISSUE' with the applicable release tracking issue number.
10+
-->
11+
12+
Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>
13+
14+
> [!NOTE]
15+
> During a Stackable release we need to check (and optionally update) the
16+
> getting-started scripts to ensure they still work after product and operator
17+
> updates.
18+
19+
```shell
20+
# Some of the scripts are in a code/ subdirectory
21+
# pushd docs/modules/superset/examples/getting_started
22+
# pushd docs/modules/superset/examples/getting_started/code
23+
pushd $(fd -td getting_started | grep examples); cd code 2>/dev/null || true
24+
25+
# Make a fresh cluster (~12 seconds)
26+
kind delete cluster && kind create cluster
27+
./getting_started.sh stackablectl
28+
29+
# Make a fresh cluster (~12 seconds)
30+
kind delete cluster && kind create cluster
31+
./getting_started.sh helm
32+
33+
popd
34+
```

‎.github/PULL_REQUEST_TEMPLATE/pre-release-rust-deps.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Bump Rust Dependencies for Stackable Release XX.(X)X
2+
3+
<!--
4+
Make sure to update the link in 'issues/.github/ISSUE_TEMPLATE/pre-release-operator-rust-deps.md'
5+
when you rename this file.
6+
-->
7+
8+
<!--
9+
Replace 'TRACKING_ISSUE' with the applicable release tracking issue number.
10+
-->
11+
12+
Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>
13+
14+
> [!NOTE]
15+
> During a Stackable release we need to update various Rust dependencies before
16+
> entering the final release period to ensure we run the latest versions of
17+
> crates. These bumps also include previously updated and released crates from
18+
> the `operator-rs` repository.
19+
20+
```[tasklist]
21+
### Tasks
22+
- [ ] Bump Rust Dependencies, see below for more details.
23+
- [ ] Add changelog entry stating which important crates were bumped (including the version).
24+
```
25+
26+
> [!NOTE]
27+
> The bumping / updating of Rust dependencies is done in multiple steps:
28+
>
29+
> 1. Update the minimum Version in the root `Cargo.toml` manifest.
30+
> 2. Run the `cargo update` command, which also updates the `Cargo.lock` file.
31+
> 3. Lastly, run `make regenerate-nix` to update the `Cargo.nix` file.
32+
33+
```[tasklist]
34+
### Bump Rust Dependencies
35+
- [ ] Bump `stackable-operator` and friends.
36+
- [ ] Bump `product-version`.
37+
- [ ] Bump all other dependencies.
38+
```

‎.github/pull_request_template.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
# Author
1212
- [ ] Changes are OpenShift compatible
1313
- [ ] CRD changes approved
14-
- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
14+
- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
1515
- [ ] Helm chart can be installed and deployed operator works
1616
- [ ] Integration tests passed (for non trivial changes)
1717
- [ ] Changes need to be "offline" compatible
@@ -22,7 +22,7 @@
2222
- [ ] Code contains useful comments
2323
- [ ] Code contains useful logging statements
2424
- [ ] (Integration-)Test cases added
25-
- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
25+
- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
2626
- [ ] Changelog updated
2727
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)
2828
```

‎.github/workflows/build.yml

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ env:
2525
CARGO_TERM_COLOR: always
2626
CARGO_INCREMENTAL: '0'
2727
CARGO_PROFILE_DEV_DEBUG: '0'
28-
RUST_TOOLCHAIN_VERSION: "1.79.0"
28+
RUST_TOOLCHAIN_VERSION: "1.80.0"
2929
RUSTFLAGS: "-D warnings"
3030
RUSTDOCFLAGS: "-D warnings"
3131
RUST_LOG: "info"
@@ -57,7 +57,7 @@ jobs:
5757
key: udeps
5858
cache-all-crates: "true"
5959
- uses: stackabletech/cargo-install-action@cargo-udeps
60-
- run: cargo udeps --workspace
60+
- run: cargo udeps --workspace --all-targets
6161

6262
# This job evaluates the github environment to determine why this action is running and selects the appropriate
6363
# target repository for published Helm charts based on this.
@@ -368,7 +368,13 @@ jobs:
368368
make -e build
369369
- name: Publish Docker image and Helm chart
370370
if: ${{ !github.event.pull_request.head.repo.fork }}
371-
run: make -e publish
371+
run: |
372+
# We want to publish helmcharts only once as they have a common name, while still publishing both images with architecture specific tags
373+
if [ "$(uname -m)" = "x86_64" ]; then
374+
make -e publish
375+
else
376+
make -e docker-publish
377+
fi
372378
# Output the name of the published image to the Job output for later use
373379
- id: printtag
374380
name: Output image name and tag
@@ -424,7 +430,7 @@ jobs:
424430
steps:
425431
- name: Install preflight
426432
run: |
427-
wget https://github.com/redhat-openshift-ecosystem/openshift-preflight/releases/download/1.9.4/preflight-linux-amd64
433+
wget https://github.com/redhat-openshift-ecosystem/openshift-preflight/releases/download/1.10.0/preflight-linux-amd64
428434
chmod +x preflight-linux-amd64
429435
- name: Check container
430436
run: |

‎.github/workflows/pr_pre-commit.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
name: pre-commit
3+
4+
on:
5+
pull_request:
6+
7+
jobs:
8+
pre-commit:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
12+
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
13+
with:
14+
python-version: '3.12'
15+
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
16+
with:
17+
extra_args: "" # Disable --all-files until we have time to fix druid/stackable/bin/run-druid

‎.github/workflows/pr_reviewdog.yaml

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,17 +23,6 @@ jobs:
2323
with:
2424
github_token: ${{ secrets.GITHUB_TOKEN }}
2525

26-
flake8:
27-
runs-on: ubuntu-latest
28-
steps:
29-
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
30-
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # tag=v5.1.0
31-
with:
32-
python-version: "3.12"
33-
- uses: reviewdog/action-flake8@99c2cfecdbc9111ec223b85b08af0e13a9a098dc # v3.10.0
34-
with:
35-
github_token: ${{ secrets.GITHUB_TOKEN }}
36-
3726
hadolint:
3827
runs-on: ubuntu-latest
3928
steps:

‎.markdownlint.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,10 @@ MD013:
1414
# Number of characters for code blocks
1515
code_block_line_length: 9999
1616

17+
# MD033/no-inline-html
18+
MD033:
19+
allowed_elements: [h1, img, p]
20+
1721
# MD024/no-duplicate-heading/no-duplicate-header - Multiple headings with the same content
1822
MD024:
1923
# Only check sibling headings

‎.pre-commit-config.yaml

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
---
2+
exclude: ^(Cargo\.nix|crate-hashes\.json|nix/.*)$
3+
24
# See https://pre-commit.com for more information
35
# See https://pre-commit.com/hooks.html for more hooks
46
repos:
@@ -28,11 +30,25 @@ repos:
2830
rev: v0.40.0
2931
hooks:
3032
- id: markdownlint
33+
types: [text]
34+
files: \.md(\.j2)*$
35+
36+
- repo: https://github.com/koalaman/shellcheck-precommit
37+
rev: v0.10.0
38+
hooks:
39+
- id: shellcheck
40+
args: ["--severity=info"]
3141

32-
- repo: https://github.com/PyCQA/flake8
33-
rev: 7.0.0
42+
# WARNING (@NickLarsenNZ): Nix users need to install ruff first.
43+
# If you do not, you will need to delete the cached ruff binary shown in the
44+
# error message
45+
- repo: https://github.com/astral-sh/ruff-pre-commit
46+
rev: v0.5.1
3447
hooks:
35-
- id: flake8
48+
# Run the linter.
49+
- id: ruff
50+
# Run the formatter.
51+
- id: ruff-format
3652

3753
- repo: local
3854
hooks:

‎.readme/README.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 -->
12
{%- set title="Stackable Operator for Apache Hadoop" -%}
23
{%- set operator_name="hdfs" -%}
34
{%- set operator_docs_slug="hdfs" -%}

‎.readme/partials/borrowed/documentation.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 -->
12
## Documentation
23

34
The stable documentation for this operator can be found [here](https://docs.stackable.tech/home/stable/{{operator_docs_slug}}).

‎.readme/partials/borrowed/footer.md.j2

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 -->
12
## About The Stackable Data Platform
23

34
This operator is written and maintained by [Stackable](https://stackable.tech) and it is part of a larger data platform.
@@ -24,7 +25,6 @@ We develop and test our operators on the following cloud platforms:
2425
* Kubernetes (for an up to date list of supported versions please check the release notes in our [docs](https://docs.stackable.tech))
2526
* Red Hat OpenShift
2627

27-
2828
## Other Operators
2929

3030
These are the operators that are currently part of the Stackable Data Platform:

‎.readme/partials/borrowed/header.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 -->
12
<p align="center">
23
<img width="150" src="./.readme/static/borrowed/Icon_Stackable.svg" alt="Stackable Logo"/>
34
</p>

‎.readme/partials/borrowed/links.md.j2

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
1+
<!-- markdownlint-disable MD041 -->
22
{% if no_jenkins_job_badge %}{% else %}![Build Actions Status](https://ci.stackable.tech/buildStatus/icon?job={{operator_name}}%2doperator%2dit%2dnightly&subject=Integration%20Tests){% endif %}
33
[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/stackabletech/{{operator_name}}-operator/graphs/commit-activity)
44
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-green.svg)](https://docs.stackable.tech/home/stable/contributor/index.html)

‎.readme/partials/borrowed/overview_blurb.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1+
<!-- markdownlint-disable MD041 MD051 -->
12
It is part of the Stackable Data Platform, a curated selection of the best open source data apps like Apache Kafka, Apache Druid, Trino or Apache Spark, [all](#other-operators) working together seamlessly. Based on Kubernetes, it runs everywhere – [on prem or in the cloud](#supported-platforms).

‎.readme/partials/borrowed/related_reading.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 MD022 MD032 -->
12
{%- if related_reading_links -%}
23
## Related Reading
34
{% for (text, link) in related_reading_links %}

‎.readme/partials/main.md.j2

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD041 -->
12
This is a Kubernetes operator to manage [Apache Hadoop](https://hadoop.apache.org/).
23

34
{% filter trim %}

‎CHANGELOG.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,19 @@ All notable changes to this project will be documented in this file.
44

55
## [Unreleased]
66

7+
### Changed
8+
9+
- Reduce CRD size from `1.4MB` to `136KB` by accepting arbitrary YAML input instead of the underlying schema for the following fields ([#574]):
10+
- `podOverrides`
11+
- `affinity`
12+
13+
[#574]: https://github.com/stackabletech/hdfs-operator/pull/574
14+
15+
## [24.7.0] - 2024-07-24
16+
717
### Added
818

9-
- Add support for version `3.4.0` ([#545]).
19+
- Add experimental support for version `3.4.0` ([#545], [#557]). We do NOT support upgrading from 3.3 to 3.4 yet!
1020

1121
### Changed
1222

@@ -29,6 +39,7 @@ All notable changes to this project will be documented in this file.
2939
[#545]: https://github.com/stackabletech/hdfs-operator/pull/545
3040
[#546]: https://github.com/stackabletech/hdfs-operator/pull/546
3141
[#549]: https://github.com/stackabletech/hdfs-operator/pull/549
42+
[#557]: https://github.com/stackabletech/hdfs-operator/pull/557
3243

3344
## [24.3.0] - 2024-03-20
3445

0 commit comments

Comments
 (0)