Skip to content

Commit 81e0092

Browse files
vasil-cfvasil-cf
authored
fix: 2.8 docs mongo (#64)
1 parent 4604be7 commit 81e0092

File tree

11 files changed

+1299
-1493
lines changed

11 files changed

+1299
-1493
lines changed

‎codefresh/README.md

Lines changed: 108 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
2424
- [Configuration with ALB (Application Load Balancer)](#configuration-with-alb-application-load-balancer)
2525
- [Configuration with Private Registry](#configuration-with-private-registry)
2626
- [Configuration with multi-role CF-API](#configuration-with-multi-role-cf-api)
27-
- [Indexes in MongoDB](#indexes-in-mongodb)
2827
- [High Availability](#high-availability)
2928
- [Mounting private CA certs](#mounting-private-ca-certs)
3029
- [Installing on OpenShift](#installing-on-openshift)
@@ -36,6 +35,7 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
3635
- [X-Frame-Options response header](#x-frame-options-response-header)
3736
- [Image digests in containers](#image-digests-in-containers)
3837
- [Configuring OIDC Provider](#configuring-oidc-provider)
38+
- [Maintaining MongoDB indexes](#maintaining-mongodb-indexes)
3939
- [Upgrading](#upgrading)
4040
- [To 2.0.0](#to-2-0-0)
4141
- [To 2.0.12](#to-2-0-12)
@@ -80,8 +80,6 @@ See [Use OCI-based registries](https://helm.sh/docs/topics/registries/)
8080

8181
**Important:** only helm 3.8.0+ is supported
8282

83-
**Important:** Read about [Indexes in MongoDB](#indexes-in-mongodb) before installation
84-
8583
Edit default `values.yaml` or create empty `cf-values.yaml`
8684

8785
- Pass `sa.json` (as a single line) to `.Values.imageCredentials.password`
@@ -94,7 +92,7 @@ imageCredentials:
9492
password: '{ "type": "service_account", "project_id": "codefresh-enterprise", "private_key_id": ... }'
9593
```
9694
97-
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl` and `.Values.global.firebaseSecret`
95+
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl`, `.Values.global.firebaseSecret`, `.Values.global.env.MONGOOSE_AUTO_INDEX`, `.Values.global.env.MONGO_AUTOMATIC_INDEX_CREATION`
9896

9997
```yaml
10098
global:
@@ -119,9 +117,9 @@ global:
119117
# name: my-secret
120118
# key: firebase-secret
121119
122-
# -- Enable auto-index creation in MongoDB
120+
# -- Enable index creation in MongoDB
123121
# This is required for first-time installations!
124-
# For upgrades, you should set it to `false`!
122+
# Before usage in Production, you must set it to `false` or remove it!
125123
env:
126124
MONGOOSE_AUTO_INDEX: "true"
127125
MONGO_AUTOMATIC_INDEX_CREATION: "true"
@@ -181,6 +179,17 @@ ingress-nginx:
181179
--timeout 15m
182180
```
183181

182+
### ⚠️ **MANDATORY** Post-Installation Action Items
183+
184+
Once your Codefresh On-Prem instance is installed, configured, and confirmed to be ready for production use, the following variables must be set to `false` or removed:
185+
186+
```yaml
187+
global:
188+
env:
189+
MONGOOSE_AUTO_INDEX: "false"
190+
MONGO_AUTOMATIC_INDEX_CREATION: "false"
191+
```
192+
184193
## Chart Configuration
185194

186195
See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments, visit the chart's [values.yaml](./values.yaml), or run these configuration commands:
@@ -797,64 +806,6 @@ cfapi-test-reporting:
797806
enabled: true
798807
```
799808
800-
⚠️ ⚠️ ⚠️
801-
### Indexes in MongoDB
802-
⚠️ ⚠️ ⚠️
803-
804-
Indexes in MongoDB are essential for efficient query performance, especially as your data grows. Without proper indexes, MongoDB must perform full collection scans to find matching documents, which can significantly slow down operations and increase resource usage. For production environments, ensuring that all frequently queried fields are indexed is vital to maintain optimal performance and scalability.
805-
806-
Auto-index creation in MongoDB is disabled by default in Codefresh On-Prem to prevent unexpected performance issues in production environments during upgrades. When enabled, MongoDB will automatically create indexes for fields used in queries, which can lead to high CPU and disk usage, increased I/O, and potential service disruptions—especially on large datasets. By requiring manual index management, administrators can plan index creation during maintenance windows, ensuring system stability and predictable performance before upgrading Codefresh On-Prem.
807-
808-
It is critical to ensure that your MongoDB indexes are always aligned with the latest recommended state for your Codefresh On-Prem version. Outdated or missing indexes can lead to degraded performance, slow queries, and increased resource consumption. Always review release notes and update or create indexes as specified during upgrades or when new collections/fields are introduced. Regularly auditing and maintaining your indexes helps ensure optimal system reliability and scalability.
809-
810-
The indexes list is located at the [codefresh-io/codefresh-onprem-helm](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes) repository.
811-
The indexes are stored in JSON files with keys and options specified.
812-
813-
The directory structure is:
814-
815-
```console
816-
codefresh-onprem-helm
817-
├── indexes
818-
│ ├── <DB_NAME> # MongoDB database name
819-
│ │ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
820-
```
821-
822-
#### Enabling auto-index creation
823-
824-
For first-time installations, you **must** enable auto-index creation by setting the following values:
825-
826-
```yaml
827-
global:
828-
env:
829-
MONGOOSE_AUTO_INDEX: "true"
830-
MONGO_AUTOMATIC_INDEX_CREATION: "true"
831-
```
832-
833-
You **should** disable it for the next upgrades by setting these variables to `false`:
834-
835-
```yaml
836-
global:
837-
env:
838-
MONGOOSE_AUTO_INDEX: "false"
839-
MONGO_AUTOMATIC_INDEX_CREATION: "false"
840-
```
841-
842-
#### Creating Indexes manually
843-
844-
> **Note!** If you have a large amount of MongoDB data, it is recommended to create indexes manually. Enabling auto-index creation can cause performance degradation during the index creation process with large datasets.
845-
846-
Depending on your MongoDB service (e.g., Atlas, self-hosted), you can create indexes using the MongoDB shell or the Atlas UI.
847-
848-
Ref:
849-
- [Create an Index in Atlas DB](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/#create-an-index)
850-
- [Create an Index with mongosh](https://www.mongodb.com/docs/manual/reference/method/db.collection.createIndex/)
851-
852-
##### ⚠️ Upgrading when the indexes are maintained manually
853-
854-
If you maintain indexes manually and you upgrade your On-Prem installation you must create all indexes which were introduced from your current version up to the version you upgrade to manually.
855-
856-
You can follow [Upgrading section](#upgrading) to see what changes were made for indexes in every specific release.
857-
858809
### High Availability
859810
860811
The chart installs the non-HA version of Codefresh by default. If you want to run Codefresh in HA mode, use the example values below.
@@ -1504,6 +1455,87 @@ To see all the claims supported by Codefresh OIDC provider, see `claims_supporte
15041455

15051456
Use [obtain-oidc-id-token](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/obtain-oidc-id-token/step.yaml#L27-L58) and [aws-sts-assume-role-with-web-identity](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/aws-sts-assume-role-with-web-identity/step.yaml#L29-L63) steps to exchange the OIDC token (JWT) for a cloud access token.
15061457

1458+
## Maintaining MongoDB Indexes
1459+
1460+
Sometimes, in new releases of Codefresh On-Prem, index requirements change. When this happens, it's mentioned in the [Upgrading section](#upgrading) for the specific release.
1461+
1462+
> ℹ️ If you're upgrading from version `X` to version `Y`, and index requirements were updated in any of the intermediate versions, you only need to align your indexes with the index requirements of version `Y`. To do that, follow [Index alignment](#index-alignment) instructions.
1463+
1464+
### Index alignment
1465+
1466+
The required index definitions for each release can be found at the following resources:
1467+
1468+
- `2.6` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.6/indexes>
1469+
- `2.7` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.7/indexes>
1470+
- `2.8` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes>
1471+
1472+
The indexes are stored in JSON files with keys and options specified.
1473+
1474+
The directory structure is:
1475+
1476+
```console
1477+
indexes
1478+
├── <DB_NAME> # MongoDB database name
1479+
│ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
1480+
```
1481+
1482+
**Overview of the index alignment process:**
1483+
1484+
1. Identify the differences between the indexes in your MongoDB instance and the required index definitions.
1485+
2. Create any missing indexes one by one. (It's important not to create them in bulk.)
1486+
3. Perform the upgrade of Codefresh On-Prem installation.
1487+
4. Then remove any unnecessary indexes.
1488+
1489+
> ⚠️ **Note! Any changes to indexes should be performed during a defined maintenance window or during periods of lowest traffic to MongoDB.**
1490+
>
1491+
> Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds. ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/index-creation/#index-build-impact-on-database-performance))
1492+
>
1493+
> Even minor changes to indexes (e.g., index removal) can cause brief but noticeable performance degradation ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/query-plans/#plan-cache-flushes))
1494+
1495+
#### Self-hosted MongoDB
1496+
1497+
For self-hosted MongoDB, follow the instructions below:
1498+
1499+
- Connect to the MongoDB server using the [mongosh](https://www.mongodb.com/docs/mongodb-shell/install/) shell. Open your terminal or command prompt and run the following command, replacing `<connection_string>` with the appropriate MongoDB connection string for your server:
1500+
1501+
```shell
1502+
mongosh "<connection_string>"
1503+
```
1504+
1505+
- Retrieve the list of indexes for a specific collection:
1506+
1507+
```js
1508+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').getIndexes()
1509+
```
1510+
1511+
- Compare your indexes with the required indexes for the target release, and adjust them by creating any missing indexes or removing any unnecessary ones
1512+
1513+
**Index creation**
1514+
1515+
> ⚠ **Note! Always create indexes sequentially, one by one. Don't create them in bulk.**
1516+
1517+
- To create an index, use the `createIndex()` method:
1518+
1519+
```js
1520+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').createIndex(<keys_object>, <options_object>)
1521+
```
1522+
1523+
After executing the `createIndex()` command, you should see a result indicating that the index was created successfully.
1524+
1525+
**Index removal**
1526+
1527+
- To remove an index, use the `dropIndex()` method with `<index_name>`:
1528+
1529+
```js
1530+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').dropIndex('<index_name>')
1531+
```
1532+
1533+
#### Atlas Database
1534+
1535+
If you're hosting MongoDB on [Atlas](https://www.mongodb.com/atlas/database), use the following [Manage Indexes](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/) guide to View, Create or Remove indexes.
1536+
1537+
> ⚠️ **Important!** In Atlas, for production environments, it is recommended to use rolling index builds by enabling the "Build index via rolling process" checkbox. ([*MongoDB official documentation*](https://www.mongodb.com/docs/v6.0/tutorial/build-indexes-on-replica-sets/))
1538+
15071539
## Upgrading
15081540

15091541
### To 2-0-0
@@ -2039,24 +2071,22 @@ cfapi:
20392071

20402072
### To 2-6-0
20412073

2074+
> ⚠️ **WARNING! MongoDB indexes changed!**
2075+
>
2076+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2077+
20422078
### [What's new in 2.6.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-26)
20432079

20442080
#### Affected values
20452081

20462082
[Image digests in containers](#image-digests-in-containers)
20472083

2048-
#### Auto-index creation in MongoDB
2049-
2050-
[Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
2051-
2052-
#### ⚠️ New indexes in MongoDB
2053-
2054-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following index **before** the upgrade:
2055-
2056-
- [Database: `read-models`, collection: `images-binaries`, index: `accountId_1_imageName_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/read-models/images-binaries.json#L75-L94)
2057-
20582084
### To 2-7-0
20592085

2086+
> ⚠️ **WARNING! MongoDB indexes changed!**
2087+
>
2088+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2089+
20602090
### [What's new in 2.7.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-27)
20612091

20622092
#### Affected values
@@ -2086,17 +2116,12 @@ global:
20862116
- "value"
20872117
```
20882118

2089-
#### ⚠️ New indexes in MongoDB
2090-
2091-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following indexes **before or right after** the upgrade:
2092-
2093-
> ⚠ **Note!** In case if you create indexes **before** the upgrade, please, create `codefresh.feature-store-versioned` collection manually in advance.
2094-
2095-
- [Database: `codefresh`, collection: `feature-store-versioned`, index: `createdAt_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/feature-store-versioned.json#L2-L9)
2096-
- [Database: `codefresh`, collection: `feature-store-versioned`, index: `LDRedisStoreVersion_1__id_-1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/feature-store-versioned.json#L10-L17)
2097-
20982119
### To 2-8-0
20992120

2121+
> ⚠️ **WARNING! MongoDB indexes changed!**
2122+
>
2123+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2124+
21002125
### [What's new in 2.8.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-28)
21012126

21022127
### ⚠️ ⚠️ ⚠️ Breaking changes. Read before upgrading!
@@ -2135,13 +2160,6 @@ mongodb:
21352160
enabled: false
21362161
```
21372162

2138-
#### ⚠️ New indexes in MongoDB
2139-
2140-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following indexes **before** the upgrade:
2141-
2142-
- [Database: `codefresh`, collection: `users`, index: `account_1__id_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/users.json#L2-L9)
2143-
- [Database: `codefresh`, collection: `users`, index: `role_1_account_1__id_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/users.json#L10-L17)
2144-
21452163
### PostgreSQL update
21462164

21472165
Default PostgreSQL image is changed from 13.x to 17.x

0 commit comments

Comments
 (0)