You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _migration-assistant/deploying-migration-assistant/configuration-options.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -118,7 +118,7 @@ Performing a live capture migration requires that a Capture Proxy be configured
118
118
|`trafficReplayerExtraArgs`|`"--sigv4-auth-header-service-region es,us-east-1 --speedup-factor 5"`| Extra arguments for the Traffic Replayer command, including options for auth headers and other parameters specified by the [Traffic Replayer](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java). |
119
119
|`targetClusterProxyServiceEnabled`|`true`| Enables the target cluster proxy service deployment using a CloudFormation stack. |
120
120
121
-
For arguments available in `captureProxyExtraArgs`, see the `@Parameter` fields in [`CaptureProxy.java`](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/CaptureProxy.java). For `trafficReplayerExtraArgs`, see the `@Parameter` fields in [TrafficReplayer.java](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java).
121
+
For arguments available in `captureProxyExtraArgs`, see the `@Parameter` fields in [`CaptureProxy.java`](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/CaptureProxy.java). For `trafficReplayerExtraArgs`, see the `@Parameter` fields in [`TrafficReplayer.java`](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java).
Copy file name to clipboardExpand all lines: _migration-assistant/deploying-migration-assistant/iam-and-security-groups-for-existing-clusters.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ For an OpenSearch Domain, two main configurations are typically required to ensu
25
25
- An open access policy that allows all access.
26
26
- Configured to allow at least the AWS Identity and Access Management (IAM) task roles for the applicable migration services (Traffic Replayer, Migration Console, `Reindex-from-Snapshot`) to access the domain.
27
27
28
-
### Managed service role mapping (Cross-managed-cluster migrations)**
28
+
### Managed service role mapping (Cross-managed-cluster migrations)
29
29
30
30
When migrating between two managed clusters, for example, when both domains were created using Amazon OpenSearch Service, provide Migration Assistant components with sufficient permissions to modify both the source and target clusters.
Copy file name to clipboardExpand all lines: _migration-assistant/migration-phases/backfill.md
+34-9Lines changed: 34 additions & 9 deletions
Original file line number
Diff line number
Diff line change
@@ -50,16 +50,15 @@ After you have routed the client based on your use case, test adding records aga
50
50
51
51
In the migration console, run the following command:
52
52
53
-
```bash
54
-
console kafka describe-topic-records
55
-
```
56
-
{% include copy.html %}
53
+
```bash
54
+
console kafka describe-topic-records
55
+
```
56
+
{% include copy.html %}
57
57
58
-
Note the records in the logging topic.
58
+
Note the records in the logging topic.
59
59
60
60
After a short period, execute the same command again and compare the increased number of records against the expected HTTP requests.
61
61
62
-
63
62
## Creating a snapshot
64
63
65
64
Create a snapshot for your backfill using the following command:
@@ -139,17 +138,43 @@ console backfill scale 5
139
138
140
139
We recommend slowly scaling up the fleet while monitoring the health metrics of the target cluster to avoid over-saturating it. [Amazon OpenSearch Service domains](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/monitoring.html) provide a number of metrics and logs that can provide this insight.
141
140
141
+
### Pausing the migration
142
+
143
+
To pause a migration, use the following command:
144
+
145
+
```shell
146
+
console backfill pause
147
+
```
148
+
149
+
This will stop all existing workers from running while leaving the backfill operation in a state from which it can be restarted. When you want to restart the migration, perform one of the following actions:
150
+
151
+
- Run `console backfill start`.
152
+
- Scale up the worker count by running `console backfill scale <worker_count>`.
153
+
142
154
### Stopping the migration
143
155
144
-
Backfill requires manually stopping the fleet. Once all the data has been migrated, you can shut down the fleet and all its workers using the following command:
145
-
Backfill requires manually stopping the fleet. Once all the data has been migrated, you can shut down the fleet and all its workers using the following command:
156
+
Completing the backfill process requires manually stopping the migration. Stopping the migration shuts down all workers and cleans up all metadata used to track and coordinate the migration. Once the status checks report that your data has been completely migrated, you can stop the migration with the following command:
157
+
146
158
```shell
147
159
console backfill stop
148
160
```
161
+
{% include copy.html %}
162
+
163
+
Migration Assistant should return the following response:
164
+
165
+
```shell
166
+
Backfill stopped successfully.
167
+
Service migration-aws-integ-reindex-from-snapshot set to 0 desired count. Currently 0 running and 5 pending.
168
+
Archiving the working state of the backfill operation...
169
+
RFS Workers are still running, waiting for them to complete...
170
+
Backfill working state archived to: /shared-logs-output/migration-console-default/backfill_working_state/working_state_backup_20241115174822.json
171
+
```
172
+
173
+
You cannot restart a stopped migration. Instead, you can pause the backfill process using `console backfill pause`.
149
174
150
175
### Amazon CloudWatch metrics and dashboard
151
176
152
-
Migration Assistant creates an Amazon CloudWatch dashboard that you can use to visualize the health and performance of the backfill process. It combines the metrics for the backfill workers and, for those migrating to Amazon OpenSearch Service, the target cluster.
177
+
Migration Assistant creates an Amazon CloudWatch dashboard, named `MigrationAssistant_ReindexFromSnapshot_Dashboard`, that you can use to visualize the health and performance of the backfill process. It combines the metrics for the backfill workers and, for those migrating to Amazon OpenSearch Service, the target cluster.
153
178
154
179
You can find the backfill dashboard in the CloudWatch console based on the AWS Region in which you have deployed Migration Assistant. The metric graphs for your target cluster will be blank until you select the OpenSearch domain you're migrating to from the dropdown menu at the top of the dashboard.
Copy file name to clipboardExpand all lines: _migration-assistant/migration-phases/live-traffic-migration/using-traffic-replayer.md
+10-3Lines changed: 10 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -284,9 +284,16 @@ The `readable-tuples.log` should appear similar to the following:
284
284
```
285
285
286
286
287
-
## Metrics
287
+
## Amazon CloudWatch metrics and dashboard
288
+
Migration Assistant creates an Amazon CloudWatch dashboard named `MigrationAssistant_ReindexFromSnapshot_Dashboard` to visualize the health and performance of the backfill process. This dashboard combines metrics for the backfill workers and migration to Amazon OpenSearch Service, providing insights into the performance and health of the Capture Proxy and Traffic Replayer components, including metrics such as:
288
289
289
-
Traffic Replayer emits various OpenTelemetry metrics to Amazon CloudWatch, and traces are sent through AWS X-Ray. The following are some useful metrics that can help evaluate cluster performance.
290
+
- The number of bytes read and written.
291
+
- The number of active connections.
292
+
- The replay speed multiplier.
293
+
294
+
You can find the Capture and Replay dashboard in the AWS Management Console for CloudWatch Dashboards in the AWS Region where you deployed Migration Assistant.
295
+
296
+
Traffic Replayer emits various OpenTelemetry metrics to Amazon CloudWatch, and traces are sent through AWS X-Ray. The following are some useful metrics that can help evaluate migration performance.
290
297
291
298
### `sourceStatusCode`
292
299
@@ -309,4 +316,4 @@ The following metrics are also reported:
309
316
310
317
## CloudWatch considerations
311
318
312
-
Metrics pushed to CloudWatch may experience a visibility lag of around 5 minutes. CloudWatch also retains higher-resolution data for a shorter period than lower-resolution data. For more information, see [Amazon CloudWatch concepts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html).
319
+
Metrics and dashboards pushed to CloudWatch may experience a visibility lag of around 5 minutes. CloudWatch also retains higher-resolution data for a shorter period than lower-resolution data. For more information, see [Amazon CloudWatch concepts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html).
Copy file name to clipboardExpand all lines: _migration-assistant/migration-phases/removing-migration-infrastructure.md
+12-5Lines changed: 12 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -7,15 +7,22 @@ parent: Migration phases
7
7
8
8
# Removing migration infrastructure
9
9
10
-
After a migration is complete all resources should be removed except for the target cluster, and optionally your Cloudwatch Logs, and Traffic Replayer logs.
10
+
After a migration is complete, you should remove all resources except for the target cluster and, optionally, your Amazon CloudWatch logs and Traffic Replayer logs.
11
11
12
-
To remove all the CDK stack(s) which get created during a deployment you can execute a command similar to below within the CDK directory
12
+
To remove the AWS Cloud Development Kit (AWS CDK) stack(s) created during a deployment, run the following command within the CDK directory:
13
13
14
-
```bash
14
+
```bash
15
+
cd deployment/cdk/opensearch-service-migration
15
16
cdk destroy "*" --c contextId=<CONTEXT_ID>
16
17
```
17
18
{% include copy.html %}
18
19
19
-
Follow the instructions on the command-line to remove the deployed resources from the AWS account.
20
+
Follow the instructions on the commandline to remove the deployed resources from your AWS account.
20
21
21
-
The AWS Management Console can also be used to remove Migration Assistant resources and confirm that they are no longer in the account.
22
+
You can also use the AWS Management Console to remove Migration Assistant resources and confirm that they are no longer present in the account.
23
+
24
+
## Uninstalling Migration Assistant for Amazon OpenSearch Service
25
+
26
+
You can uninstall Migration Assistant for Amazon OpenSearch Service from the AWS Management Console or by using the AWS Command Line Interface (AWS CLI). Manually remove the contents of the Amazon Simple Storage Service (Amazon S3) bucket that matches the syntax `cdk-<unique id>-assets-<account id>-<region>`, the bucket created by Migration Assistant. Migration Assistant for Amazon OpenSearch Service does not automatically delete Amazon S3 buckets.
27
+
28
+
To delete the stored data and the AWS CloudFormation stacks created by Migration Assistant, see [Uninstall the solution](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/uninstall-the-solution.html) in the Amazon OpenSearch Service documentation.
0 commit comments