Skip to content

Upgrading

Jump to bottom Edit New page
Blake Niemyjski edited this page Jan 31, 2017 · 17 revisions

Please ensure that you have created backups before upgrading!

If you are upgrading from v1 or v2 you will need to upgrade to v3.0 before upgrading to the latest release.

After you've followed the steps below to upgrade to the v4.0.0 release. You can the follow setup section to finish upgrading to the latest version of Exceptionless.

Upgrading from v3 to v4

This process requires you to setup and configure a new Elasticsearch 5 instance and reindex your existing Elasticsearch 1.x data into the Elasticsearch 5 instance using external reindexing. This tutorial assumes you have Elasticsearch 5 Kibana instance installed and configured.

  1. Edit the Elasticsearch 5's elasticsearch.yml configuration file with the value that contains the IP address or hostname of the 1.x server (reindex.remote.whitelist: 10.0.0.9:9200) and restart the elasticsearch service. This allows you to reindex the 1.x data into your new 5.x instance.

  2. Open Kibana (E.G., (http://localhost:5601[http://localhost:5601]) and execute the following scripts to reindex your data.

    1. Create Organization Index with the correct mappings:

PUT organizations-v1 { "settings": { "index.number_of_replicas": 1, "index.number_of_shards": 3 "analysis": { "analyzer": { "keyword_lowercase": { "type": "custom", "filter": [ "lowercase" ], "tokenizer": "keyword" } } } }, "aliases": { "organizations": {} }, "mappings": { "organization": { "dynamic": false, "properties": { "id": { "type": "keyword" }, "created_utc": { "type": "date" }, "updated_utc": { "type": "date" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "stripe_customer_id": { "type": "keyword" }, "has_premium_features": { "type": "boolean" }, "plan_id": { "type": "keyword" }, "plan_name": { "type": "keyword", "ignore_above": 256 }, "subscribe_date": { "type": "date" }, "billing_status": { "type": "float" }, "billing_price": { "type": "double" }, "is_suspended": { "type": "boolean" }, "retention_days": { "type": "integer" }, "invites": { "type": "object", "properties": { "token": { "type": "keyword" }, "email_address": { "type": "text", "analyzer": "keyword_lowercase" } } }, "usage": { "type": "object", "properties": { "date": { "type": "date" }, "total": { "type": "float" }, "blocked": { "type": "float" }, "limit": { "type": "float" }, "too_big": { "type": "float" } } }, "overage_hours": { "type": "object", "properties": { "date": { "type": "date" }, "total": { "type": "float" }, "blocked": { "type": "float" }, "limit": { "type": "float" }, "too_big": { "type": "float" } } } } }, "project": { "dynamic": false, "properties": { "id": { "type": "keyword" }, "created_utc": { "type": "date" }, "updated_utc": { "type": "date" }, "organization_id": { "type": "keyword" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "next_summary_end_of_day_ticks": { "type": "long" } } }, "token": { "dynamic": false, "properties": { "id": { "type": "keyword" }, "created_utc": { "type": "date" }, "updated_utc": { "type": "date" }, "expires_utc": { "type": "date" }, "organization_id": { "type": "keyword" }, "project_id": { "type": "keyword" }, "default_project_id": { "type": "keyword" }, "user_id": { "type": "keyword" }, "refresh": { "type": "keyword" }, "scopes": { "type": "keyword" }, "type": { "type": "byte" } } }, "user": { "dynamic": false, "properties": { "id": { "type": "keyword" }, "created_utc": { "type": "date" }, "updated_utc": { "type": "date" }, "organization_ids": { "type": "keyword" }, "full_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "email_address": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "keyword_lowercase" }, "verify_email_address_token": { "type": "keyword" }, "password_reset_token": { "type": "keyword" }, "roles": { "type": "keyword" }, "o_auth_accounts": { "type": "object", "properties": { "provider": { "type": "keyword" }, "provider_user_id": { "type": "keyword" }, "username": { "type": "keyword" } } } } }, "webhook": { "dynamic": false, "properties": { "id": { "type": "keyword" }, "created_utc": { "type": "date" }, "organization_id": { "type": "keyword" }, "project_id": { "type": "keyword" }, "url": { "type": "keyword" }, "event_types": { "type": "keyword" } } } } }

2. Reindex Organization data from 1.x into 5.x **Please make sure you update the host name**:

POST _reindex { "source": { "remote": { "host": "http://10.0.0.9:9200" }, "index": "organizations-v1" }, "dest": { "index": "organizations-v1", "op_type": "create" }, "script": { "inline": "if (ctx._source.modified_utc != null) { ctx._source.updated_utc = ctx._source.remove('modified_utc'); }", "lang": "painless" } }

3. Create Stack Index with the correct mappings:

PUT stacks-v1 { "settings": { "index.number_of_replicas": 1, "index.number_of_shards": 3 }, "aliases": { "stacks": {} }, "mappings": { "stacks": { "include_in_all": false, "dynamic": false, "properties": { "id": { "type": "keyword" }, "organization_id": { "type": "keyword" }, "project_id": { "type": "keyword" }, "signature_hash": { "type": "keyword" }, "type": { "type": "keyword" }, "first_occurrence": { "type": "date" }, "last_occurrence": { "type": "date" }, "title": { "type": "text", "boost": 1.1, "include_in_all": true }, "description": { "type": "text", "include_in_all": true }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "boost": 1.2, "include_in_all": true }, "references": { "type": "text", "include_in_all": true }, "date_fixed": { "type": "date" }, "fixed": { "type": "boolean" }, "fixed_in_version": { "type": "keyword" }, "is_hidden": { "type": "boolean" }, "is_regressed": { "type": "boolean" }, "occurrences_are_critical": { "type": "boolean" }, "total_occurrences": { "type": "integer" } } } } }

4. Reindex Stack data from 1.x into 5.x **Please make sure you update the host name**:

POST _reindex { "source": { "remote": { "host": "http://10.0.0.9:9200" }, "index": "stacks-v1" }, "dest": { "index": "stacks-v1", "op_type": "create" } }

5. Create Event Template so daily indexes can be created with the correct mappings:

PUT _template/events-v1 { "template": "events-v1-", "settings": { "number_of_shards": 1, "number_of_replicas": 1, "analysis": { "analyzer": { "comma_whitespace": { "type": "pattern", "pattern": "[,\s]+" }, "email": { "type": "custom", "filter": [ "email", "lowercase", "unique" ], "tokenizer": "keyword" }, "version_index": { "type": "custom", "filter": [ "version_pad1", "version_pad2", "version_pad3", "version_pad4", "version", "lowercase", "unique" ], "tokenizer": "whitespace" }, "version_search": { "type": "custom", "filter": [ "version_pad1", "version_pad2", "version_pad3", "version_pad4", "lowercase" ], "tokenizer": "whitespace" }, "whitespace_lower": { "type": "custom", "filter": [ "lowercase" ], "tokenizer": "whitespace" }, "typename": { "type": "custom", "filter": [ "typename", "lowercase", "unique" ], "tokenizer": "typename_hierarchy" }, "standardplus": { "type": "custom", "filter": [ "standard", "typename", "lowercase", "stop", "unique" ], "tokenizer": "comma_whitespace" } }, "filter": { "email": { "type": "pattern_capture", "patterns": [ "(\w+)", "(\p{L}+)", "(\d+)", "(.+)@", "@(.+)" ] }, "typename": { "type": "pattern_capture", "patterns": [ "\.(\w+)", "([^\()]+)" ] }, "version": { "type": "pattern_capture", "patterns": [ "^(\d+)\.", "^(\d+\.\d+)", "^(\d+\.\d+\.\d+)" ] }, "version_pad1": { "type": "pattern_replace", "pattern": "(\.|^)(\d{1})(?=\.|-|$)", "replacement": "$10000$2" }, "version_pad2": { "type": "pattern_replace", "pattern": "(\.|^)(\d{2})(?=\.|-|$)", "replacement": "$1000$2" }, "version_pad3": { "type": "pattern_replace", "pattern": "(\.|^)(\d{3})(?=\.|-|$)", "replacement": "$100$2" }, "version_pad4": { "type": "pattern_replace", "pattern": "(\.|^)(\d{4})(?=\.|-|$)", "replacement": "$10$2" } }, "tokenizer": { "comma_whitespace": { "type": "pattern", "pattern": "[,\s]+" }, "typename_hierarchy": { "type": "path_hierarchy", "delimiter": "." } } } }, "mappings": { "events": { "dynamic": "false", "include_in_all": false, "_all": { "analyzer": "standardplus", "search_analyzer": "whitespace_lower" }, "_size": { "enabled": true }, "dynamic_templates": [ { "idx_reference": { "match": "-r", "mapping": { "ignore_above": 256, "type": "keyword" } } } ], "properties": { "count": { "type": "integer" }, "created_utc": { "type": "date" }, "data": { "properties": { "@environment": { "properties": { "architecture": { "type": "keyword" }, "ip_address": { "type": "text", "index": false, "copy_to": [ "ip" ], "include_in_all": true }, "machine_name": { "type": "text", "boost": 1.1, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "include_in_all": true }, "o_s_name": { "type": "text", "copy_to": [ "os" ] } } }, "@error": { "properties": { "data": { "properties": { "@target": { "properties": { "ExceptionType": { "type": "text", "index": false, "copy_to": [ "error.targettype" ], "include_in_all": true }, "Method": { "type": "text", "index": false, "copy_to": [ "error.targetmethod" ], "include_in_all": true } } } } } } }, "@level": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "@location": { "properties": { "country": { "type": "keyword" }, "level1": { "type": "keyword" }, "level2": { "type": "keyword" }, "locality": { "type": "keyword" } } }, "@request": { "properties": { "client_ip_address": { "type": "text", "index": false, "copy_to": [ "ip" ], "include_in_all": true }, "data": { "properties": { "@browser": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "@browser_major_version": { "type": "text" }, "@browser_version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "@device": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "@is_bot": { "type": "boolean" }, "@os": { "type": "text", "index": false, "copy_to": [ "os" ] }, "@os_major_version": { "type": "text" }, "@os_version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "path": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "include_in_all": true }, "user_agent": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "@simple_error": { "properties": { "data": { "properties": { "@target": { "properties": { "ExceptionType": { "type": "text", "index": false, "copy_to": [ "error.targettype" ], "include_in_all": true } } } } } } }, "@submission_method": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "@user": { "properties": { "identity": { "type": "text", "boost": 1.1, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "email", "search_analyzer": "whitespace_lower", "include_in_all": true }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "include_in_all": true } } }, "@user_description": { "properties": { "description": { "type": "text", "include_in_all": true }, "email_address": { "type": "text", "boost": 1.1, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "email", "search_analyzer": "simple", "include_in_all": true } } }, "@version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "version_index", "search_analyzer": "version_search" } } }, "date": { "type": "date" }, "error": { "include_in_all": true, "properties": { "code": { "type": "keyword", "boost": 1.1 }, "message": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "targetmethod": { "type": "text", "boost": 1.2, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "typename", "search_analyzer": "whitespace_lower" }, "targettype": { "type": "text", "boost": 1.2, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "typename", "search_analyzer": "whitespace_lower" }, "type": { "type": "text", "boost": 1.1, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "typename", "search_analyzer": "whitespace_lower" } } }, "geo": { "type": "geo_point" }, "id": { "type": "keyword", "include_in_all": true }, "idx": { "type": "object", "dynamic": "true" }, "ip": { "type": "text", "analyzer": "comma_whitespace" }, "is_deleted": { "type": "boolean" }, "is_first_occurrence": { "type": "boolean" }, "is_fixed": { "type": "boolean" }, "is_hidden": { "type": "boolean" }, "message": { "type": "text", "include_in_all": true }, "organization_id": { "type": "keyword" }, "os": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "project_id": { "type": "keyword" }, "reference_id": { "type": "keyword" }, "source": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "include_in_all": true }, "stack_id": { "type": "keyword" }, "tags": { "type": "text", "boost": 1.2, "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "include_in_all": true }, "type": { "type": "keyword" }, "updated_utc": { "type": "date" }, "value": { "type": "double" } } } } }

6. Reindex Event data from 1.x into 5.x **Please make sure you update the host name**:

POST _reindex { "source": { "remote": { "host": "http://10.0.0.9:9200" }, "index": "events-v1-*", "size": 200 }, "dest": { "index": "events-v1-error" }, "script": { "lang": "painless", "inline": "ctx._index = 'events-v1-' + DateTimeFormatter.ofPattern('yyyy.MM.dd').format(OffsetDateTime.parse(ctx._source.date).toInstant().atZone(ZoneOffset.UTC)); if (ctx._source.updated_utc == null) { ctx._source.updated_utc = ctx._source.created_utc; } if (ctx._source.is_deleted == null) { ctx._source.is_deleted = false; } if (!ctx.containsKey('data') || !(ctx.data.containsKey('@error') || ctx.data.containsKey('@simple_error'))) return null;def types = [];def messages = [];def codes = [];def err = ctx.data.containsKey('@error') ? ctx.data['@error'] : ctx.data['@simple_error'];def curr = err;while (curr != null) { if (curr.containsKey('type')) types.add(curr.type); if (curr.containsKey('message')) messages.add(curr.message); if (curr.containsKey('code')) codes.add(curr.code); curr = curr.inner;}if (ctx.error == null) ctx.error = new HashMap();ctx.error.type = types;ctx.error.message = messages;ctx.error.code = codes;" } }

7. Delete the previous Event Template:

DELETE _template/events-v1


## Upgrading from v2 to v3
_Please note that upgrading from [v2](https://github.com/exceptionless/Exceptionless/releases/tag/v2.0.0) to [v3](https://github.com/exceptionless/Exceptionless/releases/tag/v3.0.0) requires that `Redis` is installed and configured._

1. Download and extract the [v3](https://github.com/exceptionless/Exceptionless/releases/tag/v3.0.0) release to a temp folder.
2. Update the connection strings in the `App_Data\JobRunner\Job.exe.config` config file.
1. You'll also need to add a `Migration:MongoConnectionString` connection string for the migration jobs to run.

    ```xml
    <add name="Migration:MongoConnectionString" connectionString="mongodb://localhost/exceptionless" />
    ```
3. Open the terminal and run the following jobs to migrate data from previous major versions of Exceptionless. `Jobs.exe` can be found in the `\wwwroot\App_Data\JobRunner\` folder.

Job.exe -t "Exceptionless.EventMigration.OrganizationMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"


## Upgrading from v1 to v3
_Please note that upgrading from v1 to [v3](https://github.com/exceptionless/Exceptionless/releases/tag/v3.0.0) requires that `Redis` is installed and configured._

1. Download and extract the [v3](https://github.com/exceptionless/Exceptionless/releases/tag/v3.0.0) release to a temp folder.
2. Update the connection strings in the `App_Data\JobRunner\Job.exe.config` config file.
   1. You'll also need to add a `Migration:MongoConnectionString` connection string for the migration jobs to run.

       ```xml
       <add name="Migration:MongoConnectionString" connectionString="mongodb://localhost/exceptionless" />
       ```
3. Open the terminal and run the following jobs to migrate data from previous major versions of Exceptionless. `Jobs.exe` can be found in the `\wwwroot\App_Data\JobRunner\` folder.

Job.exe -t "Exceptionless.EventMigration.StackMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core" Job.exe -t "Exceptionless.EventMigration.QueueEventMigrationsJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core" Job.exe -t "Exceptionless.EventMigration.EventMigrationJob, Exceptionless.EventMigration" -c -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core" Job.exe -t "Exceptionless.EventMigration.OrganizationMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"

Documentation has been moved to https://exceptionless.com/docs/

Exceptionless Documentation

Individual Client Docs

Clone this wiki locally