What's new in 3.1 (#4116)

Author: andreyaksenov

doc/release/3.1.0.rst

@@ -0,0 +1,307 @@
+Tarantool 3.1
+=============
+
+Planned release date: April 2024
+
+Releases on GitHub: not released yet
+
+The 3.1 release of Tarantool continues the development of a new cluster configuration approach introduced in the :ref:`3.0 version <3-0-new_declarative_configuration>` and adds the following main product features and improvements for the Community and Enterprise editions:
+
+*   **Community Edition (CE)**
+
+    *   Improved developer experience for handling errors using the ``box.error`` module.
+    *   Introduced fixed-size numeric field types: ``uint8``, ``int8``, ``uint16``, and more.
+    *   Added RPC functionality for accessing custom roles from the configuration.
+    *   Made the ``tt`` utility used to manage instances fully compatible with the latest Tarantool version.
+
+*   **Enterprise Edition (EE)**
+
+    *   Introduced an external coordinator for automatic and manual failover.
+    *   Improved the stability of work with the centralized configuration stored in etcd.
+
+
+.. _3-1-features-for-developers:
+
+Developing applications
+-----------------------
+
+.. _3-1-error-handling:
+
+Error handling
+~~~~~~~~~~~~~~
+
+This release improves the developer experience for handling errors using the :ref:`box.error <box-error-submodule>` module.
+Below are listed the most notable features and changes.
+
+
+.. _3-1-error_payload_fields:
+
+Error payload fields
+********************
+
+With the 3.1 release, you can add a custom payload to an error.
+The payload is passed as key-value pairs where a key is a string and a value is any Lua object.
+In the example below, the ``description`` key is used to keep the custom payload.
+
+..  code-block:: lua
+
+        custom_error = box.error.new({ type = 'CustomInternalError',
+                                       message = 'Internal server error',
+                                       description = 'Some error details'  -- payload
+        })
+
+A payload field value can be accessed using the dot syntax:
+
+..  code-block:: tarantoolsession
+
+    tarantool> custom_error.description
+    ---
+    - Some error details
+    ...
+
+
+
+
+
+.. _3-1-error_stack:
+
+Error stacks
+************
+
+The 3.1 release simplifies creating error chains.
+In the earlier versions, you need to set an error cause using the :ref:`set_prev(error_object) <box_error-set_prev>` method, for example:
+
+..  code-block:: lua
+
+    local ok, err = pcall(my_func)
+    if not ok then
+        local err2 = box.error.new{type = "MyAppError", message = "my_func failed"}
+        err2:set_prev(err)
+        err2:raise()
+    end
+
+Using this approach, you need to construct a new error without raising it, then set its cause using ``set_prev()``, and only then raise it.
+Starting with the 3.1 version, you can use a new ``prev`` argument when constructing an error:
+
+..  code-block:: lua
+
+    local ok, err = pcall(my_func)
+    if not ok then
+        box.error{type = "MyAppError", message = "my_func failed", prev = err}
+    end
+
+
+.. _3-1-error_serialization:
+
+Error serialization improvements
+********************************
+
+The 3.1 release allows you to increase the verbosity of error serialization.
+Before the 3.1 release, a serialized error representation included only an error message:
+
+..  code-block:: tarantoolsession
+
+    tarantool> box.error.new({ type = 'CustomInternalError', message = 'Internal server error'})
+    ---
+    - Internal server error
+    ...
+
+
+Starting with the 3.1 version, a serialized error also includes other fields that might be useful for analyzing errors:
+
+..  code-block:: lua
+
+    tarantool> box.error.new({ type = 'CustomInternalError', message = 'Internal server error'})
+    ---
+    - code: 0
+      base_type: CustomError
+      type: CustomInternalError
+      custom_type: CustomInternalError
+      message: Internal server error
+      trace:
+      - file: '[C]'
+        line: 4294967295
+    ...
+
+Logging an error using a built-in :ref:`logging module <log-module>` prints an error message followed by a tab space (``\t``) and all the payload fields serialized as a JSON map, for example:
+
+..  code-block:: none
+
+    main/104/app.lua/tarantool I> Internal server error {"code":0,"base_type":"CustomError","type":"CustomInternalError", ... }
+
+Given that this change may change the behavior of existing code, a new ``box_error_serialize_verbose`` :ref:`compat <compat-module>` option is introduced.
+To try out an increased verbosity of error serialization, set this option to ``new``:
+
+..  code-block:: tarantoolsession
+
+    tarantool> require('compat').box_error_serialize_verbose = 'new'
+    ---
+    ...
+
+
+.. _3-1-fixed_size_numeric_types:
+
+Fixed-size numeric types
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The 3.1 release introduces fixed-size numeric :ref:`types <index-box_data-types>` that might be useful to store data unencoded in an array for effective scanning.
+The following numeric types are added:
+
+*   ``uint8``: an integer in a range ``[0 .. 255]``.
+*   ``int8``: an integer in a range ``[-128 .. 127]``.
+*   ``uint16``: an integer in a range ``[0 .. 65,535]``.
+*   ``int16``: an integer in a range ``[-32,768 .. 32,767]``.
+*   ``uint32``: an integer in a range ``[0 .. 4,294,967,295]``.
+*   ``int32``: an integer in a range ``[-2,147,483,648 .. 2,147,483,647]``.
+*   ``uint64``: an integer in a range ``[0 .. 18,446,744,073,709,551,615]``.
+*   ``int64``: an integer in a range ``[-9,223,372,036,854,775,808 .. 9,223,372,036,854,775,807]``.
+*   ``float32``: a 32-bit floating point number.
+*   ``float64``: a 64-bit floating point number.
+
+
+
+.. _3-1-accessing_configuration:
+
+Accessing configuration of other cluster members
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In Tarantool 3.0, the :ref:`config <config-module>` module provides the ability to work with a current instance's configuration only.
+Starting with the 3.1 version, you can get all the instances that constitute a cluster and obtain the configuration of any instance of this cluster.
+
+The ``config:instances()`` function lists all instances of the cluster:
+
+..  code-block:: tarantoolsession
+
+    sharded_cluster:router-a-001> require('config'):instances()
+    ---
+    - storage-a-001:
+        group_name: storages
+        instance_name: storage-a-001
+        replicaset_name: storage-a
+      storage-b-002:
+        group_name: storages
+        instance_name: storage-b-002
+        replicaset_name: storage-b
+      router-a-001:
+        group_name: routers
+        instance_name: router-a-001
+        replicaset_name: router-a
+      storage-a-002:
+        group_name: storages
+        instance_name: storage-a-002
+        replicaset_name: storage-a
+      storage-b-001:
+        group_name: storages
+        instance_name: storage-b-001
+        replicaset_name: storage-b
+    ...
+
+To get the specified configuration value for a certain instance, pass an instance name as an argument to ``config:get()``:
+
+..  code-block:: tarantoolsession
+
+    sharded_cluster:router-a-001> require('config'):get('iproto', {instance = 'storage-b-001'})
+    ---
+    - readahead: 16320
+      net_msg_max: 768
+      listen:
+      - uri: 127.0.0.1:3304
+      threads: 1
+      advertise:
+        peer:
+          login: replicator
+        client: null
+        sharding:
+          login: storage
+    ...
+
+
+
+.. _3-1-administration-and-maintenance:
+
+Administration and maintenance
+------------------------------
+
+.. _3-1-failover_coordinator:
+
+Failover coordinator (EE)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Tarantool Enterprise Edition 3.1 introduces an external failover coordinator that monitors a Tarantool cluster and performs automatic leadership change if a current replica set leader is inaccessible.
+
+A failover coordinator requires the :ref:`replication.failover <configuration_reference_replication_failover>` configuration option to be set to ``supervised``:
+
+..  code-block:: yaml
+
+    replication:
+      failover: supervised
+
+    # ...
+
+To start a failover coordinator, execute the ``tarantool`` command with the ``failover`` option and pass a path to a :ref:`YAML configuration file <configuration_overview>`:
+
+.. code-block:: console
+
+    $ tarantool --failover --config /path/to/config
+
+A failover coordinator connects to all the instances, polls them for their status, and controls that each replica set with ``replication.failover`` set to ``supervised`` has only one writable instance.
+
+Optionally, you can configure failover timeouts and other parameters in the ``failover`` section at the :ref:`global level <configuration_scopes>`:
+
+..  code-block:: yaml
+
+    failover:
+      call_timeout: 1
+      lease_interval: 15
+      renew_interval: 5
+      stateboard:
+        renew_interval: 1
+        keepalive_interval: 5
+
+
+.. _3-1-sharding:
+
+Sharding
+~~~~~~~~
+
+The 3.1 release includes new :ref:`sharding <configuration_reference_sharding>` options that provide additional flexibility for configuring a sharded cluster.
+A new ``sharding.weight`` specifies the relative amount of data that a replica set can store.
+In the example below, the ``storage-a`` replica set can store twice as much data as ``storage-b``:
+
+..  code-block:: yaml
+
+    # ...
+    replicasets:
+      storage-a:
+        sharding:
+          weight: 2
+        # ...
+      storage-b:
+        sharding:
+          weight: 1
+        # ...
+
+
+
+The ``sharding.rebalancer_mode`` option configures whether a rebalancer is selected manually or automatically.
+This option can have one of three values:
+
+*   ``auto`` (default): if there are no replica sets with the ``rebalancer`` sharding role (:ref:`sharding.roles <configuration_reference_sharding_roles>`), a replica set with the rebalancer will be selected automatically among all replica sets.
+*   ``manual``: one of the replica sets should have the ``rebalancer`` sharding role. The rebalancer will be in this replica set.
+*   ``off``: rebalancing is turned off regardless of whether a replica set with the ``rebalancer`` sharding role  exists or not.
+
+
+.. _3-1-compatibility_tt:
+
+Compatibility with the tt utility
+---------------------------------
+
+With this release, the ``tarantoolctl`` utility used to administer Tarantool instances is completely removed from Tarantool packages.
+The latest version of the :ref:`tt utility <tt-cli>` is fully compatible with Tarantool 3.1 and covers all the required functionality:
+
+*   Setting up a development environment: initializing the environment and installing different Tarantool versions.
+*   Various capabilities for developing cluster applications: creating applications from templates, managing modules, and building and packaging applications.
+*   Managing cluster instances: starting and stopping instances, connecting to remote instances for administration, and so on.
+*   Importing and exporting data (Enterprise Edition only).
+
+Learn how to migrate from ``tarantoolctl`` to ``tt`` in the :ref:`tarantoolctl-migration-to-tt` section.

doc/release/index.rst

@@ -101,6 +101,7 @@ For information about earlier versions, see :doc:`eol_versions`.
 ..  toctree::
     :maxdepth: 1
 
+    3.1.0
     3.0.0
     2.11.0
     eol_versions