2002.0.18

Changelog (24 related PRs)

• Docker Updates—
  • Now, the Docker environment supports the cron configuration defined in the crons property of the .magento.app.yaml file.
  • New Docker Container—Added a TLS termination proxy container to facilitate the Varnish SSL termination over HTTPS.
  • New Docker Image—Added a Node.js image to support Gulp and other capabilities, such as Jasmine JS Unit Testing.
  • Docker build modes—Now you can choose to launch the Docker environment in Production mode or Developer mode. Developer mode supports active development with full, writable filesystem permissions. This improves the developer experience by providing an easier way to develop and sync extensions, for example.
  • Fixed an issue that caused Docker deploy to fail with a Name or service not known error if the cache is configured for a service that is not available. Now, you can remove a service from the .magento/services.yaml file. The Docker configuration generator updates the service in the docker/config.php.dist file automatically.
  • Added interactive validations for service compatibility. Now, if a requested service is incompatible with the Magento version or other services, the interactive mode prompts the user with a message and a choice to continue. See the Service versions available for Docker. Use the -n option to skip the interactivity for CICD purposes.
  • Fixed an issue with the Docker compose db-dump command that erased existing dumps.
• Fixed an issue that assigned Redis session, default, and page_cache cache storage to the same database ID.
• Environment variable updates—
  • The new ELASTICSUITE_CONFIGURATION environment variable retains your customized service settings between deployments.
  • Added the SCD_MAX_EXECUTION_TIMEOUT environment variable so you can increase the time to complete the static content deployment from the .magento.env.yaml file.
  • Added the MAGENTO_CLOUD_LOCKS_DIR environment variable to configure the path to the mount point for the lock provider on the cloud infrastructure. The lock provider prevents the launch of duplicate cron jobs and cron groups. This variable is supported on Magento version 2.2.5 and later and automatically configured.
  • Changed the SCD_THREAD environment variable default values to automatically determine the optimal value based on the detected CPU thread count.
• Fixed an issue with a patch for DB Isolation Mechanism that caused an error when upgrading to ECE-Tools versions 2002.0.16 and 2002.0.17
• Added a patch that replaces Google Image Charts with Image-Charts. See the DevBlog article Google Image Charts deprecation and update for M1.
• Added validation for the SEARCH_CONFIGURATION variable. Deploy fails when the 'engine' option is not set and _merge is not required.
• Fixed an issue that exposed sensitive data after an exception occurs. Now the sensitive information is masked appropriately.
• Improved the fault-tolerant settings of the ECE-Tools package. In the case when Magento 2 cannot read data from the Redis slave instance, a reading is made from the Redis master instance.

2002.0.17

Changelog (8 related PRs)

The ECE-Tools version 2002.0.17 includes an important security patch. See Tech Resources: Magento Open Source Patches.

• Service updates —Supported by Magento versions 2.3.1 and later 2.3.x versions or 2.2.8 and later 2.2.x versions.

  • Added support for Elasticsearch version 6.x.
  • Added support for Redis version 5.0.

• New Docker images—Added the following services to the Docker build:

  • Elasticsearch 6.5
  • Redis 5.0

• New environment variable —Previously, there was a hard-coded timeout for SCD compression. Now you can configure the SCD compression timeout using the SCD_COMPRESSION_TIMEOUT environment variable.

• Added the --use-rewrites option to the Magento install command so that it uses web server rewrites for generated links in the storefront and Admin access to improve security and customer experience.

• Added timestamps to the var/log/install_upgrade.log file so that it shows dates for the Magento installation and upgrade events.

2002.0.16

Changelog (30 related PRs)

• Docker updates—

  • Now, the default service configuration generated in the Docker environment is the same as the default configuration in the Cloud template.

  • You can send mail from your Docker environment using the sendmail service

  • Added the ability to configure Xdebug to debug in the Cloud Docker environment.

  • Fixed an issue with web service permissions when generating the docker-compose.yml file.

• Upgrade improvement—Added validation to confirm that the autoload property in the composer.json file contains required configuration changes before upgrading to Magentgo v2.3.

• The compression process in deploying static content now includes all assets—naively generated or customized—and occurs during the build phase at the beginning of the build:transfer section. Previously, the compression process occurred before applying custom minification and bundling of static assets.

• Fixed a database connection error that occurred during deployment immediately after configuring an additional database and service relationship. Also, this fix addresses an issue that occurred during the configuration process of MBI for Starter. For Starter, this upgrade is a "must have" for using MBI.

• Fixed a validation issue with the database configuration that caused the deploy process to fail.

• Updated the constraint with the appropriate version of the symfony/yaml package to use with PHP constants. Constant parsing does not work when using a symfony/yaml package version earlier than 3.2.

• Environment configuration check—Added validation to check the PHP version and warn users if they are not using the latest recommended version.

• Fixed an issue with processing malformed JSON variables. Now, if a JSON variable causes a syntax error, a warning appears in the cloud.log file and deployment continues using the default variable.

• Fixed a connection error that occurred during deployment immediately after disabling the Redis service.

• Logging changes—Updated the log level from Info to Notice for the following build and deploy process events:

  • Begin and end of the process for reconciling installed modules in composer.json with shared configuration settings in the app/etc/config.php file
  • Begin and end of the configuration validation process
  • Begin and end of the setup:di:compile process for generating classes

• New environment variables—

  • RESOURCE_CONFIGURATION deploy variable—Use this variable to map a resource name to a database connection.
  • X_FRAME_CONFIGURATION global variable—Use this variable to change the X-Frame-Options header configuration for rendering a Magento page in a <frame>, <iframe>, or <object>.

• Environment variable updates—Changed the following environment variables:

  • WARM_UP_PAGES—Added the capability to preload the cache for specified pages on all domains defined for a Magento store. Previously, if your site was configured with multiple domains, the post-deploy process failed to preload the cache for the specified pages on non-default domains and returned the following error in the post-deploy log: ERROR: Warming up failed: <uri>
  • SCD_COMPRESSION_LEVEL—Updated the documentation and the sample .magento.env.yaml file with the correct default values for SCD compression level. See the definitions in the build variables and the deploy variables content.
  • SCD_EXCLUDE_THEMES—This environment variable is deprecated. Use the SCD_MATRIX variable to control theme configuration. See the definitions in the build variables and the deploy variables content.
  • SCD_MATRIX—Fixed the validation process to prevent a problem that occurred when the SCD_MATRIX ignored a theme value that contained different character cases. See the definitions in the build variables and the deploy variables content.
  • ADMIN variables—
    • Improved security when managing credentials for the Magento Admin user using environment variables. You can no longer use the ADMIN_EMAIL, ADMIN_USERNAME, and ADMIN_PASSWORD environment variables to override admin credentials during upgrades. If you cannot access the Admin panel, use the Forgot password feature or the Magento CLI admin:user:create command to create a new admin user.
    • ADMIN_EMAIL is no longer required when upgrading or applying patches.

2002.0.15

Changelog (21 related PRs)

• Docker updates—
  • Now the Docker generator uses the services specified in the .magento.app.yaml and .magento/services.yaml configuration files when building your Docker environment. You can choose a different service version using build parameters.
  • Added PHP 7.2 image—Added support for PHP 7.2 in Cloud Docker; updated the Launch Docker configuration to include the docker:build --php option to specify the version of PHP compatible with your Magento Commerce version.
  • Added a Cron container based on the PHP-CLI image.
  • Added the following services to the Docker build:
    • RabbitMQ 3.5 and 3.7
    • ElasticSearch 1.7, 2.4, and 5.2
    • Redis 3.2 and 4.0
  • Now you have the DB dump capability when using the Cloud Docker database container. Also, you can share files between a host machine and a container using the docker/mnt directory.
• Configure with PHP constants—Added support for PHP constants in the .magento.env.yaml configuration file.
• New environment variable—By default, only the Production environment has Google Analytics enabled. You can enable Google Analytics on the Staging and Integration environments using the ENABLE_GOOGLE_ANALYTICS environment variable.
• Fixed an issue that removed customized cron configurations from the env.php file after a redeployment. Now, custom cron configurations safely remain in the env.php file.
• Fixed inconsistencies in the messages and log levels for build, deploy, and post-deploy phases. Increased beginning and ending log message levels from info to notice for all phases and sub-phases. Added beginning and ending log messages, where appropriate.
• Fixed an issue involving cron processes that prevented the start of the post-deploy phase, when configured. Now, if you have the post-deploy hook enabled, the cron processes are enabled again at the beginning of the post-deploy phase.
• Resolved an issue that prevented a successful installation of Magento when specifying a custom database configuration. Previously, the installation process used the database configuration from the MAGENTO_CLOUD_RELATIONSHIP variable even if you designated customized connection information in the DATABASE_CONFIGURATION environment variable.
• Corrected the config:dump command so that it includes each website locale in the system section of the config.php file.
• Fixed an issue that resulted in warm-up errors during the post-deploy phase by correcting the source base URL reference.
• Fixed an issue that generated files improperly during the setup:di:compile process, which affected the Amazon Pay module.

2002.0.14

Changelog (20 related PRs)

• Verify Ideal State—The ideal-state wizard now verifies the current configuration during each deployment and provides clear instructions for updating the configuration to achieve a faster, zero-downtime deployment.
• PCI Compliance—Updated the messaging protocols for Magento to require Transport Layer Security (TLS) version 1.2 when connecting with third-party messaging services. If you are using a message service that does not support TLS version 1.2, you must upgrade your service. Otherwise, the following error message displays when your Magento application tries to connect to the message server to send an email: Unable to connect via TLS.
• Deployment improvement—Added validation to warn customers if a Staging or Production environment has dev, debug, or debug_logging options enabled to prevent performance issues caused by excessive logging activity.
• Deployment fixes—
  • Now maintenance mode is enabled at the start of the deploy phase and disabled at the end. If the deployment fails, the site remains in maintenance mode until the deployment issues are resolved. Previously, the site returned to production mode even if the deployment failed.
  • Reworked the deploy phase validation checks to downgrade the error level for the following deployment issues from CRITICAL to WARNING so that the deployment completes. Previously, these issues caused the deployment to fail.
    • Environment configuration contains incorrect values for deploy or cloud variables.
    • The Elasticsearch version on the cloud infrastructure is incompatible with the version of the elasticsearch/elasticsearch module supported by Magento. See the Elasticsearch troubleshooting article in the Magento Support Knowledgebase.
  • Fixed an issue with the shared configuration settings in the app/etc/config.php file that caused recursion detected errors during deployment.
• Cron-related fixes—
  • Fixed a cron scheduling issue that prevented jobs from running if you specify a cron frequency other than the default (1 minute).
  • Fixed an issue in the deploy phase that allowed cron jobs to continue running during deployment, which can cause database locks and other critical issues. Now, all cron jobs are stopped before the deploy phase begins and restarted after the deployment completes.
  • Fixed the cron job workflow in versions 2.2.x to unlock frozen cron jobs so that they can be stopped before beginning deployment. Previously, a frozen cron job caused the deployment to stall.
• Changed the format of the config.php file generated by the vendor/bin/ece-tools config:dump command to use short array syntax and 4-space indentation to comply with Magento coding standards.
• Fixed a deployment error that occurs when the .magento.env.yaml contains {{ base_url }} and {{ unsecure_base_url }} placeholders for web configurations instead of the default URL configuration for a Cloud project.

2002.0.13

Changelog (35 related PRs)

• Enable zero-downtime deployment—Now Magento Cloud queues requests with required database changes during deployment and applies the changes as soon as the deployment completes. Requests can be held for up to 5 minutes to ensure that no sessions are lost. See Static content deployment options to reduce deployment downtime on Cloud.

• Docker Compose for Cloud—Made the following improvements to the Docker setup and configuration process:

  • Added a command—docker:config:convert to convert PHP configuration files to Docker ENV format to simplify environment configuration. Now, you copy the PHP configuration files to the Docker directory and convert them to Docker ENV files.
  • The Magento Cloud installation process now supports deploying to both read-only and read-write file systems to more closely emulate the Cloud file system.
  • Redis service support—Added a Redis image, which is deployed to a Docker container and configured automatically to work with your Docker installation.
  • Varnish service support— Added a Varnish image, which is deployed automatically to a Docker container. After deployment, you can manually configure Varnish following Magento best practices.
  • Secure site access—Added SSL support to access your Magento store and Admin panel.

• Improved ECE-Tools extension support—Downgraded the minimum version requirement for the guzzlehttp/guzzle package in the ECE-Tools composer.json file to version 6.2 so that the ECE-Tools package is compatible with more extensions.

• Apply custom changes to your Magento application during the build phase—We split the build phase into two separate processes so that you can use hooks to apply custom changes to the generated static content before packaging the application for deployment. The build:generate process generates code, applies patches, and generates static content. The build:transfer process transfers the generated code and static content to the final destination.

• Environment configuration checks—Improved validation of the environment configuration to warn customers about version incompatibilities and configuration errors before building and deploying Magento Cloud.

  • Added version-specific validation to identify unsupported or deprecated environment variables and values.
  • Added an Elasticsearch compatibility check to warn users about Elasticsearch configuration issues. Now, the deployment fails if the version of Elasticsearch service on the server is incompatible with Magento. Previously, the deployment succeeded even if the Elasticsearch version was incompatible, which caused product catalog issues after site deployment.
    You can resolve the incompatibility by submitting a Support ticket to upgrade Elasticsearch to a compatible version, or change the Magento configuration to specify a compatible version of the Elasticsearch PHP client.
    • For Magento version 2.1.x to 2.2.2, upgrade Elasticsearch to version 2.4.
    • For Magento version 2.2.3 and later, upgrade Elasticsearch to version 5.2.
    • If you have Elasticsearch 1.x or 2.x and do not want to upgrade, update the Magento Elasticsearch PHP client version requirement in composer.json to "elasticsearch/elasticsearch": "~2.0".
  • Improved validation of environment variables to identify configuration settings that can cause conflicts during the build, deploy, and post-deploy phases. For example, a warning message displays during the install and upgrade process if the global setting for static content deployment conflicts with settings on the build or deploy phase.

• Environment variable updates—Changed the following environment variables:

  • SKIP_HTML_MINIFICATION global variable—Changed the default value to true to enable on-demand HTML content minification, which minimizes downtime when deploying to Staging and Production environments. This configuration is required for zero-downtime deployments.
  • CLEAN_STATIC_FILES deploy variable—Added the capability to manage the clean static files processing for static content generated during the build phase based on the CLEAN_STATIC_FILES environment variable setting. Previously, static content files generated during the build phase were always cleaned.

• Logging—Made the following changes to improve log messages and reduce log size:

  • Deployment failure log entries now include the command output from the operations that cause the failures even if your environment configuration does not specify debug level logging. See MIN_LOGGING_LEVEL.
  • Added logging for deployment failures that occur when generated factories required by some extensions cannot be generated correctly because the file system is in a read-only state.
  • Reduced the deploy log size and fixed formatting issues caused by setup commands that use the interactive progress bar.
  • Eliminated unnecessary verbosity and updated the priority levels for some log statements.

• Cron-specific fixes—

  • Changed the default cron job configuration settings for history lifetime from 3d (4320 min) to 1h (60 min) to prevent performance issues and deployment failures that can occur when the cron queue fills too quickly.
  • Improved the cron job management process during the deploy phase to prevent database locks and other critical issues. Now, all cron jobs stop during the deploy phase and restart after deployment completes.
  • Fixed an issue with the locking mechanism for scheduling consumers launched by cron jobs in Magento versions 2.2.0 and later to prevent cron jobs from launching duplicate consumers.

• Fixed an issue with the static content compression process (gzip) that caused not overwritten and no such file or directory errors when referencing the compressed file during the deployment process.

• Fixed an issue that prevented the php ./vendor/bin/ece-tools config:dump command from removing redundant sections from the config.php file during the dump process if the store locale is not specified. Now you can easily move your configuration files between environments. After you update to ECE-Tools v2002.0.13, regenerate older config.php files with the improved config:dump command.

• Fixed an issue that caused an error during the deploy phase if the route configuration in the .magento/routes.yaml file redirects from an apex domain to a www domain.

• Fixed an issue with the _merge option for the SEARCH_CONFIGURATION variable that caused incorrect merge results if you do not include the engine parameter in the updated .magento.env.yaml configuration file. Now, the merge operation correctly overwrites only the values you specify in the updated .magento.env.yaml without requiring you to set the engine parameter.

• Fixed a Redis configuration issue that incorrectly enabled session locking for Magento versions 2.2.1 and later, which can cause slow performance and timeouts. Now, session locking is disabled by default. The issue was caused by a change in the default behavior of the disable_locking parameter introduced in v1.3.4 of the Redis session handler package. See colinmollenhour/php-redis-session-abstract package.

2002.0.12

Changelog (30 related PRs)

• Docker Compose for Cloud—Added a command—docker:build—to generate a Docker Compose configuration from the Cloud ECE-Tools repository.
• Change Locales—Now you can change store locale without the exporting and importing configuration process. While Magento is in Production and the SCD_ON_DEMAND is enabled, the Magento store and admin locale options are available.
• Site map and Robots—Created a workflow to add a robots.txt file and generate a sitemap.xml file for a single domain configuration without requiring a change to the infrastructure.
• Wizards—Added two wizards to help you with Cloud configuration:
  • ideal-state—configure the ideal state for minimal deployment downtime
  • master-slave—configure load balancing for database and Redis
• Module Refresh—Added a Cloud command—module:refresh—to enable modules that were disabled or not explicitly enabled, similar to the way that it is done automatically during a build.
• Added the ability to choose to merge or overwrite configuration for services using the _merge option in CACHE, SESSION, QUEUE, and SEARCH configurations.
• Environment Configuration sample file—We added a .magento.env.yaml sample file to the ece-tools package that includes a detailed description and possible values for each environment variable.
  • We also added a deep validation for the .magento.env.yaml configuration that prevents failures in the deployment process caused by unexpected values. When a failure occurs, you now receive a detailed error message that begins with: Environment configuration is not valid. Please correct .magento.env.yaml file with next suggestions:
• Added the following Environment variables:
  • Now you can define multiple locales for each theme using the new SCD_MATRIX environment variable, which reduces the amount of theme files to deploy.
  • Added the DATABASE_CONFIGURATION environment variable to customize your database connections for deployment.
  • The new MIN_LOGGING_LEVEL variable overrides the minimum logging level for all output streams without making changes to the code.
• Fixed an issue that caused downtime between the deploy and post-deploy phase. Now, the post_deploy phase begins immediately after the deploy phase ends.
• Fixed an issue that did not clean the successful cron jobs, those with status = success, from the schedule.
• Fixed an issue with the post_deploy hook that cleared the cache in the deploy phase instead of the post-deploy phase of the project.
• Fixed an issue when using SCD with multiple locales, which generated the same js-translation.json file for each locale.
• Optimized the db:dump command in the ECE-Tools package to avoid locking tables and increase speed.

2002.0.11

Changelog (20 related PRs)

• Configuring read-only connections to non-master nodes—This release adds the ability to configure a read-only connection to a non-master node to receive read-only traffic (for Redis and for MariaDB).
• Configuration Wizard—Added a wizard to help verify your configuration for static content deployment.
• Symfony Console support—Added support for Symfony Console 4 with Magento 2.3.
• Cron scheduling optimizations—Improved the queue management and enhanced logging to help with debugging cron-related issues.
• Deployment validation fails if an ADMIN_EMAIL or ADMIN_USERNAME value is the same as an existing Magento administrator account.
• Removed SOLR support for 2.2.x versions. 2.1.x versions retain the ability to enable SOLR.
• The first installation of the Staging & Production environments of a PRO project now includes different index prefixes for ElasticSearch to prevent possible conflicts while identifying records belonging to each environment.
• Fixed an issue that interrupted the build phase for legacy architecture during static content deployment.
• Cron-specific Improvements—Re-worked the cron implementation:
  • Fixed an issue that caused the cron queue to fill quickly. Now it clears the outdated cron jobs in a more reliable way.
  • Re-organized the sequence of cron jobs so that all jobs in separate threads launch prior to the general group.
  • Improved logging to better assist in debugging cron issues.
  • NOTE—This release addresses many cron-related issues. If you currently use some cron-related patches in m2-hotfixes, remove them.
• SCD-specific improvements—
  • You can use the VERBOSE_COMMANDS and the SCD_COMPRESSION_LEVEL environment variables during both build and deploy phases.
  • Fixed an issue that caused deployment to fail with a random error when encountering an unexpected value for the SCD_COMPRESSION_LEVEL environment variable. Improved the configuration validation to provide meaningful notifications.
  • Fixed the behavior of the SCD_COMPRESSION_LEVEL environment variable configuration flow so the overrides work as expected.
  • Fixed an issue that prevented the configuration of the SCD_THREADS environment variable in the .magento.env.yaml file deploy stage.

2002.0.10

Changelog (23 related PRs)

• Static Content Deployment (SCD)—There is a new, alternative deployment process to generate static content when requested (on-demand). This decreases downtime and improves cache handling by generating the most critical assets.
  • New environment variable—Added the SCD_ON_DEMAND global environment variable to generate static content when requested.
  • Post-deploy hook—Added a post_deploy hook for the .magento.app.yaml file that clears the cache and pre-loads (warms) the cache after the container begins accepting connections. It is available only for Pro projects that contain Staging and Production environments in the Project Web UI and for Starter projects. Although not required, this works in tandem with the SCD_ON_DEMAND environment variable.
• Optimization—Optimized moving or copying files during deployment to improve deployment speed and decrease loads on the file system.
• Deployment Logging—Added the ability to enable Syslog and Graylog Extended Log Format (GELF) handlers for outputting logs during the deployment process.
• Added the following Environment variables:
  • CRYPT_KEY—Provide a cryptographic key to another environment when moving a database.
  • SKIP_HTML_MINIFICATION—Global environment variable that skips copying the static view files in the var/view_preprocessed directory and generates minified HTML when requested.
  • SCD_ON_DEMAND—Global environment variable to generate static content when requested.
  • WARM_UP_PAGES—You can list the pages to use to pre-load the cache. Available in the new Post-deploy variables.
• Fixed an issue that involved a locally applied patch breaking the deployment on an instance. Now, ECE-Tools can detect that a patch has been applied.
• Fixed a conflict between JavaScript bundling and GZIP functionality. Now these features work correctly together.
• Fixed an issue that caused ece-tools CLI commands to fail when using earlier PHP 7.0.x versions.
• Fixed an issue that prevented static content deployment with the compact strategy in multiple threads.
• Fixed a Redis session locking issue that caused an Admin login delay. Also, the fix is available for 2.1.x.

2002.0.9

Changelog (29 related PRs)

You must upgrade the Magento Cloud Metapackage to get this and all future updates.

• ECE-Tools—The ECE-Tools package now supports Magento 2.1.x.
• Redis configuration—You can now configure Redis page and default cache and Redis session storage using an environment variable.
• Search, AMQP, and Redis service improvements—We unified the service configuration flow so that it now behaves the same way for all services. Manually editing the env.php file to configure services is no longer supported. You must use environment variables or the .magento.env.yaml file instead.
• Environment variables—
  • The use of env:STATIC_CONTENT_THREADS was deprecated and will be removed in a future release. Use the STATIC_CONTENT_THREADS environment variable instead.
  • The STATIC_CONTENT_EXCLUDE_THEMES environment variable was deprecated. You must use the SCD_EXCLUDE_THEMES environment variable instead.
• Logging—We simplified logging around built-in patching operations.
• We removed developer mode support and the APPLICATION_MODE environment variable because they were causing unexpected behavior.
• We fixed an issue that was causing static content deployment failures related to Redis. Now, multi-threaded static content deployment runs as designed.
• We fixed an issue that was preventing users from saving modifications to configuration fields in the Admin, which are marked as sensitive after running the app:config:dump command.
• We added support for an earlier version of symfony/yaml to fix conflicts with some packages, which are not yet compatible with the latest version.