Merge pull request #32 from paliarush/Guest-Configs

Access to Guest configs from host IDE

Author: paliarush

CHANGELOG.md

@@ -12,14 +12,19 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 ### Changed
 
  - Removed requirement for public github token due to Composer limitations (issue is fixed on Composer side)
+ - Changed requirement for minimum box version from 1.0 to 1.1
 
 ### Fixed
 
  - Fixed permissions during Magento installation on Windows hosts
+ - Fixed issue with Magento compiler on Windows hosts
+ - Fixed "stdin: is not a tty" warning
 
 ### Added
 
  - Added host wrapper script for bin/magento command on guest
+ - Added ability to modify guest config files (PHP, Apache etc) directly from host IDE
+ - Added ability to choose if PhpStorm configs should be removed during project reinitialization
 
 ## [v2.0.0] - 2016-02-05
 
@@ -49,18 +54,18 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 
 ### Added
 
- - Integrated vagrant host manager plugin to allow automatic /etc/hosts update 
- - Added support of EE linked to CE using symlinks on *nix hosts 
+ - Integrated vagrant host manager plugin to allow automatic /etc/hosts update
+ - Added support of EE linked to CE using symlinks on *nix hosts
  - Added ${MAGENTO_ROOT} environment variable, which stores installation path on the guest
  - Added support of Rabbit MQ
- - Added possibility to specify tokens for repo.magento.com composer repository 
+ - Added possibility to specify tokens for repo.magento.com composer repository
  - git is now installed on guest machine
- - Removed 'magento' MySQL user, password of 'root' user removed 
- - Database for integration tests are created by default 
- - Added script for clearing Magento cache from host command line 
+ - Removed 'magento' MySQL user, password of 'root' user removed
+ - Database for integration tests are created by default
+ - Added script for clearing Magento cache from host command line
  - Configured XDebug to allow remote debugging
- - Fixed max_nesting_level issue with XDebug enabled  
+ - Fixed max_nesting_level issue with XDebug enabled
  - Apache is run by 'vagrant' user
- - Enabled Magento cron jobs 
+ - Enabled Magento cron jobs
  - Enabled XDebug by default
  - Created vagrant configuration for Magneto 2 CE developer's environment installation

README.md

@@ -19,6 +19,7 @@
    * [Use Magento CLI (bin/magento)](#use-magento-cli-binmagento)
    * [Debugging with XDebug](#debugging-with-xdebug)
    * [Connecting to MySQL DB](#connecting-to-mysql-db)
+   * [Accessing PHP and other config files](#accessing-php-and-other-config-files)
    * [Multiple Magento instances](#multiple-magento-instances)
    * [Reset environment](#reset-environment)
 
@@ -210,14 +211,21 @@ To debug Magento Setup script, go to [Magento installation script](scripts/guest
 
 Answer can be found [here](https://github.com/paliarush/magento2-vagrant-for-developers/issues/8)
 
+### Accessing PHP and other config files
+
+It is possible to view/modify majority of guest machine config files directly from IDE on the host. They will be accessible in [etc/guest](etc/guest) directory only when guest machine is running. The list of accessible configs includes: PHP, Apache, Mysql, Varnish, RabbitMQ.
+Do not edit any symlinks using PhpStorm because it may break your installation.
+
+After editing configs in IDE it is still required to restart related services manually.
+
 ### Multiple Magento instances
- 
+
 To install several Magento instances based on different code bases, just follow [Installation steps](#installation-steps) to initialize project in another directory on the host.
 Unique IP address, SSH port and domain name will be generated for each new instance if not specified manually in `etc/config.yaml`
 
 ### Reset environment
 
-It is possible to reset project environment to default state, which you usually get just after project initialization. The following command will delete vagrant box, vagrant project settings and PhpStorm project settings. After that it will initialize project from scratch. Magento 2 code base (`magento2ce` directory) will stay untouched.
+It is possible to reset project environment to default state, which you usually get just after project initialization. The following command will delete vagrant box, vagrant project settings and PhpStorm project settings. After that it will initialize project from scratch. Magento 2 code base (`magento2ce` directory) and [etc/config.yaml](etc/config.yaml.dist) will stay untouched, but guest config files (located in [etc/guest](etc/guest)) will be removed.
 
 Go to 'vagrant-magento' created earlier and run in command line:
 
@@ -232,3 +240,11 @@ Go to 'vagrant-magento' created earlier and run in command line:
 ```
 bash init_project.sh -fc
 ```
+
+To reset PhpStorm project configuration, in addition to `-f` specify `-p` option:
+
+```
+bash init_project.sh -fp
+OR
+bash init_project.sh -fcp
+```

Vagrantfile

@@ -34,27 +34,28 @@ guest_memory = config_data['guest']['memory']
 # NFS will be used for *nix and OSX hosts, if not disabled explicitly in config
 use_nfs_for_synced_folders = !OS.is_windows && (config_data['guest']['use_nfs'] == 1)
 
-host_magento_dir = Dir.pwd + '/magento2ce'
+host_vagrant_dir = Dir.pwd + ''
+host_magento_dir = host_vagrant_dir + '/magento2ce'
 
 VAGRANT_API_VERSION = 2
 Vagrant.configure(VAGRANT_API_VERSION) do |config|
     config.vm.box = "paliarush/magento2.ubuntu"
-    config.vm.box_version = "~> 1.0"
+    config.vm.box_version = "~> 1.1"
 
     config.vm.provider "virtualbox" do |vb|
         vb.memory = guest_memory
     end
 
     config.vm.synced_folder '.', '/vagrant', disabled: true
-    config.vm.synced_folder './etc', '/vagrant/etc'
+    config.vm.synced_folder './etc', '/vagrant/etc', mount_options: ["dmode=775,fmode=664"]
     config.vm.synced_folder './scripts', '/vagrant/scripts'
     config.vm.synced_folder './.idea', '/vagrant/.idea', create: true
     if use_nfs_for_synced_folders
         guest_magento_dir = host_magento_dir
         config.vm.synced_folder host_magento_dir, guest_magento_dir, type: "nfs", create: true
     else
         guest_magento_dir = '/var/www/magento2ce'
-        config.vm.synced_folder host_magento_dir + '/var/generation', guest_magento_dir + '/var/generation', create: true
+        config.vm.synced_folder host_magento_dir + '/var', guest_magento_dir + '/var', create: true
         config.vm.synced_folder host_magento_dir + '/app/etc', guest_magento_dir + '/app/etc', create: true
     end
 
@@ -64,16 +65,27 @@ Vagrant.configure(VAGRANT_API_VERSION) do |config|
         magento_host_name,                          #3
         config_data['environment']['use_php7'],     #4
         host_magento_dir,                           #5
-        OS.is_windows ? "1" : "0"                   #6
+        OS.is_windows ? "1" : "0",                  #6
+        host_vagrant_dir                            #7
     ]
 
+    config.vm.provision "fix_no_tty", type: "shell", run: "always" do |s|
+        s.privileged = false
+        s.inline = "sudo sed -i '/tty/!s/mesg n/tty -s \\&\\& mesg n/' /root/.profile"
+    end
+
     config.vm.provision "configure_environment", type: "shell" do |s|
         s.path = "scripts/provision/configure_environment.sh"
         s.args = shell_script_args
     end
 
-    config.vm.provision "export_env_variables", type: "shell", run: "always" do |s|
-        s.path = "scripts/provision/export_env_variables.sh"
+    config.vm.provision "configure_environment_recurring", type: "shell", run: "always" do |s|
+        s.path = "scripts/provision/configure_environment_recurring.sh"
+        s.args = shell_script_args
+    end
+
+    config.vm.provision "export_env_variables_recurring", type: "shell", run: "always" do |s|
+        s.path = "scripts/provision/export_env_variables_recurring.sh"
         s.args = shell_script_args
     end
 

docs/phpstorm-configuration-windows-hosts.md

@@ -7,7 +7,7 @@ This solution is applicable to any Vagrant configuration for Magento instance, s
 
     ```
     Vagrant.configure(2) do |config|
-      config.vm.synced_folder '../magento2ce/var/generation', '/var/www/magento2ce/var/generation'
+      config.vm.synced_folder '../magento2ce/var', '/var/www/magento2ce/var'
       config.vm.synced_folder '../magento2ce/app/etc', '/var/www/magento2ce/app/etc'
     end
     ```

etc/.gitignore

@@ -5,3 +5,4 @@
 !/composer/auth.json.dist
 !/config.yaml.dist
 !/magento2_virtual_host.conf.dist
+!/guest

etc/guest/.gitignore

@@ -0,0 +1 @@
+/*

init_project.sh

@@ -37,16 +37,21 @@ rm -f "${config_path}.back"
 # Clean up the project before initialization if "-f" option was specified. Remove codebase if "-fc" is used.
 force_project_cleaning=0
 force_codebase_cleaning=0
-while getopts 'fc' flag; do
+force_phpstorm_config_cleaning=0
+while getopts 'fcp' flag; do
   case "${flag}" in
     f) force_project_cleaning=1 ;;
     c) force_codebase_cleaning=1 ;;
+    p) force_phpstorm_config_cleaning=1 ;;
     *) error "Unexpected option ${flag}" ;;
   esac
 done
 if [ ${force_project_cleaning} -eq 1 ]; then
     vagrant destroy -f
-    rm -rf ${vagrant_dir}/.idea ${vagrant_dir}/.vagrant
+    mv ${vagrant_dir}/etc/guest/.gitignore ${vagrant_dir}/etc/.gitignore.back
+    rm -rf ${vagrant_dir}/.vagrant ${vagrant_dir}/etc/guest
+    mkdir ${vagrant_dir}/etc/guest
+    mv ${vagrant_dir}/etc/.gitignore.back  ${vagrant_dir}/etc/guest/.gitignore
     if [ ${force_codebase_cleaning} -eq 1 ]; then
         rm -rf ${magento_ce_dir}
     fi
@@ -77,4 +82,26 @@ bash "${vagrant_dir}/scripts/host/composer.sh" install
 cd ${vagrant_dir}
 vagrant up
 
-bash "${vagrant_dir}/scripts/host/configure_php_storm.sh"
+set +x
+echo "Configuring PhpStorm..."
+if [ ${force_project_cleaning} -eq 1 ] && [ ${force_phpstorm_config_cleaning} -eq 1 ]; then
+    rm -rf ${vagrant_dir}/.idea
+fi
+if [ ! "$(ls -A ${vagrant_dir}/.idea)" ]; then
+    bash "${vagrant_dir}/scripts/host/configure_php_storm.sh"
+fi
+
+bold=$(tput bold)
+regular=$(tput sgr0)
+echo "
+${bold}[Important]${regular}
+    Please use ${bold}${vagrant_dir}${regular} directory as PhpStorm project root, NOT ${bold}${magento_ce_dir}${regular}."
+
+use_nfs=$(bash "${vagrant_dir}/scripts/get_config_value.sh" "guest_use_nfs")
+if [[ ${host_os} == "Windows" || ${use_nfs} == 0 ]]; then
+    echo "
+${bold}[Optional]${regular}
+    To verify that deployment configuration for ${bold}${magento_ce_dir}${regular} in PhpStorm is correct,
+        use instructions provided here: ${bold}https://github.com/paliarush/magento2-vagrant-for-developers/blob/2.0/docs/phpstorm-configuration-windows-hosts.md${regular}.
+    If not using PhpStorm, you can set up synchronization using rsync"
+fi

scripts/guest/link_configs

@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+
+function process_configs () {
+    configs_path=$1
+    configs=$2
+    for config in "${configs[@]}"
+    do
+        if [ ! -d /vagrant/etc/guest/${config} ] && [ ! -f /vagrant/etc/guest/${config} ]; then
+            if [ -d ${configs_path}/${config} ] || [ -f ${configs_path}/${config} ]; then
+                sudo rm -rf "${configs_path}/${config}.back"
+                sudo cp -rp ${configs_path}/${config} "${configs_path}/${config}.back"
+                sudo mv ${configs_path}/${config} /vagrant/etc/guest/${config}
+                sudo ln -s /vagrant/etc/guest/${config} ${configs_path}/${config}
+            fi
+        fi
+    done
+}
+
+# Enable trace printing and exit on the first error
+set -ex
+
+# Below configuration is required to allow managing mysql as a service
+if ! cat /etc/apparmor.d/local/usr.sbin.mysqld | grep -q '/vagrant/etc/guest' ; then
+    echo "
+        /vagrant/etc/guest/mysql/*.pem r,
+        /vagrant/etc/guest/mysql/conf.d/ r,
+        /vagrant/etc/guest/mysql/conf.d/* r,
+        /vagrant/etc/guest/mysql/*.cnf r," >> /etc/apparmor.d/local/usr.sbin.mysqld
+fi
+
+# Make guest configs visible and editable in the host IDE
+
+# Configs located under /etc/*
+config_dir="/etc"
+# See unlink_configs script
+configs=( apache2 php mysql varnish rabbitmq )
+process_configs ${config_dir} ${configs}

scripts/guest/m-clear-cache

@@ -4,14 +4,12 @@ echo "Clearing Magento cache..."
 
 rm -f ${MAGENTO_ROOT}/app/etc/paths.php
 #clear var
-for dir in `ls ${MAGENTO_ROOT}/var` ; do
-    rm -rf ${MAGENTO_ROOT}/var/${dir}/* ;
-done
+cd ${MAGENTO_ROOT} && mv var/.htaccess var_htaccess.back && rm -rf var/* var/.[^.]* && mv var_htaccess.back var/.htaccess
 #clear pub/statics
-cd ${MAGENTO_ROOT}/pub && mv static/.htaccess static_htaccess.back && rm -rf static/* && mv static_htaccess.back static/.htaccess
+cd ${MAGENTO_ROOT}/pub && mv static/.htaccess static_htaccess.back && rm -rf static && mkdir static && mv static_htaccess.back static/.htaccess
 #clear integration tests tmp
-cd ${MAGENTO_ROOT}/dev/tests/integration && mv tmp/.gitignore tmp_gitignore.back && rm -rf tmp/* && mv tmp_gitignore.back tmp/.gitignore
+cd ${MAGENTO_ROOT}/dev/tests/integration && mv tmp/.gitignore tmp_gitignore.back && rm -rf tmp && mkdir tmp && mv tmp_gitignore.back tmp/.gitignore
 #clear unit tests tmp
-cd ${MAGENTO_ROOT}/dev/tests/unit && rm -rf tmp/*
+cd ${MAGENTO_ROOT}/dev/tests/unit && rm -rf tmp && mkdir tmp
 
 echo "Magento cache cleared."

scripts/guest/m-reinstall

@@ -25,6 +25,7 @@ setupOptions[amqp_host]='localhost'
 setupOptions[amqp_port]='5672'
 setupOptions[amqp_user]='guest'
 setupOptions[amqp_password]='guest'
+setupOptions[amqp_virtualhost]='/'
 
 cd ${MAGENTO_ROOT}
 
@@ -62,11 +63,12 @@ install_cmd="./bin/magento setup:install \
     --use-rewrites=1"
 
 # Configure Rabbit MQ
-if [ -f "${MAGENTO_ROOT}/app/code/Magento/Amqp/registration.php" ]; then
+if [ -f "${MAGENTO_ROOT}/app/etc/enterprise/di.xml" ] && [ -d "${MAGENTO_ROOT}/app/code/Magento/Amqp" ]; then
     install_cmd="${install_cmd} \
     --amqp-host=${setupOptions[amqp_host]} \
     --amqp-port=${setupOptions[amqp_port]} \
     --amqp-user=${setupOptions[amqp_user]} \
+    --amqp-virtualhost=${setupOptions[amqp_virtualhost]} \
     --amqp-password=${setupOptions[amqp_password]}"
 fi
 
@@ -91,15 +93,6 @@ sed -i "s|${MAGENTO_ROOT}|${MAGENTO_ROOT_HOST}|g" "/vagrant/.idea/misc.xml"
 set +x
 echo "
 Magento application was deployed to ${MAGENTO_ROOT} and installed successfully
+
 Access storefront at ${setupOptions[base_url]}
 Access admin panel at ${setupOptions[base_url]}${setupOptions[admin_frontname]}/"
-
-if [[ ${is_windows_host} == 1 || ${use_nfs} == 0 ]]; then
-    echo "
-    [Optional] To finish developer environment set up:
-        1. Please create new PhpStorm project using 'magento2ce' directory on your host
-        (this directory should already contain Magento repository cloned earlier)
-
-        2. Use instructions provided here https://github.com/paliarush/vagrant-magento/blob/2.0/docs/phpstorm-configuration-windows-hosts.md
-        to set up synchronization in PhpStorm (or using rsync) with ${MAGENTO_ROOT} directory"
-fi