Skip to content

Update template-only-bin scripts #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,22 @@ See [`navapbc/platform`](https://github.com/navapbc/platform) for other template

To get started using the template application on your project:

1. Run the [download and install script](./template-only-bin/download-and-install-template.sh) in your project's root directory.
1. Run the [download and install script](./template-only-bin/download-and-install-template) in your project's root directory.

```bash
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template.sh | bash -s
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template | bash -s
```

This script will:

1. Clone the template repository
2. Copy the template files into your project directory
3. Remove any files specific to the template repository, like this README.
3. Ignore any files specific to the template repository, like this README.

You can optionally pass in a branch, commit hash, or release that you want to install. For example:

```bash
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template.sh | bash -s -- <commit_hash>
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template | bash -s -- <commit_hash>
```
2. [Follow the steps in `app-rails/README.md`](./app-rails/README.md) to set up the application locally.
3. Optional, if using the Platform infrastructure template: [Follow the steps in the `template-infra` README](https://github.com/navapbc/template-infra#installation) to set up the various pieces of your infrastructure.
Expand All @@ -58,16 +58,16 @@ To get started using the template application on your project:

If you have previously installed this template and would like to update your project to use a newer version of this application:

1. Run the [download and install script](./template-only-bin/download-and-install-template.sh) in your project's root directory and pass in the branch, commit hash, or release that you want to update to, followed by the name of your application directory (e.g. `app-rails`).
1. Run the [update script](./template-only-bin/update-template) in your project's root directory and pass in the branch, commit hash, or release that you want to update to, followed by the name of your application directory (e.g. `app-rails`).

```bash
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template.sh | bash -s -- <commit_hash> <app_name>
curl https://raw.githubusercontent.com/navapbc/template-application-rails/main/template-only-bin/download-and-install-template | bash -s -- <commit_hash> <app_name>
```

This script will:

1. Clone the template repository
2. Copy the template files into your project directory
3. Remove any files specific to the template repository, like this README.
3. Ignore any files specific to the template repository, like this README.

⚠️ Warning! This will modify existing files. Review all changes carefully after executing the script by running `git diff`.
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@ template_short_name="app-${template_name##*-}"
target_version=${1:-"main"}
app_name=${2:-"${template_short_name}"}

echo "template_short_name: ${template_short_name}"
echo "app_name: ${app_name}"
echo "target_version: ${target_version}"

git clone "https://github.com/navapbc/${template_name}.git"
cd "${template_name}"

Expand All @@ -33,7 +29,7 @@ git checkout "$target_version"
cd - &> /dev/null

echo "Installing ${template_name}..."
"./${template_name}/template-only-bin/install-template.sh" "${template_name}" "${app_name}"
curl "https://raw.githubusercontent.com/navapbc/${template_name}/rocket/update-template-only-bin-scripts/template-only-bin/install-template" | bash -s -- "${template_name}" "${app_name}"

echo "Storing template version in a file..."
cd "${template_name}"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#!/usr/bin/env bash
# -----------------------------------------------------------------------------
# This script installs an application template to your project.
# Run this script using ./download-and-install-template.sh
# Run this script using ./download-and-install-template. Expected to be run
# from the project's root directory.
#
# Positional parameters:
# template_name (required) – the name of the template to install
Expand All @@ -16,34 +17,27 @@ template_name=$1
template_short_name="app-${template_name##*-}"
app_name=$2

echo "template_short_name: ${template_short_name}"
echo "app_name: ${app_name}"

curr_dir=$(pwd)
script_dir=$(dirname $0)
template_dir="${script_dir}/.."

cd $template_dir
cd "${template_name}"

if [ "$template_short_name" != "$app_name" ]; then
if [ "${template_short_name}" != "${app_name}" ]; then
echo "Modifying template to use ${app_name} instead of ${template_short_name}..."
"./template-only-bin/rename-template-app.sh" "${app_name}" "${template_short_name}"
curl "https://raw.githubusercontent.com/navapbc/${template_name}/rocket/update-template-only-bin-scripts/template-only-bin/rename-template-app" | bash -s -- "${template_short_name}" "${app_name}"
fi

echo "Copying files from $template_name..."
# Note: Keep this list of paths in sync with INCLUDE_PATHS in update-template.sh
# Note: Keep this list in sync with the files listed in update-template
# Copy only relevant files that should be included in the project repo.
echo "Copying files from ${template_name}..."
# Copy top level paths.
cp -r \
.github \
.gitignore \
.grype.yml \
"${app_name}" \
docker-compose.yml \
docker-compose.mock-production.yml \
docs \
$curr_dir
cd - >& /dev/null
${curr_dir}
# Copy nested paths. Make any missing directories.
mkdir -p "${curr_dir}/.github/workflows" && cp ".github/workflows/ci-${app_name}.yml" "${curr_dir}/.github/workflows"
mkdir -p "${curr_dir}/docs" && cp -r "docs/${app_name}" "${curr_dir}/docs"

echo "Removing files relevant only to template development..."
# Note: Keep this list of paths in sync with EXCLUDE_OPT in update-template.sh
rm -rf .github/workflows/template-only-*
rm -rf .github/ISSUE_TEMPLATE
cd - >& /dev/null
85 changes: 85 additions & 0 deletions template-only-bin/rename-template-app
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this parameters of this script makes it seem like it supports renaming from any current name to any new name, but i imagine the find/replace logic won't work for some edge cases, like if the current name is something too common.

i know we have a goal of switching to copier soon, where effectively "current name" would become something like {{app_name}}. have we considered the following options:

  • option a: going ahead and naming the current name {{app_name}}, and in all template-only github actions workflows, we first run the rename script to an actual app name (e.g. app) so that CI can run properly (i haven't thought through all of the details to see if this is feasible, but just floating the high level idea first)
  • option b: reducing the amount of flexibility (and therefore complexity around edge cases) in this script by sticking with an app name like template-only-app rather than an app name that differs per application template? then current_name can be hardcoded to template-only-app, and we have more consistency across our application templates (and can further simplify some of the scripts here).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this parameters of this script makes it seem like it supports renaming from any current name to any new name, but i imagine the find/replace logic won't work for some edge cases, like if the current name is something too common.

It currently supports renaming from anything to anything. The regex is restricted to word boundaries, so if you try to rename app to my-app, it won't replace happen with hmy-appen. But yes, if the user were to name it from app-rails to class or else... they would have some bad times when trying to rename it back to something less common.

i know we have a goal of switching to copier soon,

🤩

option a: going ahead and naming the current name {{app_name}}, and in all template-only github actions workflows, we first run the rename script to an actual app name (e.g. app) so that CI can run properly (i haven't thought through all of the details to see if this is feasible, but just floating the high level idea first)

I have a separate script I'm finishing up that effectively "installs an app" for the infra template, allowing the user to provide the app name as an argument. That's in a forthcoming PR.

option b: reducing the amount of flexibility (and therefore complexity around edge cases) in this script by sticking with an app name like template-only-app rather than an app name that differs per application template? then current_name can be hardcoded to template-only-app, and we have more consistency across our application templates (and can further simplify some of the scripts here).

Per the interest and arguments in navapbc/platform#21, I was sticking with app-*.

Is the idea behind option b that all the application templates will use template-only-app and in the installation instructions, the user specifies the name they want to use?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the idea behind option b is that all application templates will use template-only-app and force the project team to decide on an app name (which if they really don't want to think about it they can rename it back to something generic like app). The benefit is to simplify the code path in the install/update paths to limit the amount of dynamic code (the logic for generating template_short_name for example and guaranteeing the "current_name" is the same string in the rename script). I admit it's not a huge simplification.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to postpone digging into this question until after I've had a chance to make a proof-of-concept of how things might work with copier.

Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env bash
# -----------------------------------------------------------------------------
# This script renames the template application in a project.
# Run this script in a project's root directory.
#
# The project name is the name of the folder in your project's root directory. Use
# lowercase letters and hyphens. Do not use spaces. Underscores may have unexpected side
# effects. Choose a unique string that will avoid collisions with commonly used words.
# By default, the application name is `app-rails`.
#
# Positional parameters:
# current_name (required) – the current name for the application
# new_name (required) - the new name for the application
# -----------------------------------------------------------------------------
set -euo pipefail

# Helper to get the correct sed -i behavior for both GNU sed and BSD sed (installed by default on macOS)
# Hat tip: https://stackoverflow.com/a/38595160
sedi () {
sed --version >/dev/null 2>&1 && sed -i -- "$@" || sed -i "" "$@"
}
# Export the function so it can be used in the `find -exec` calls later on
export -f sedi

current_name=$1
new_name=$2
default_name="app-rails"

# Debug:
echo "---------------------------------------------------------------------"
echo "current_name: ${current_name}"
echo "new_name: ${new_name}"
echo

if [[ "${current_name}" == "${new_name}" ]]; then
# Debug:
echo "No rename required: ${current_name} == ${new_name}"
exit 0
fi

# Note: Keep this list in sync with the files copied in install-template and update-template
declare -a include_paths
include_paths=(.github/workflows/ci-app-rails.yml)
include_paths+=(.grype.yml)
include_paths+=(app-rails)
include_paths+=(docker-compose.yml)
include_paths+=(docker-compose.mock-production.yml)
include_paths+=(docs/app-rails)

# Loop through the paths to be included in this template.
for include_path in "${include_paths[@]}"; do
# If the application does not use the default name (i.e. it has already been renamed),
# change the include path to use the correct current_name.
if [[ "${current_name}" != "${default_name}" ]]; then
include_path=$(echo "${include_path}" | sed "s/${default_name}/${current_name}/g")
fi

echo "Checking '${include_path}' to rename '${current_name}' to '${new_name}'..."

# Skip if the path does not exist.
if [[ ! -d "${include_path}" ]] && [[ ! -f "${include_path}" ]]; then
echo "Skipping ahead. ${include_path} does not exist in this repo"
continue
fi

# Construct the correct string substitution that respects word boundaries.
# Hat tip: https://unix.stackexchange.com/a/393968
if sed --version >/dev/null 2>&1; then
word_boundary_replacement="s/\<${current_name}\>/${new_name}/g"
else
word_boundary_replacement="s/[[:<:]]${current_name}[[:>:]]/${new_name}/g"
fi

# Replace occurrances of the current_name with the new_name in the path.
# If the path is a file, replace in the file.
# If the path is a directory, recursively replace in all files in the directory.
LC_ALL=C find "${include_path}" -type f -exec bash -c "sedi \"${word_boundary_replacement}\" \"{}\"" \;

# Rename included paths that contain the current_name.
if [[ "${include_path}" =~ "${current_name}" ]]; then
new_include_path=$(echo "${include_path}" | sed "s/${current_name}/${new_name}/g")
echo "Renaming path from '${include_path}' to '${new_include_path}'..."
mv "${include_path}" "${new_include_path}"
fi
done
46 changes: 0 additions & 46 deletions template-only-bin/rename-template-app.sh

This file was deleted.

75 changes: 75 additions & 0 deletions template-only-bin/update-template
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
# -----------------------------------------------------------------------------
# This script updates an application template in your project.
# This script from your project's root directory.
#
# Positional parameters:
# target_version (optional) – the version of the template application to install.
# Defaults to main. Can be any target that can be checked out, including a branch,
# version tag, or commit hash.
# app_name (optional) – the name of the application, in either snake- or kebab-case
# Defaults to app-rails.
# -----------------------------------------------------------------------------
set -euo pipefail

# Helper to get the correct sed -i behavior for both GNU sed and BSD sed (installed by default on macOS)
# Hat tip: https://stackoverflow.com/a/38595160
sedi () {
sed --version >/dev/null 2>&1 && sed -i -- "$@" || sed -i "" "$@"
}
# Export the function so it can be used in the `find -exec` calls later on
export -f sedi

template_name="template-application-rails"
# Use shell parameter expansion to get the last word, where the delimiter between
# words is `-`.
# See https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
template_short_name="app-${template_name##*-}"

target_version=${1:-"main"}
app_name=${2:-"${template_short_name}"}
current_version=$(cat ".${template_name}-version")

git clone "https://github.com/navapbc/${template_name}.git"

echo "Checking out $target_version..."
cd "${template_name}"
git checkout "$target_version"
cd - &> /dev/null

# Note: Keep this list in sync with the files copied in install-template
cd "${template_name}"
include_paths=" \
.github/workflows/ci-${template_short_name}.yml
.grype.yml \
${template_short_name} \
docker-compose.yml \
docker-compose.mock-production.yml \
docs/${template_short_name}"
git diff $current_version $target_version -- $include_paths > update.patch
cd - &> /dev/null

if [ "$template_short_name" != "$app_name" ]; then
echo "Modifying patch to use ${app_name} instead of ${template_short_name}..."
# Construct the correct string substitution that respects word boundaries.
# Hat tip: https://unix.stackexchange.com/a/393968
if sed --version >/dev/null 2>&1; then
word_boundary_replacement="s/\<${template_short_name}\>/${app_name}/g"
else
word_boundary_replacement="s/[[:<:]]${template_short_name}[[:>:]]/${app_name}/g"
fi
sedi "${word_boundary_replacement}" "${template_name}/update.patch"
fi

echo "Applying patch..."
git apply --allow-empty "${template_name}/update.patch"

echo "Storing template version in a file..."
cd "${template_name}"
git rev-parse HEAD >../".${template_name}-version"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarification question: is the template version file being stored in the application folder? or the project repo root?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, in the project root dir. Not the application dir. Same as template-application-nextjs and template-application-flask.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying. I think that poses a problem if a repo has two application using the same application template. Should we consider moving the template file into the app folder?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I agree that we should move the .template-*-version files to within the app folders. I'll start a 🔒 slack discussion thread about this.

cd - &> /dev/null

echo "Cleaning up ${template_name} folder..."
rm -fr "${template_name}"

echo "...Done."
Loading