Skip to content

Commit 915fb4e

Browse files
author
Cerem Cem ASLAN
committed
added Responsive Remote Development support
1 parent 14d10b9 commit 915fb4e

File tree

3 files changed

+190
-12
lines changed

3 files changed

+190
-12
lines changed

README.md

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,15 @@
11
# Description
22

3-
This toolset is intended to use for managing remote Linux devices (RaspberryPi in mind, but any remote Linux system will work) from host Linux systems, by basically simplifying 6 tasks, where you need to:
3+
This toolset is intended for administrating remote Linux devices that are directly connected or behind a proxy server (RaspberryPi in mind, but any remote Linux system will work), by simplifying 7 tasks:
44

5-
1. ..make `ssh` for performing remote tasks (either directly or over a link up server).
6-
2. ..use simple drag and drop style file transfers (by `sshfs`).
7-
3. ..backup the target's entire root filesystem (by `rsync`).
8-
4. ..create fast and efficient **differential full backups** (by hardlinks or by BTRFS snapshots).
9-
5. ..create a separate bootable system disk from any of your backups.
10-
6. ..clone a target with a new identity.
5+
1. `make ssh` to connect the remote shell (either directly or over a link up server).
6+
2. Responsively edit remote files via local IDE almost independent from the internet connection speed and interruptions ("Responsive remote development").
7+
3. Use simple drag and drop style file transfers (by `sshfs`).
8+
4. Backup the target's entire root filesystem (by `rsync`).
9+
5. Create fast and efficient **differential full backups** (by hardlinks or by BTRFS snapshots).
10+
6. Create a separate physical bootable system disk from any of your backups.
11+
7. Clone the current device with a new identity to create a new device.
1112

12-
This simplification is achieved by:
13-
14-
* Placing separate scripts for each task described above and providing a simple `Makefile`.
15-
* Keeping the scripts, configuration files and backups in a folder called `your-project`.
1613

1714
# Install
1815

@@ -67,14 +64,32 @@ make ssh
6764

6865
Makes ssh connection either directly or via the link up server according to [your connection type](#set-connection-type).
6966

67+
### Responsive Remote Development
68+
69+
Responsive remote development means keeping a local folder in sync with a remote folder.
70+
71+
1. `cp ./sync-config-example.sh path/to/your/project/folder/my-sync-config.sh`
72+
2. Edit `my-sync-config.sh` accordingly. See `./sync-with-sgw.sh --help` for options.
73+
3. Send your project folder to your remote system and watch for changes by:
74+
75+
./sync-with-sgw.sh -c path/to/your/project/folder/my-sync-config.sh --dry-run
76+
77+
This will keep `path/to/your/project/folder/` and `$dest_dir` (within your config file) in sync. Remove the `--dry-run` switch for real transfer if the transfer summary is as you expected.
78+
79+
Synchronization will exclude the `.git` folder and the other files/folders listed in `path/to/your/project/folder/.gitignore`.
80+
81+
`run_before_sync` hooks can be used to build, bundle, copy files or perfom any other tasks before the actual synchronization. Synchronization will fail and display a visual error message if any of the hooks fails.
82+
7083
### Mount target root
7184

7285
```bash
7386
make mount-root
7487
```
7588
Mounts the root folder to `your-project/NODE_ROOT`, which you can use for drag-n-drop style file transfers.
7689

77-
You can later unmount with `make umount-root` without using `sudo` command.
90+
You can later unmount with `make umount-root` without using `sudo` command.
91+
92+
This feature is only practical with fast (generally on local) connections.
7893

7994
### Sync target's root folder
8095

sync-config-example.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
proxy_host=aktos1
2+
dest_host=aea@localhost:7104
3+
dest_dir='~/apps/xy2-100-v3/'
4+
run_before_sync+=("make all")
5+
use_gitignore=true

sync-with-sgw.sh

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
#!/usr/bin/env bash
2+
_sdir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
3+
set -eu
4+
5+
show_help(){
6+
cat <<HELP
7+
$(basename $0) -c path/to/config
8+
9+
Synchronizes the directory that the configuration file present with \$dest_dir
10+
Always excluded paths: '.git'
11+
12+
Options:
13+
14+
--dry-run : Runs RSYNC in dry run mode.
15+
16+
Configuration (a valid BASH script):
17+
18+
proxy_host="USER@HOST:PORT" # SSH host to use as the jump server
19+
proxy_host="foo" # Same as above, use "foo" target from .ssh/config
20+
21+
dest_host="USER@HOST:PORT" # Destination host
22+
dest_host="bar" # Destination host from .ssh/config
23+
24+
dest_dir='./path/to/dest_dir/' # Notice the / at the end
25+
26+
use_gitignore=true # use .gitignore file to extract exclude dirs
27+
run_before_sync+=("path/to/hookscript") # execute those scripts before sync
28+
29+
HELP
30+
}
31+
32+
die(){
33+
>&2 echo
34+
>&2 echo "$@"
35+
exit 1
36+
}
37+
38+
help_die(){
39+
>&2 echo
40+
>&2 echo "$@"
41+
show_help
42+
exit 1
43+
}
44+
45+
# Parse command line arguments
46+
# ---------------------------
47+
# Initialize parameters
48+
config=
49+
dry_run=
50+
# ---------------------------
51+
args_backup=("$@")
52+
args=()
53+
_count=1
54+
while [ $# -gt 0 ]; do
55+
key="${1:-}"
56+
case $key in
57+
-h|-\?|--help|'')
58+
show_help # Display a usage synopsis.
59+
exit
60+
;;
61+
# --------------------------------------------------------
62+
-c|--config) shift
63+
config="$1"
64+
;;
65+
--dry-run)
66+
dry_run="--dry-run"
67+
;;
68+
# --------------------------------------------------------
69+
-*) # Handle unrecognized options
70+
help_die "Unknown option: $1"
71+
;;
72+
*) # Generate the new positional arguments: $arg1, $arg2, ... and ${args[@]}
73+
if [[ ! -z ${1:-} ]]; then
74+
declare arg$((_count++))="$1"
75+
args+=("$1")
76+
fi
77+
;;
78+
esac
79+
[[ -z ${1:-} ]] && break || shift
80+
done; set -- "${args_backup[@]-}"
81+
# Use $arg1 in place of $1, $arg2 in place of $2 and so on,
82+
# "$@" is in the original state,
83+
# use ${args[@]} for new positional arguments
84+
85+
[[ -f "$config" ]] || die "Configuration file is required."
86+
source "$config"
87+
88+
SRC_DIR="$(dirname "$config")"
89+
90+
read SGW_USERNAME SGW_HOST SGW_PORT_ON_SERVER <<< $(echo $dest_host | sed 's/@/ /' | sed 's/:/ /')
91+
[[ -z $SGW_HOST ]] && { SGW_HOST=$SGW_USERNAME; SGW_USERNAME=''; }
92+
read PROXY_USERNAME PROXY_HOST PROXY_PORT <<< $(echo $proxy_host | sed 's/@/ /' | sed 's/:/ /')
93+
[[ -z $PROXY_HOST ]] && { PROXY_HOST=$PROXY_USERNAME; PROXY_USERNAME=''; }
94+
95+
ignores=(--exclude '.git')
96+
gitignore_file="$SRC_DIR/.gitignore"
97+
if ${use_gitignore:-false} && [[ -f "$gitignore_file" ]]; then
98+
while IFS=: read -r line; do
99+
ignores+=(--exclude "$line")
100+
done < <(grep "" "$gitignore_file")
101+
fi
102+
103+
script_name="$(basename $0)"
104+
105+
echo_blue () {
106+
echo -e "\e[1;34m$*\e[0m"
107+
}
108+
109+
echo_yellow () {
110+
echo -e "\e[1;33m$*\e[0m"
111+
}
112+
113+
echo_green () {
114+
echo -e "\e[1;32m$*\e[0m"
115+
}
116+
117+
RSYNC="nice -n19 ionice -c3 rsync"
118+
119+
timestamp(){
120+
date +'%Y-%m-%d %H:%M'
121+
}
122+
123+
previous_sync_failed=false
124+
while :; do
125+
hook_failed=false
126+
for cmd in "${run_before_sync[@]}"; do
127+
echo_blue "Running hook before sync: $cmd"
128+
eval $cmd || { hook_failed=true; break; }
129+
done
130+
131+
if ! $hook_failed; then
132+
echo_blue "$(timestamp): Synchronizing..."
133+
134+
[[ -z $dry_run ]] || set -x
135+
if $RSYNC -avzhP --delete $dry_run "${ignores[@]}" \
136+
-e "ssh -A -J ${PROXY_HOST} -p ${SGW_PORT_ON_SERVER}" \
137+
"$SRC_DIR" \
138+
${SGW_USERNAME}@localhost:"${dest_dir}"; then
139+
140+
$previous_sync_failed && notify-send -u critical "$script_name Succeeded." "$(timestamp)"
141+
else
142+
period=10
143+
$previous_sync_failed || notify-send -u critical "$script_name Failed." "Retrying in $period seconds."
144+
sleep $period
145+
echo_yellow "Retrying..."
146+
previous_sync_failed=true
147+
continue
148+
fi
149+
$previous_sync_failed || notify-send "Sync done." "$(timestamp): ${dest_dir}"
150+
151+
previous_sync_failed=false
152+
else
153+
notify-send -u critical "$script_name Failed." "$cmd failed ($(timestamp))"
154+
fi
155+
156+
echo_green "Waiting for directory changes..."
157+
inotifywait -q -e modify,create,delete -r "$SRC_DIR"
158+
done

0 commit comments

Comments
 (0)