Initial backup fails as files are not added to commit

[bliepp]

Code of Conduct

• I promise that i have used markdown syntax to make the code more readable which helps to fix the problem and that I have tested the --fix command mentioned above without success.

What happened

After I installed Klipper-Backup according to the docs I tried to do an initial backup using ./script.sh. However, an error message appears.

What did you expect to happen

I expected it to confirm that my config has been uploaded to GitHub

How to reproduce

Install first, then run ./script.sh

Debug

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username=bliepp
github_repository=Ender3-Klipper-Config
branch_name="main"
commit_username="bliepp"
commit_email="12013202+bliepp@users.noreply.github.com"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 8
drwxr-xr-x  2 bliepp bliepp 4096 Sep 19 14:40 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 14:40 ..
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

./script.sh: line 144: /home/bliepp/config_backup/.git/config: No such file or directory
------------------------------

Initialized empty Git repository in /home/bliepp/config_backup/.git/
Skipping symbolic link: printer_data/config/mainsail.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 16
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 14:40 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 14:40 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 14:40 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 14:40 printer_data
------------------------------

error: 'printer_data/config/' does not have a commit checked out
fatal: adding files failed
On branch main

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	README.md
	printer_data/

nothing added to commit but untracked files present (use "git add" to track)
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
From https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git
error: src refspec main does not match any
error: failed to push some refs to 'https://github.com/bliepp/Ender3-Klipper-Config.git'

Additional information

I'm aware that it says "Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)". However, changing it to public didn't change anything (the message became "The GitHub repo bliepp/Ender3-Klipper-Config exists (public)" but the error didn't go away, though).

[Tylerjet]

i see towards the bottom that git prints out your token as https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git can you double check your .env file to make sure that the token is actually set?

[bliepp]

i see towards the bottom that git prints out your token as https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git can you double check your .env file to make sure that the token is actually set?

Thanks for the quick reply. Yes, it is actually set. I removed my token it from the output of `script.sh --debug" before posting.

[Tylerjet]

ahh ok, when installing did you use install.sh to configure settings or did you edit the .env manually?

[bliepp]

I used install.sh. However, it doesn't seem to be an issue with the settings but with my printer_data directory. When I change script.sh on line 253 (where files are actually added) from git add . to git add README.md everything (i.e. only the README.md) commits just fine. If I change it to git add printer_dataor git add printer_data/* it only removes the previously commited README.md but does not add the directory (however, it doesn't fail). So I guess the error is that it does not recognize printer_data as addable.

It's still strange as it at least should commit the README.md, shouldn't it?

[Tylerjet]

could you run with --debug again and post that, as i see in the initial one it didn't print out things like your .git/config file, etc... since it didn't get created until the script ran.

[Tylerjet]

Yeah my wondering is that there are some addtional git errors about pushing/accessing the remote repo so if those resolve the rest might too

[bliepp]

Okay, here's what I found:

1. If I push to an non-empty repo (e.g. with an existing file) the commit and push always succeeds, no matter if I only add the README.md via modifying line 253, only printer_data, or use the original script. It always succeeds as in something happens without an error.
2. If I push to an empty repo (i.e. freshly created without any contents) it fails if printer_data is in the list of files to add (via a dot like in the original script or by explicitly stating printer_dataas the git add argument).
3. printer_data never gets added, commited or pushed to remote, no matter if a error occurs or not. If no error occurs, printer_data is simply ignored.

Very strange

[bliepp]

could you run with --debug again and post that, as i see in the initial one it didn't print out things like your .git/config file, etc... since it didn't get created until the script ran.

Sure. Here's the output for modifying it to explicitly only add README.md to the commit:

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script2.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username="bliepp"
github_repository="Ender3-Klipper-Config"
branch_name="main"
commit_username="bliepp"
commit_email="12013202+bliepp@users.noreply.github.com"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 16
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 15:19 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 15:20 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 15:19 .git
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

[init]
    defaultBranch = main
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[user]
	name = bliepp
	email = 12013202+bliepp@users.noreply.github.com
[remote "origin"]
	url = https://********@github.com/bliepp/Ender3-Klipper-Config.git
	fetch = +refs/heads/*:refs/remotes/origin/*
------------------------------

Skipping symbolic link: printer_data/config/mainsail.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 20
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 15:20 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 15:20 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 15:20 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 15:20 printer_data
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------

[main (root-commit) 7a4fbd7] New backup from 19/09/24 - 15:20:56
 1 file changed, 4 insertions(+)
 create mode 100644 README.md
From https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 378 bytes | 378.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/bliepp/Ender3-Klipper-Config.git
 * [new branch]      main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.
bliepp@ender3:~/klipper-backup $ 

[Tylerjet]

ok and then can you do one with klipper-backup reset to the latest commit so we can see the error when it tries to add and push the printer_data

[Tylerjet]

also after line 221 can you put

echo "Backing up: $file"

that should let us see what files it actually tries to copy over to the backup_config folder

[bliepp]

ok and then can you do one with klipper-backup reset to the latest commit so we can see the error when it tries to add and push the printer_data

Oh okay now I get what you mean.

also after line 221 can you put

echo "Backing up: $file"

Sure, here it is:

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script2.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username="bliepp"
github_repository="Ender3-Klipper-Config"
branch_name="main"
commit_username="bliepp"
commit_email="12013202+bliepp@users.noreply.github.com"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 16
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 18:43 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:43 ..
drwxr-xr-x  8 bliepp bliepp 4096 Sep 19 18:43 .git
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

[init]
    defaultBranch = main
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[user]
	name = bliepp
	email = 12013202+bliepp@users.noreply.github.com
[remote "origin"]
	url = https://********@github.com/bliepp/Ender3-Klipper-Config.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main
------------------------------

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
From https://github.com/bliepp/Ender3-Klipper-Config
 * branch            main       -> FETCH_HEAD
Already up to date.
Backing up: /home/bliepp/printer_data/config/axes.cfg
Backing up: /home/bliepp/printer_data/config/bed.cfg
Backing up: /home/bliepp/printer_data/config/display.cfg
Backing up: /home/bliepp/printer_data/config/extruders.cfg
Backing up: /home/bliepp/printer_data/config/features.cfg
Backing up: /home/bliepp/printer_data/config/.git
Backing up: /home/bliepp/printer_data/config/macros.cfg
Skipping symbolic link: printer_data/config/mainsail.cfg
Backing up: /home/bliepp/printer_data/config/mcu.cfg
Backing up: /home/bliepp/printer_data/config/moonraker.conf
Backing up: /home/bliepp/printer_data/config/.moonraker.conf.bkp
Backing up: /home/bliepp/printer_data/config/printer.cfg
Backing up: /home/bliepp/printer_data/config/probe.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 20
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 18:43 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:43 ..
drwxr-xr-x  8 bliepp bliepp 4096 Sep 19 18:43 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 18:43 printer_data
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------

error: 'printer_data/config/' does not have a commit checked out
fatal: adding files failed
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	README.md
	printer_data/

nothing added to commit but untracked files present (use "git add" to track)
[main e75f0f7] New backup from 19/09/24 - 18:43:32 - No new changes pushed
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 232 bytes | 232.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/bliepp/Ender3-Klipper-Config.git
   4a196da..e75f0f7  main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.

So this means klipper-backup reads and copies the files correctly but git doesn't seem add them to the commit, right?

[Tylerjet]

okay can you do

cd ~
rm -rf config_backup

and then run script.sh again