Root Back Up

[nichols89ben]

Hey, great script, been using it for awhile without issues. I downloaded a version in July and was having issues when calling to back up root "/" and getting source folder does not exist. I added a few lines and was able to get it working. I downloaded the most recent version and added the edits and it errored out. Question is, how can I call the root directory for back up and not get source folder does not exist. Thanks!

[basnijholt]

Hi @nichols89ben, glad you have found it useful!

Would you be able to share the full error and which lines you edited precisely?

[nichols89ben]

Hi @nichols89ben, glad you have found it useful!

Would you be able to share the full error and which lines you edited precisely?

For sure, I call/buid this from a bash script for automation and some checks, I attached that script also for referance

Error:
rsync-time-machine.py: [ERROR] Source and destination folder cannot be empty.

**Adjusments, not super familiar with python, gpt helped. I know I was going at it for awhile and the bellow edits worked since July

Line 564 from:
dest_folder = dest_folder.rstrip("/")
src_folder = src_folder.rstrip("/")
to:
if src_folder != "/":
src_folder = src_folder.rstrip("/")
if dest_folder != "/":
dest_folder = dest_folder.rstrip("/")

Line 815 added:
if src_folder != "/":

**I moved the root directory recently and clean up some of my scripts, I tried re running today and get this error now
I get it when running the 7/2023 version as well as the most recent one i downloaded a few days ago. The scripts backs everything up correctly
No error in the logs and finishes the back up, but errors out at the end. The only thing I can tell is it doesnt re link the newest back up to the latest backup link
and leaves the back up in progress marker.

rsync-time-machine.py: /alamo/alamo_jarvis/backup.inprogress already exists - the previous backup failed or was interrupted. Backup will resume from there.
rsync-time-machine.py: Previous backup found - doing incremental backup from /scripts_services/backups/jarvis
rsync-time-machine.py: Starting backup...
rsync-time-machine.py: From: //
rsync-time-machine.py: To:   /alamo/alamo_jarvis/2024-02-26-210020/
rsync-time-machine.py: Running command:
rsync-time-machine.py: rsync -D --numeric-ids --links --hard-links --one-file-system --itemize-changes --times --recursive --perms --owner --group --stats --human-readable --progress --info=progress2 --log-file '/scripts_services/backups/jarvis/logs_jarvis/2024-02-26-210020.log' --exclude-from '/scripts_services/backups/jarvis/exclusions.txt' --link-dest='/scripts_services/backups/jarvis' -- '//' '/alamo/alamo_jarvis/2024-02-26-210020/'
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/streams.py", line 524, in readline
    line = await self.readuntil(sep)
  File "/usr/lib/python3.10/asyncio/streams.py", line 602, in readuntil
    raise exceptions.LimitOverrunError(
asyncio.exceptions.LimitOverrunError: Separator is not found, and chunk exceed the limit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 930, in <module>
    main()
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 912, in main
    backup(
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 873, in backup
    log_file = start_backup(
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 781, in start_backup
    run_cmd(cmd)
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 434, in run_cmd
    return asyncio.run(async_run_cmd(cmd, ssh))
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 396, in async_run_cmd
    stdout, stderr = await asyncio.gather(
  File "/scripts_services/backups/jarvis/rsync_time_machine_jarvis/rsync_time_machine.py", line 418, in read_stream
    line = await stream.readline()
  File "/usr/lib/python3.10/asyncio/streams.py", line 533, in readline
    raise ValueError(e.args[0])

ValueError: Separator is not found, and chunk exceed the limit
rsync-time-machine.py:

**Call Script
tm_health_check.txt

[basnijholt]

Just before I saw your message I released v1.3.2 with basically the same code changes you made.

But even with that it seems to not work. I will reopen.

[nichols89ben]

Just before I saw your message I released v1.3.2 with basically the same code changes you made.

But even with that it seems to not work. I will reopen.

No worries, thanks for doing that. I haven't dug into the new issue. Might be something on my end, but did you have any thought?

[basnijholt]

Thanks for the quick reply!

Would you be able to test #54?

This is the full file: https://raw.githubusercontent.com/basnijholt/rsync-time-machine.py/26468532948b2b1d4e47d6c43ae91e7d8489f530/rsync_time_machine.py

[nichols89ben]

Thanks for the quick reply!

Would you be able to test #54?

This is the full file: https://raw.githubusercontent.com/basnijholt/rsync-time-machine.py/26468532948b2b1d4e47d6c43ae91e7d8489f530/rsync_time_machine.py

I gave it a try and its in process, it seems like its hung up on something. The back up was just done so there isnt anything to really back up. Should I delete the in progress marker and try and re run?

Log:
2024/02/26 22:04:26 [594568] building file list

Terminal output:
jarvis@jarvis:/scripts_services/backups/jarvis$ sudo ./tm_health_check.sh
Error: Running Homebrew as root is extremely dangerous and no longer supported.
As Homebrew does not drop privileges on installation you would be giving all
build scripts full access to your system.
rsync-time-machine.py: /alamo/alamo_jarvis/backup.inprogress already exists - the previous backup failed or was interrupted. Backup will resume from there.
rsync-time-machine.py: Previous backup found - doing incremental backup from /scripts_services/backups/jarvis
rsync-time-machine.py: Starting backup...
rsync-time-machine.py: From: /
rsync-time-machine.py: To: /alamo/alamo_jarvis/2024-02-26-220401/
rsync-time-machine.py: Running command:
rsync-time-machine.py: rsync -D --numeric-ids --links --hard-links --one-file-system --itemize-changes --times --recursive --perms --owner --group --stats --human-readable --progress --info=progress2 --log-file '/scripts_services/backups/jarvis/logs_jarvis/2024-02-26-220401.log' --exclude-from '/scripts_services/backups/jarvis/exclusions.txt' --link-dest='/scripts_services/backups/jarvis' -- '/' '/alamo/alamo_jarvis/2024-02-26-220401/'

Processes:
jarvis@jarvis:~$ ps aux | grep tm_health_check.sh
root 594140 0.0 0.0 14416 5632 pts/1 S+ 22:04 0:00 sudo ./tm_health_check.sh
root 594141 0.0 0.0 14416 2508 pts/3 Ss 22:04 0:00 sudo ./tm_health_check.sh
root 594142 0.0 0.0 10104 3584 pts/3 S+ 22:04 0:00 /bin/bash ./tm_health_check.sh
jarvis 601234 0.0 0.0 9220 2560 pts/8 S+ 22:12 0:00 grep --color=auto tm_health_check.sh

[nichols89ben]

Thanks for the quick reply!

Would you be able to test #54?

This is the full file: https://raw.githubusercontent.com/basnijholt/rsync-time-machine.py/26468532948b2b1d4e47d6c43ae91e7d8489f530/rsync_time_machine.py

I ended up stopping the script as it wasnt moving, deleting the marker and trying again, it behaved like it did before, it looped three times. I rebooted and tried my old script in the original directory and it ran without errors. I tried the same script with no modifications, put it in my adjusted directory and still got the same errors. It seems related to changing the directory of the scripts somehow. (the source and destination didn't change at any point)

Errors.txt

[nichols89ben]

Thanks for the quick reply!

Would you be able to test #54?

This is the full file: https://raw.githubusercontent.com/basnijholt/rsync-time-machine.py/26468532948b2b1d4e47d6c43ae91e7d8489f530/rsync_time_machine.py

Sorry for all the replies. I went document by document and directory by directory, then line by line of my script and was able to narrow it down too passing the --info=progress2 flag. Beyond my scope as too why, I use it with other rsync scripts, but removing that flag as stopped all the errors. By no means critical for me but was happy to figure it out.