first commit

Author: bitdruid

.gitignore

@@ -0,0 +1,8 @@
+.venv/
+.test/
+pywaybackup/__pycache__/
+waybackup_snapshots/
+dist/
+pywaybackup.egg-info/
+build/
+```

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 bitdruid
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,79 @@
+# archive wayback downloader
+
+[![PyPI](https://img.shields.io/pypi/v/pywaybackup)](https://pypi.org/project/pywaybackup/)
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/pywaybackup)](https://pypi.org/project/pywaybackup/)
+![Release](https://img.shields.io/badge/Release-alpha-red)
+![Python Version](https://img.shields.io/badge/Python-3.6-blue)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+Downloading archived web pages from the [Wayback Machine](https://archive.org/web/).
+
+Internet-archive is a nice source for several OSINT-information. This script is a work in progress to query and fetch archived web pages.
+
+## Installation
+
+### Pip
+
+1. Install the package <br>
+   ```pip install pywaybackup```
+2. Run the script <br>
+   ```waybackup -h```
+
+### Manual
+
+1. Clone the repository <br>
+   ```git clone https://github.com/bitdruid/waybackup.git```
+2. Install <br>
+   ```pip install .```
+   - in a virtual env or use `--break-system-package`
+
+## Usage
+
+This script allows you to download content from the Wayback Machine (archive.org). You can use it to download either the latest version or all versions of web page snapshots within a specified range.
+
+### Arguments
+
+- `-h`, `--help`: Show the help message and exit.
+- `-v`, `--version`: Show the script's version.
+
+#### Required Arguments
+
+- `-u URL`, `--url URL`: The URL of the web page to download. This argument is required.
+
+#### Mode Selection (Choose One)
+
+- `-c`, `--current`: Download the latest version of each file snapshot. This option is mutually exclusive with `-f/--full`.
+- `-f`, `--full`: Download snapshots of all timestamps. This option is mutually exclusive with `-c/--current`.
+
+#### Optional Arguments
+
+- `-l`, `--list`: Only print the snapshots available within the specified range. Does not download the snapshots.
+- `-r RANGE`, `--range RANGE`: Specify the range in years for which to search and download snapshots.
+- `-o OUTPUT`, `--output OUTPUT`: The folder where downloaded files will be saved.
+
+#### Additional
+
+- `--retry [RETRY_FAILED]`: Retry failed downloads. You can specify the number of retry attempts as an integer. If no number is provided, the script will keep retrying indefinitely.
+- `--worker [AMOUNT]`: The number of worker to use for downloading (simultaneous downloads). Default is 1. Beware: Using too many worker will lead into refused connections from the Wayback Machine. Duration about 1.5 minutes.
+
+### Examples
+
+Download latest snapshot of all files:<br>
+`waybackup -u http://example.com -c`
+
+Download latest snapshot of all files with retries:<br>
+`waybackup -u http://example.com -c --retry 3`
+
+Download all snapshots sorted per timestamp with a specified range:<br>
+`waybackup -u http://example.com -f -r 5`
+
+Download all snapshots sorted per timestamp with a specified range and save to a specified folder with 3 worker:<br>
+`waybackup -u http://example.com -f -r 5 -o /home/user/Downloads/snapshots --worker 3`
+
+List available snapshots per timestamp without downloading:<br>
+`waybackup -u http://example.com -f -l`
+
+## Contributing
+
+I'm always happy for some feature requests to improve the usability of this script.
+Feel free to give suggestions and report issues. Project is still far from being perfect.

dev/pip_build.sh

@@ -0,0 +1,19 @@
+#!bin/bash
+
+# path of the script
+SCRIPT_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+TARGET_PATH="$SCRIPT_PATH/.."
+
+# check if venv is activated
+if [ -z "$VIRTUAL_ENV" ]; then
+    echo "Please activate your virtual environment"
+    exit 1
+fi
+
+# build 
+python $TARGET_PATH/setup.py sdist bdist_wheel --verbose
+python -m twine upload dist/*
+#pip install -e $TARGET_PATH
+
+# clean up
+rm -rf $TARGET_PATH/build $TARGET_PATH/dist # $TARGET_PATH/*.egg-info

dev/venv_create.sh

@@ -0,0 +1,17 @@
+#!bin/bash
+
+# path of the script
+SCRIPT_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+TARGET_PATH="$SCRIPT_PATH/.."
+echo "Preparing virtual environment in $TARGET_PATH"
+# Create a virtual environment
+if [ ! -d "..$SCRIPT_PATH/.venv" ]; then
+    python3 -m venv "$TARGET_PATH/.venv"
+fi
+
+# update pip
+"$TARGET_PATH/.venv/bin/python" -m pip install --upgrade pip
+"$TARGET_PATH/.venv/bin/python" -m pip install twine wheel
+
+# install requirements
+"$TARGET_PATH/.venv/bin/python" -m pip install -r "$TARGET_PATH/requirements.txt"

pywaybackup/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.4.2"