|
| 1 | +# theWatcher |
| 2 | + |
| 3 | +<!-- <h1 align="center"> |
| 4 | + <img src="#" alt="theWatcher" width="150px"> |
| 5 | +</h1> --> |
| 6 | + |
| 7 | +theWatcher is a security vulnerability collection and analysis tool. It retrieves data from multiple sources, applies AI-based classification and summarization when possible, and generates concise reports. |
| 8 | + |
| 9 | +## Features |
| 10 | + |
| 11 | +- Collect vulnerabilities from: |
| 12 | + - Full Disclosure |
| 13 | + - Exploit-DB |
| 14 | + - NIST |
| 15 | +- Summarize vulnerabilities using AI (OpenAI's Gemini-based model), grouping them by technology and highlighting trends |
| 16 | +- Filter vulnerabilities by severity |
| 17 | +- Limit the number of items collected from each source |
| 18 | + |
| 19 | +## Prerequisites |
| 20 | + |
| 21 | +- Python 3.7+ |
| 22 | +- An optional Google Generative AI API key (placed in a .env file as GEMINI_API_KEY) for AI-based classification and summarization. |
| 23 | + - You can request access to the API [here](https://aistudio.google.com). The tool does not need the paid version of the API; the free tier is sufficient and does not require a credit card. |
| 24 | + |
| 25 | +## Installation |
| 26 | + |
| 27 | +1. Clone the repository. |
| 28 | +2. (Optional) Create a virtual environment and activate it. |
| 29 | +3. Install dependencies: |
| 30 | + pip install -r requirements.txt |
| 31 | + |
| 32 | +4. (Optional) Create a .env file in the project root with the following content: |
| 33 | + GEMINI_API_KEY=YOUR_KEY_HERE |
| 34 | + |
| 35 | +## Usage |
| 36 | + |
| 37 | +Run the main script from the command line: |
| 38 | +python main.py [options] |
| 39 | + |
| 40 | +Some useful options: |
| 41 | +- -c, --collect |
| 42 | + Collect vulnerabilities from the specified sources. |
| 43 | +- -s, --summarize |
| 44 | + Generate a summary report from collected vulnerabilities. |
| 45 | +- -F, --full-scan |
| 46 | + Collect from all sources (including NIST) over the last 30 days (implies --collect and --summarize). |
| 47 | +- -Q, --quick-scan |
| 48 | + Collect from all sources over the last 7 days, limiting each source to 50 items (implies --collect and --summarize). |
| 49 | +- -m, --max-items |
| 50 | + Maximum number of vulnerabilities to retrieve per source. |
| 51 | +- -t, --type [all|sources|nist] |
| 52 | + Overall scope (all sources or just NIST or custom sources). |
| 53 | +- -S, --sources [fulldisclosure exploitdb nist] |
| 54 | + Specify which sources to query. |
| 55 | +- -N, --no-ai |
| 56 | + Disable AI-based classification and summarization. |
| 57 | +- -o, --output-dir |
| 58 | + Select output directory for saved JSON and report files. |
| 59 | + |
| 60 | +## Quick Demo |
| 61 | + |
| 62 | + |
| 63 | + |
| 64 | +Here’s a quick command to collect data from all sources for the last 15 days and generate a report: |
| 65 | + |
| 66 | +```console |
| 67 | +python main.py -c -s -d 15 --type all --max-items 50 |
| 68 | +``` |
| 69 | + |
| 70 | +## Additional Usage Examples |
| 71 | + |
| 72 | +```console |
| 73 | +# Collect vulnerabilities from multiple sources without AI |
| 74 | +python main.py --collect --sources fulldisclosure exploitdb --no-ai |
| 75 | + |
| 76 | +# Run a comprehensive scan and summarization in headless mode |
| 77 | +python main.py -F --output-dir ./reports |
| 78 | +``` |
| 79 | + |
| 80 | +### Examples |
| 81 | + |
| 82 | +1) Full scan of all sources, storing 100 items per source: |
| 83 | + python main.py --full-scan |
| 84 | + |
| 85 | +2) Quick scan: |
| 86 | + python main.py --quick-scan |
| 87 | + |
| 88 | +3) Custom scan (only Full Disclosure and Exploit-DB for the last 15 days without AI): |
| 89 | + python main.py -c -d 15 -S fulldisclosure exploitdb --no-ai |
| 90 | + |
| 91 | +## Example Report |
| 92 | + |
| 93 | +Here is an example of a generated report: |
| 94 | + |
| 95 | +[Example Report](./example/all_report.md) |
| 96 | + |
| 97 | +## Contributing |
| 98 | + |
| 99 | +Feel free to open a pull request or submit an issue if you find any bugs or want to request new features. |
| 100 | + |
| 101 | +## Development |
| 102 | + |
| 103 | +This tool is under active development. New sources, AI APIs, and features will be added continuously. |
| 104 | + |
| 105 | +## License |
| 106 | + |
| 107 | +This project is licensed under the [MIT License](LICENSE). |
0 commit comments