feat(hackertarget): support Hackertarget API key via api-keys.yml (fixes #2122)

[raghavendrakari]

This pull request adds optional Hackertarget API key support so users can avoid the provider’s 50 free-scan-per-IP rate limit. When an API key is available (either via the project’s Core loader if present or through an api-keys.yml file), theHarvester will append apikey= to Hackertarget requests for hostsearch and reversedns. If no key is configured, the tool’s behavior is unchanged.

Implementation is intentionally minimal and backwards-compatible. A small helper _get_hackertarget_key() attempts to obtain a key from the canonical Core API (tries Core.load_api_keys() / Core.get_api_keys() when available) and falls back to safely parsing api-keys.yml from common locations. A second helper _append_apikey_to_url(url, apikey) appends the apikey query parameter while preserving existing parameters. SearchHackerTarget.do_search() uses these helpers to produce the final request URLs passed to AsyncFetcher.fetch_all().

The change includes unit tests that mock AsyncFetcher.fetch_all so no network requests or real API keys are required. The tests validate URL construction and response parsing both with and without a configured API key. Locally the new tests pass; run them with python -m pytest tests/test_hackertarget_apikey.py.

I kept the YAML fallback to maximize usability for contributors who do not yet expose a single canonical loader; if maintainers prefer to require the project loader only, I will remove the fallback and call the specified function exclusively. I am also happy to add a brief README note and an example api-keys.yml snippet, or to extend tests to cover invalid/expired keys if desired.

fixes (#2122)

[leebaird]

Do lines 36-38 include checking for keys located located at: ~/.theHarvester/api-keys.yml

[L1ghtn1ng]

please update where I have left comments and please also run ruff format and ruff check against your changes please

[leebaird]

Thanks for your input on this.

[raghavendrakari]

Thanks for the review - I’ve applied the requested changes and pushed them to the feature branch.

What I changed

I added ~/.theHarvester/api-keys.yml to the list of fallback locations so local user keys are discovered in addition to repo-local files.

I narrowed the YAML error handling so we only catch file I/O and YAML parse errors (OSError, yaml.YAMLError) — we no longer use a broad except Exception: that could hide other problems.

I updated the unit test to use httpx instead of requests (the project uses httpx now) and adjusted the dummy exception accordingly.

I ran style fixes (ruff format / ruff check --fix) and applied the suggested tidyups (sorted imports, removed the redundant 'r' mode on open(), etc.).

What I ran locally:

ruff format . && ruff check . — fixed the issues reported by ruff.

python -m pytest -q tests/test_hackertarget_apikey.py — tests are mocked and pass locally (3 passed).

Notes / next steps

I left the YAML fallback in place to make this easy for contributors who don’t yet expose a single Core loader. If you prefer to require the canonical Core loader only, I can remove the fallback and wire _get_hackertarget_key() to whatever Core function you prefer - tell me which one and I’ll update it.

@leebaird
@L1ghtn1ng
@KelvinDekker

[L1ghtn1ng]

I prefer the core.py way please