Amount of found API key: {{ps_a}} Amount of WEB elements found: {{ps_w}}
Amount of exposed passwords found: {{ps_p}}
+ PageSearch process listing:
+ {{ ps_string }}
From df7b19728c36dad54cce05a38a8c445fe866d048 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Thu, 15 May 2025 22:53:35 +0300
Subject: [PATCH 07/45] Updated default report template visual
---
.../default_report_temp.html | 515 ++++++++++--------
1 file changed, 293 insertions(+), 222 deletions(-)
diff --git a/service/pdf_report_templates/default_report_temp.html b/service/pdf_report_templates/default_report_temp.html
index 5123a2b..51481f0 100644
--- a/service/pdf_report_templates/default_report_temp.html
+++ b/service/pdf_report_templates/default_report_temp.html
@@ -1,226 +1,297 @@
-
+
- OSINT report for {{ sh_domain }}
+
- Open Source Information Research Report {{org}}
-
- Table of contents 1. General scan information
- [BASIC SCAN INFO]
- 2. WHOIS information
- 3. Social medias links
- 4. Subdomains information
- 5. DNS & SSL information
- 6. Services & frameworks information
- 7. Basic pre-pentest information
- [DORKING SCAN INFO]
- 8. Dorking queries and results links (if was selected)
- [PAGESEARCH SCAN INFO]
- 9. PageSearch results (if was selected)
- [API SCAN INFO]
- 10. VirusTotal API scan results (if was selected)
- 11. SecurityTrails API scan results (if was selected)
- 12. HudsonRock API scan results (if was selected)
-
-
- GENERAL SCAN INFORMATION Total subdomains found: {{a_tsf}}
- Total social media links found: {{a_tsm}}
- Status of robots.txt extraction: {{robots_txt_result}}
- Status of sitemap.xml extraction: {{sitemap_xml_result}}
- Status of sitemap.xml links extraction: {{sitemap_links}}
- Google Dorking status: {{dorking_status}}
- PageSearch conduction: {{pagesearch_ui_mark}}
- Snapshotting conduction: {{snapshotting_ui_mark}}
- Report creation time: {{ctime}}
-
-
- WHOIS INFORMATION Domain: {{sh_domain}}
- Full URL: {{full_url}}
- IP address: {{ip_address}}
- Registrar: {{registrar}}
- Creation date: {{creation_date}}
- Expiration date: {{expiration_date}}
- Organization name: {{org}}
- Contact e-mails: {{mails}}
-
-
- SOCIAL MEDIAS SEARCH RESULTS FACEBOOK:
- {% for link in fb_links %}
- => {{ link }}
- TWITTER (+ X.com):
- {% for link in tw_links %}
- => {{ link }}
- {% for link in xcom_links %}
- => {{ link }}
- INSTAGRAM:
- {% for link in inst_links %}
- => {{ link }}
- TELEGRAM:
- {% for link in tg_links %}
- => {{ link }}
- TIKTOK:
- {% for link in tt_links %}
- => {{ link }}
- LINKEDIN:
- {% for link in li_links %}
- => {{ link }}
- VKONTAKTE:
- {% for link in vk_links %}
- => {{ link }}
- YOUTUBE:
- {% for link in yt_links %}
- => {{ link }}
- ODNOKLASSNIKI:
- {% for link in ok_links %}
- => {{ link }}
- WECHAT:
- {% for link in wc_links %}
- => {{ link }}
-
-
- SUBDOMAINS ANALYSIS RESULTS Found subdomains:
- {% for sd in subdomains %}
- => {{ sd }}
- Subdomains IP addresses:
- {% for sdip in subdomain_ip %}
- => {{ sdip }}
- Subdomains e-mails:
- {% for smails in subdomain_mails %}
- => {{ smails }}
-
-
- DNS & SSL INFORMATION (DNS) Name servers: {{name_servers}}
- (DNS) MX addresses: {{mx_records}}
- (SSL) Issuer: {{issuer}}
- (SSL) Subject: {{subject}}
- (SSL) Not before: {{notBefore}}
- (SSL) Not after: {{notAfter}}
- (SSL) Certificate name: {{commonName}}
- (SSL) Certificate serial number: {{serialNumber}}
-
-
- SERVICES & FRAMEWORKS INFORMATION Web servers:
- {% for ws in web_servers %}
- => {{ ws }}
- CMS:
- {% for cm in cms %}
- => {{ cm }}
- Used programming languages:
- {% for pl in programming_languages %}
- => {{ pl }}
- Used web frameworks:
- {% for wf in web_frameworks %}
- => {{ wf }}
- Analytics service:
- {% for analytic in analytics %}
- => {{ analytic }}
- Used JavaScript frameworks:
- {% for jsf in javascript_frameworks %}
- => {{ jsf }}
- Tags:
- {% for tag in tags %}
- => {{ tag }}
- Common Platform Enumeration:
- {% for cpe in cpes %}
- => {{ cpe }}
-
-
- BASIC PRE-PENTEST INFORMATION Open ports:
- {% for op in ports %}
- => {{ op }}
- Hostnames:
- {% for hn in hostnames %}
- => {{ hn }}
- Potential vulnerabilities:
- {% for vuln in vulns %}
- => {{ vuln }}
-
-
- DORKING SCAN INFO {{ add_dsi | safe }}
-
-
- PAGESEARCH RESULTS
- Amount of accessible subdomains: {{ps_s}}
- Amount of email addresses: {{ps_e}}
- Amount of found documents: {{ps_f}}
- Amount of found cookies: {{ps_c}}
- Amount of found API key: {{ps_a}}
- Amount of WEB elements found: {{ps_w}}
- Amount of exposed passwords found: {{ps_p}}
- PageSearch process listing:
- {{ ps_string }}
-
- VIRUSTOTAL API SCAN RESULTS
- {{ virustotal_output }}
-
- SECURITYTRAILS API SCAN RESULTS
- {{ securitytrails_output }}
-
- HUDSONROCK API SCAN RESULTS
-
{{ hudsonrock_output }}
-
- Created using DPULSE software by OSINT-TECHNOLOGIES Visit our web-pages:
-
- https://github.com/OSINT-TECHNOLOGIES https://pypi.org/project/dpulse/
+
+Open Source Information Research Report
+{{ org }}
+Table of contents
+1. General scan information
+[BASIC SCAN INFO]
+2. WHOIS information
+3. Social medias links
+4. Subdomains information
+5. DNS & SSL information
+6. Services & frameworks information
+7. Basic pre-pentest information
+[DORKING SCAN INFO]
+8. Dorking queries and results links (if was selected)
+[PAGESEARCH SCAN INFO]
+9. PageSearch results (if was selected)
+[API SCAN INFO]
+10. VirusTotal API scan results (if was selected)
+11. SecurityTrails API scan results (if was selected)
+12. HudsonRock API scan results (if was selected)
+
+GENERAL SCAN INFORMATION
+Total subdomains found: {{ a_tsf }}
+Total social media links found: {{ a_tsm }}
+Status of robots.txt extraction: {{ robots_txt_result }}
+Status of sitemap.xml extraction: {{ sitemap_xml_result }}
+Status of sitemap.xml links extraction: {{ sitemap_links }}
+Google Dorking status: {{ dorking_status }}
+PageSearch conduction: {{ pagesearch_ui_mark }}
+Snapshotting conduction: {{ snapshotting_ui_mark }}
+Report creation time: {{ ctime }}
+
+WHOIS INFORMATION
+Domain: {{ sh_domain }}
+Full URL: {{ full_url }}
+IP address: {{ ip_address }}
+Registrar: {{ registrar }}
+Creation date: {{ creation_date }}
+Expiration date: {{ expiration_date }}
+Organization name: {{ org }}
+Contact e-mails: {{ mails }}
+
+SOCIAL MEDIAS SEARCH RESULTS
+FACEBOOK:
+
+ {% for link in fb_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TWITTER (+ X.com):
+
+ {% for link in tw_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+ {% for link in xcom_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+INSTAGRAM:
+
+ {% for link in inst_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TELEGRAM:
+
+ {% for link in tg_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TIKTOK:
+
+ {% for link in tt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+LINKEDIN:
+
+ {% for link in li_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+VKONTAKTE:
+
+ {% for link in vk_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+YOUTUBE:
+
+ {% for link in yt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+ODNOKLASSNIKI:
+
+ {% for link in ok_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+WECHAT:
+
+ {% for link in wc_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+SUBDOMAINS ANALYSIS RESULTS
+Found subdomains:
+
+ {% for sd in subdomains %}
+ ⇒ {{ sd }}
+ {% endfor %}
+
+
+Subdomains IP addresses:
+
+ {% for sdip in subdomain_ip %}
+ ⇒ {{ sdip }}
+ {% endfor %}
+
+
+Subdomains e-mails:
+
+ {% for smails in subdomain_mails %}
+ ⇒ {{ smails }}
+ {% endfor %}
+
+
+DNS & SSL INFORMATION
+(DNS) Name servers: {{ name_servers }}
+(DNS) MX addresses: {{ mx_records }}
+(SSL) Issuer: {{ issuer }}
+(SSL) Subject: {{ subject }}
+(SSL) Not before: {{ notBefore }}
+(SSL) Not after: {{ notAfter }}
+(SSL) Certificate name: {{ commonName }}
+(SSL) Certificate serial number: {{ serialNumber }}
+
+SERVICES & FRAMEWORKS INFORMATION
+Web servers:
+
+ {% for ws in web_servers %}
+ ⇒ {{ ws }}
+ {% endfor %}
+
+CMS:
+
+ {% for cm in cms %}
+ ⇒ {{ cm }}
+ {% endfor %}
+
+Used programming languages:
+
+ {% for pl in programming_languages %}
+ ⇒ {{ pl }}
+ {% endfor %}
+
+Used web frameworks:
+
+ {% for wf in web_frameworks %}
+ ⇒ {{ wf }}
+ {% endfor %}
+
+Analytics service:
+
+ {% for analytic in analytics %}
+ ⇒ {{ analytic }}
+ {% endfor %}
+
+Used JavaScript frameworks:
+
+ {% for jsf in javascript_frameworks %}
+ ⇒ {{ jsf }}
+ {% endfor %}
+
+Tags:
+
+ {% for tag in tags %}
+ ⇒ {{ tag }}
+ {% endfor %}
+
+Common Platform Enumeration:
+
+ {% for cpe in cpes %}
+ ⇒ {{ cpe }}
+ {% endfor %}
+
+
+BASIC PRE-PENTEST INFORMATION
+Open ports:
+
+ {% for op in ports %}
+ ⇒ {{ op }}
+ {% endfor %}
+
+Hostnames:
+
+ {% for hn in hostnames %}
+ ⇒ {{ hn }}
+ {% endfor %}
+
+Potential vulnerabilities:
+
+ {% for vuln in vulns %}
+ ⇒ {{ vuln }}
+ {% endfor %}
+
+
+DORKING SCAN INFO
+{{ add_dsi | safe }}
+
+PAGESEARCH RESULTS
+Amount of accessible subdomains: {{ ps_s }}
+Amount of email addresses: {{ ps_e }}
+Amount of found documents: {{ ps_f }}
+Amount of found cookies: {{ ps_c }}
+Amount of found API key: {{ ps_a }}
+Amount of WEB elements found: {{ ps_w }}
+Amount of exposed passwords found: {{ ps_p }}
+PageSearch process listing:
+{{ ps_string }}
+
+VIRUSTOTAL API SCAN RESULTS
+{{ virustotal_output }}
+
+SECURITYTRAILS API SCAN RESULTS
+{{ securitytrails_output }}
+
+HUDSONROCK API SCAN RESULTS
+{{ hudsonrock_output }}
+
+Created using DPULSE software by OSINT-TECHNOLOGIES
+
+Visit our web-pages:
+
+ https://github.com/OSINT-TECHNOLOGIES
+ https://pypi.org/project/dpulse/
+
From 5eb4c95e267eb9eeb9f79a1ecb74fa4da1db3d5d Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Fri, 16 May 2025 01:58:00 +0300
Subject: [PATCH 08/45] Fixed error with UnboundVariable (ps_string) when PS is
not activated
---
datagather_modules/data_assembler.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/datagather_modules/data_assembler.py b/datagather_modules/data_assembler.py
index 1c88def..ad6ceff 100644
--- a/datagather_modules/data_assembler.py
+++ b/datagather_modules/data_assembler.py
@@ -225,11 +225,13 @@ def data_gathering(self, short_domain, url, report_file_type, pagesearch_flag, k
print(Fore.RED + "Cant start PageSearch because no subdomains were detected")
ps_emails_return = ""
accessible_subdomains = files_counter = cookies_counter = api_keys_counter = website_elements_counter = exposed_passwords_counter = total_links_counter = accessed_links_counter = emails_amount = 'No results because no subdomains were found'
+ ps_string = 'No PageSearch listing provided because no subdomains were found'
keywords_messages_list = ['No data was gathered because no subdomains were found']
pass
elif pagesearch_flag.lower() == 'n':
accessible_subdomains = files_counter = cookies_counter = api_keys_counter = website_elements_counter = exposed_passwords_counter = total_links_counter = accessed_links_counter = emails_amount = keywords_messages_list = "No results because user did not selected PageSearch for this scan"
ps_emails_return = ""
+ ps_string = 'No PageSearch listing provided because user did not selected PageSearch mode for this scan'
pass
if dorking_flag == 'n':
From f1813adf9e8d6cc366e416ecc9bc963dd0b76367 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Fri, 16 May 2025 15:45:09 +0300
Subject: [PATCH 09/45] Reworked Dorking function, moved from MS to selenium
---
dorking/dorking_handler.py | 116 +++++++++++++++++++++++++++----------
1 file changed, 87 insertions(+), 29 deletions(-)
diff --git a/dorking/dorking_handler.py b/dorking/dorking_handler.py
index 0ff444c..7de15bf 100644
--- a/dorking/dorking_handler.py
+++ b/dorking/dorking_handler.py
@@ -1,10 +1,20 @@
import sys
+from random import random
+
sys.path.append('service')
from config_processing import read_config
from logs_processing import logging
from ua_rotator import user_agent_rotator
from proxies_rotator import proxies_rotator
+import random
+import logging
+from colorama import Fore, Style
+import undetected_chromedriver as uc
+from selenium.webdriver.common.by import By
+from selenium.webdriver.common.keys import Keys
+from config_processing import read_config
+
try:
import requests.exceptions
from colorama import Fore, Style
@@ -27,44 +37,92 @@ def proxy_transfer():
working_proxies = proxies_rotator.check_proxies(proxies_list)
return proxy_flag, working_proxies
-def solid_google_dorking(query, dorking_delay, delay_step, proxy_flag, proxies_list, pages=100):
+def solid_google_dorking(query, proxy_flag, proxies_list, pages=1):
+ result_query = []
+ request_count = 0
try:
- browser = mechanicalsoup.StatefulBrowser()
+ config_values=read_config()
+ options = uc.ChromeOptions()
+ options.binary_location = r"{}".format(config_values['dorking_browser'])
+ options.add_argument("--headless=new")
+ options.add_argument("--no-sandbox")
+ options.add_argument("--disable-dev-shm-usage")
+ options.add_argument("--disable-blink-features=AutomationControlled")
+ options.add_argument("--disable-infobars")
+ options.add_argument("--disable-extensions")
+ options.add_argument(f"user-agent={user_agent_rotator.get_random_user_agent()}")
if proxy_flag == 1:
- browser.session.proxies = proxies_rotator.get_random_proxy(proxies_list)
- else:
- pass
- browser.open("https://www.google.com/")
- browser.select_form('form[action="/search"]')
- browser["q"] = str(query)
- browser.submit_selected(btnName="btnG")
- result_query = []
- request_count = 0
+ proxy = proxies_rotator.get_random_proxy(proxies_list)
+ options.add_argument(f'--proxy-server={proxy["http"]}')
+ driver = uc.Chrome(options=options)
for page in range(pages):
try:
- for link in browser.links():
- target = link.attrs['href']
- if (target.startswith('/url?') and not target.startswith("/url?q=http://webcache.googleusercontent.com")):
- target = re.sub(r"^/url\?q=([^&]*)&.*", r"\1", target)
- result_query.append(target)
+ driver.get("https://www.google.com")
+ time.sleep(random.uniform(2, 4))
+ try:
+ accepted = False
+ try:
+ accept_btn = driver.find_element(By.XPATH, '//button[contains(text(), "Принять все") or contains(text(), "Accept all")]')
+ driver.execute_script("arguments[0].click();", accept_btn)
+ print(Fore.GREEN + 'Pressed "Accept all" button!' + Style.RESET_ALL)
+ accepted = True
+ time.sleep(random.uniform(2, 3))
+ except:
+ pass
+ if not accepted:
+ iframes = driver.find_elements(By.TAG_NAME, "iframe")
+ for iframe in iframes:
+ driver.switch_to.frame(iframe)
+ try:
+ accept_btn = driver.find_element(By.XPATH, '//button[contains(text(), "Принять все") or contains(text(), "Accept all")]')
+ driver.execute_script("arguments[0].click();", accept_btn)
+ print(Fore.GREEN + 'Pressed "Accept all" button!' + Style.RESET_ALL)
+ accepted = True
+ driver.switch_to.default_content()
+ time.sleep(random.uniform(2, 3))
+ break
+ except:
+ driver.switch_to.default_content()
+ continue
+ driver.switch_to.default_content()
+ if not accepted:
+ print(Fore.GREEN + "Google TOS button was not found. Seems good..." + Style.RESET_ALL)
+ except Exception:
+ print(Fore.RED + f'Error with pressing "Accept all" button. Closing...' + Style.RESET_ALL)
+ driver.save_screenshot("consent_error.png")
+ driver.switch_to.default_content()
+ search_box = driver.find_element(By.NAME, "q")
+ for char in query:
+ search_box.send_keys(char)
+ time.sleep(random.uniform(0.05, 0.2))
+ time.sleep(random.uniform(0.5, 1.2))
+ search_box.send_keys(Keys.RETURN)
+ time.sleep(random.uniform(2.5, 4))
+ links = driver.find_elements(By.CSS_SELECTOR, 'a')
+ for link in links:
+ href = link.get_attribute('href')
+ if href and href.startswith('http') and 'google.' not in href and 'webcache.googleusercontent.com' not in href:
+ result_query.append(href)
request_count += 1
- if request_count % delay_step == 0:
- time.sleep(dorking_delay)
- browser.session.headers['User-Agent'] = user_agent_rotator.get_random_user_agent()
- browser.follow_link(nr=page + 1)
- except mechanicalsoup.LinkNotFoundError:
- break
+ try:
+ next_button = driver.find_element(By.ID, 'pnnext')
+ next_button.click()
+ time.sleep(random.uniform(2, 3))
+ except:
+ break
except Exception as e:
- logging.error(f'DORKING PROCESSING: ERROR. REASON: {e}')
- del result_query[-2:]
+ logging.error(f'DORKING PROCESSING (SELENIUM): ERROR. REASON: {e}')
+ continue
+ driver.quit()
+ if len(result_query) >= 2:
+ del result_query[-2:]
return result_query
- except requests.exceptions.ConnectionError as e:
- print(Fore.RED + "Error while establishing connection with domain. No results will appear. See journal for details" + Style.RESET_ALL)
- logging.error(f'DORKING PROCESSING: ERROR. REASON: {e}')
except Exception as e:
logging.error(f'DORKING PROCESSING: ERROR. REASON: {e}')
+ print(Fore.RED + "Error while running Selenium dorking. See journal for details." + Style.RESET_ALL)
+ return []
-def save_results_to_txt(folderpath, table, queries, pages=10):
+def save_results_to_txt(folderpath, table, queries, pages=1):
try:
config_values = read_config()
dorking_delay = int(config_values['dorking_delay (secs)'])
@@ -80,7 +138,7 @@ def save_results_to_txt(folderpath, table, queries, pages=10):
for i, query in enumerate(queries, start=1):
f.write(f"QUERY #{i}: {query}\n")
try:
- results = solid_google_dorking(query, dorking_delay, delay_step, proxy_flag, proxies_list, pages)
+ results = solid_google_dorking(query, proxy_flag, proxies_list, pages)
if not results:
f.write("=> NO RESULT FOUND\n")
total_results.append((query, 0))
From d8b4191478f1bcf9730d5313388ae792ac4def3f Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Fri, 16 May 2025 15:45:44 +0300
Subject: [PATCH 10/45] Added new field for Dorking browser's path
---
service/config_processing.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/service/config_processing.py b/service/config_processing.py
index 49e0fa1..0dd6177 100644
--- a/service/config_processing.py
+++ b/service/config_processing.py
@@ -30,14 +30,13 @@ def create_config():
config['HTML_REPORTING'] = {'template': 'default'}
config['LOGGING'] = {'log_level': 'info'}
config['CLI VISUAL'] = {'preview_color': 'red', 'font': 'slant'}
- config['DORKING'] = {'dorking_delay (secs)': '2', 'delay_step': '5'}
+ config['DORKING'] = {'dorking_delay (secs)': '2', 'delay_step': '5', 'full_path_to_browser': r'path\to\browser\for\dorking'}
config['SNAPSHOTTING'] = {'installed_browser': 'firefox', 'opera_browser_path': 'None', 'wayback_retries': '3', 'wayback_req_pause': '2'}
config['USER-AGENTS'] = {}
for i, agent in enumerate(basic_user_agents):
config['USER-AGENTS'][f'agent_{i + 1}'] = agent
config['PROXIES'] = {'proxies_file_path': 'NONE'}
-
with open('service//config.ini', 'w') as configfile:
config.write(configfile)
@@ -60,7 +59,7 @@ def read_config():
wayback_retries_amount = config.get('SNAPSHOTTING', 'wayback_retries')
wayback_requests_pause = config.get('SNAPSHOTTING', 'wayback_req_pause')
html_report_template = config.get('HTML_REPORTING', 'template')
-
+ dorking_browser = config.get('DORKING', 'full_path_to_browser')
config_values = {
'logging_level': log_level,
@@ -74,7 +73,8 @@ def read_config():
'opera_browser_path': opera_browser_path,
'wayback_retries_amount': wayback_retries_amount,
'wayback_requests_pause': wayback_requests_pause,
- 'template': html_report_template
+ 'template': html_report_template,
+ 'dorking_browser': dorking_browser
}
return config_values
From ba7ee8e0e65e3c1fa37e85e5a14b4089eb406b4a Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:35:32 +0300
Subject: [PATCH 11/45] Delete
service/pdf_report_templates/paragraph_report_template.html
---
.../paragraph_report_template.html | 154 ------------------
1 file changed, 154 deletions(-)
delete mode 100644 service/pdf_report_templates/paragraph_report_template.html
diff --git a/service/pdf_report_templates/paragraph_report_template.html b/service/pdf_report_templates/paragraph_report_template.html
deleted file mode 100644
index cf966ab..0000000
--- a/service/pdf_report_templates/paragraph_report_template.html
+++ /dev/null
@@ -1,154 +0,0 @@
-
-
-
- Open Source Research Report
-{{org}}
-Table of Contents
-
-
1. General scan information
-
2. WHOIS information
-
3. Social medias links
-
4. Subdomains information
-
5. DNS & SSL information
-
6. Services & frameworks
-
7. Basic pre-pentest information
-
8. Dorking scan info
-
9. PageSearch results
-
10. API scan results
-
GENERAL SCAN INFO
-
-
Total subdomains: {{a_tsf}}
-
Total social media links: {{a_tsm}}
-
Status of robots.txt: {{robots_txt_result}}
-
Status of sitemap.xml: {{sitemap_xml_result}}
-
Status of sitemap links: {{sitemap_links}}
-
Google Dorking: {{dorking_status}}
-
PageSearch: {{pagesearch_ui_mark}}
-
Snapshotting: {{snapshotting_ui_mark}}
-
Report time: {{ctime}}
-
WHOIS INFORMATION
-
-
Domain: {{sh_domain}}
-
Full URL: {{full_url}}
-
IP address: {{ip_address}}
-
Registrar: {{registrar}}
-
Creation date: {{creation_date}}
-
Expiration date: {{expiration_date}}
-
Organization name: {{org}}
-
Contact e-mails: {{mails}}
-
SOCIAL MEDIAS SEARCH RESULTS
-
-
FACEBOOK:
{% for link in fb_links %}⇒ {{ link }} {% endfor %}
-
TWITTER (+ X.com):
{% for link in tw_links %}⇒ {{ link }} {% endfor %}{% for link in xcom_links %}⇒ {{ link }} {% endfor %}
-
INSTAGRAM:
{% for link in inst_links %}⇒ {{ link }} {% endfor %}
-
TELEGRAM:
{% for link in tg_links %}⇒ {{ link }} {% endfor %}
-
TIKTOK:
{% for link in tt_links %}⇒ {{ link }} {% endfor %}
-
LINKEDIN:
{% for link in li_links %}⇒ {{ link }} {% endfor %}
-
VKONTAKTE:
{% for link in vk_links %}⇒ {{ link }} {% endfor %}
-
YOUTUBE:
{% for link in yt_links %}⇒ {{ link }} {% endfor %}
-
ODNOKLASSNIKI:
{% for link in ok_links %}⇒ {{ link }} {% endfor %}
-
WECHAT:
{% for link in wc_links %}⇒ {{ link }} {% endfor %}
-
SUBDOMAINS ANALYSIS RESULTS
-
-
Found subdomains:
{% for sd in subdomains %}⇒ {{ sd }} {% endfor %}
-
Subdomains IP addresses:
{% for sdip in subdomain_ip %}⇒ {{ sdip }} {% endfor %}
-
Subdomains e-mails:
{% for smails in subdomain_mails %}⇒ {{ smails }} {% endfor %}
-
DNS & SSL INFORMATION
-
-
(DNS) Name servers: {{name_servers}}
-
(DNS) MX addresses: {{mx_records}}
-
(SSL) Issuer: {{issuer}}
-
(SSL) Subject: {{subject}}
-
(SSL) Not before: {{notBefore}}
-
(SSL) Not after: {{notAfter}}
-
(SSL) Certificate name: {{commonName}}
-
(SSL) Certificate serial number: {{serialNumber}}
-
SERVICES & FRAMEWORKS INFORMATION
-
-
Web servers:
{% for ws in web_servers %}⇒ {{ ws }} {% endfor %}
-
CMS:
{% for cm in cms %}⇒ {{ cm }} {% endfor %}
-
Used programming languages:
{% for pl in programming_languages %}⇒ {{ pl }} {% endfor %}
-
Used web frameworks:
{% for wf in web_frameworks %}⇒ {{ wf }} {% endfor %}
-
Analytics service:
{% for analytic in analytics %}⇒ {{ analytic }} {% endfor %}
-
Used JavaScript frameworks:
{% for jsf in javascript_frameworks %}⇒ {{ jsf }} {% endfor %}
-
Tags:
{% for tag in tags %}⇒ {{ tag }} {% endfor %}
-
Common Platform Enumeration:
{% for cpe in cpes %}⇒ {{ cpe }} {% endfor %}
-
BASIC PRE-PENTEST INFORMATION
-
-
Open ports:
{% for op in ports %}⇒ {{ op }} {% endfor %}
-
Hostnames:
{% for hn in hostnames %}⇒ {{ hn }} {% endfor %}
-
Potential vulnerabilities:
{% for vuln in vulns %}⇒ {{ vuln }} {% endfor %}
-
DORKING SCAN INFO
-
-PAGESEARCH RESULTS
-
-
Amount of accessible subdomains: {{ps_s}}
-
Amount of email addresses: {{ps_e}}
-
Amount of found documents: {{ps_f}}
-
Amount of found cookies: {{ps_c}}
-
Amount of found API key: {{ps_a}}
-
Amount of WEB elements found: {{ps_w}}
-
Amount of exposed passwords found: {{ps_p}}
-
VIRUSTOTAL API SCAN RESULTS
-
-SECURITYTRAILS API SCAN RESULTS
-{{ securitytrails_output }} HUDSONROCK API SCAN RESULTS
-
-Created by DPULSE (OSINT-TECHNOLOGIES)
-
-
-
-
\ No newline at end of file
From f0f72c43344dd453b8668fa6c801f252a89d22bf Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:35:40 +0300
Subject: [PATCH 12/45] Delete
service/pdf_report_templates/compromise_report_template.html
---
.../compromise_report_template.html | 120 ------------------
1 file changed, 120 deletions(-)
delete mode 100644 service/pdf_report_templates/compromise_report_template.html
diff --git a/service/pdf_report_templates/compromise_report_template.html b/service/pdf_report_templates/compromise_report_template.html
deleted file mode 100644
index 0b7b26b..0000000
--- a/service/pdf_report_templates/compromise_report_template.html
+++ /dev/null
@@ -1,120 +0,0 @@
-
-
-
- OPEN SOURCE RESEARCH REPORT
-Organization: {{org}}
-TABLE OF CONTENTS
-
-
1. General scan information
-
2. WHOIS information
-
3. Social medias links
-
4. Subdomains information
-
5. DNS & SSL information
-
6. Services & frameworks
-
7. Basic pre-pentest information
-
8. Dorking scan info
-
9. PageSearch results
-
10. API scan results
-
GENERAL SCAN INFO
-
-
Total subdomains: {{a_tsf}}
-
Social media links: {{a_tsm}}
-
Robots.txt: {{robots_txt_result}}
-Sitemap.xml: {{sitemap_xml_result}}
-Dorking: {{dorking_status}}
-
WHOIS INFORMATION
-
-
Domain: {{sh_domain}} URL: {{full_url}}
-
IP: {{ip_address}}
-Registrar: {{registrar}}
-Dates: {{creation_date}} → {{expiration_date}}
-
SOCIAL MEDIAS
-
-
Facebook:
{% for l in fb_links %}⇒ {{ l }}{% endfor %}
-
Twitter/X:
{% for l in tw_links+xcom_links %}⇒ {{ l }}{% endfor %}
-
Instagram:
{% for l in inst_links %}⇒ {{ l }}{% endfor %}
-
SUBDOMAINS
-
-
Found subdomains:
{% for sd in subdomains %}⇒ {{ sd }}{% endfor %}
-
IPs:
{% for sdip in subdomain_ip %}⇒ {{ sdip }}{% endfor %}
-
DNS/SSL
-
-
NS: {{name_servers}}
-MX: {{mx_records}}
-SSL Issuer: {{issuer}}
-NotBefore: {{notBefore}}
-NotAfter: {{notAfter}}
-
SERVICES
-
-
Web servers:
{% for ws in web_servers %}⇒ {{ ws }}{% endfor %}
-
CMS:
{% for cm in cms %}⇒ {{ cm }}{% endfor %}
-
Languages:
{% for pl in programming_languages %}⇒ {{ pl }}{% endfor %}
-
BASIC PRE-PENTEST
-
-
Open ports:
{% for op in ports %}⇒ {{ op }}{% endfor %}
-
Vulnerabilities:
{% for vuln in vulns %}⇒ {{ vuln }}{% endfor %}
-
DORKING SCAN
-
-PAGESEARCH
-
-
Subdomains: {{ps_s}}
-Emails: {{ps_e}}
-Documents: {{ps_f}}
-…
-
VIRUSTOTAL
-
-SECURITYTRAILS
-{{ securitytrails_output }} HUDSONROCK
-
-Created by DPULSE (OSINT-TECHNOLOGIES)
-
- GitHub |
- PyPI
-
-
-
-
\ No newline at end of file
From c214d4952c2662ada2f80ae9391533ecf770f1f6 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:35:48 +0300
Subject: [PATCH 13/45] Delete
service/pdf_report_templates/monospaced_report_template.html
---
.../monospaced_report_template.html | 174 ------------------
1 file changed, 174 deletions(-)
delete mode 100644 service/pdf_report_templates/monospaced_report_template.html
diff --git a/service/pdf_report_templates/monospaced_report_template.html b/service/pdf_report_templates/monospaced_report_template.html
deleted file mode 100644
index e3521a4..0000000
--- a/service/pdf_report_templates/monospaced_report_template.html
+++ /dev/null
@@ -1,174 +0,0 @@
-
-
-
- OPEN SOURCE RESEARCH REPORT
-Organization: {{org}}
-TABLE OF CONTENTS
-
-1. General info
-2. WHOIS
-3. Social medias
-4. Subdomains
-5. DNS/SSL
-6. Services
-7. Pre-pentest
-8. Dorking
-9. PageSearch
-10. APIs
-
-GENERAL SCAN INFO
-
-Subdomains: {{a_tsf}}
-Social: {{a_tsm}}
-Robots.txt: {{robots_txt_result}}
-Sitemap.xml: {{sitemap_xml_result}}
-Sitemap links: {{sitemap_links}}
-Dorking: {{dorking_status}}
-PageSearch: {{pagesearch_ui_mark}}
-Snapshotting: {{snapshotting_ui_mark}}
-Report time: {{ctime}}
-
-WHOIS INFORMATION
-
-Domain: {{sh_domain}}
-URL: {{full_url}}
-IP: {{ip_address}}
-Registrar: {{registrar}}
-Created: {{creation_date}}
-Expires: {{expiration_date}}
-Emails: {{mails}}
-
-SOCIAL MEDIAS SEARCH RESULTS
-
-FACEBOOK:
-{% for link in fb_links %}⇒ {{ link }}{% endfor %}
-TWITTER/X:
-{% for link in tw_links+xcom_links %}⇒ {{ link }}{% endfor %}
-INSTAGRAM:
-{% for link in inst_links %}⇒ {{ link }}{% endfor %}
-TELEGRAM:
-{% for link in tg_links %}⇒ {{ link }}{% endfor %}
-TIKTOK:
-{% for link in tt_links %}⇒ {{ link }}{% endfor %}
-LINKEDIN:
-{% for link in li_links %}⇒ {{ link }}{% endfor %}
-VKONTAKTE:
-{% for link in vk_links %}⇒ {{ link }}{% endfor %}
-YOUTUBE:
-{% for link in yt_links %}⇒ {{ link }}{% endfor %}
-ODNOKLASSNIKI:
-{% for link in ok_links %}⇒ {{ link }}{% endfor %}
-WECHAT:
-{% for link in wc_links %}⇒ {{ link }}{% endfor %}
-
-SUBDOMAINS ANALYSIS RESULTS
-
-FOUND SUBDOMAINS:
-{% for sd in subdomains %}⇒ {{ sd }}{% endfor %}
-IPs:
-{% for sdip in subdomain_ip %}⇒ {{ sdip }}{% endfor %}
-Emails:
-{% for smails in subdomain_mails %}⇒ {{ smails }}{% endfor %}
-
-DNS & SSL INFORMATION
-
-NAME SERVERS: {{name_servers}}
-MX RECORDS: {{mx_records}}
-SSL ISSUER: {{issuer}}
-SUBJECT: {{subject}}
-NOT BEFORE: {{notBefore}}
-NOT AFTER: {{notAfter}}
-COMMON NAME: {{commonName}}
-SERIAL: {{serialNumber}}
-
-SERVICES & FRAMEWORKS INFORMATION
-
-WEB SERVERS:
-{% for ws in web_servers %}⇒ {{ ws }}{% endfor %}
-CMS:
-{% for cm in cms %}⇒ {{ cm }}{% endfor %}
-PROGRAMMING LANGUAGES:
-{% for pl in programming_languages %}⇒ {{ pl }}{% endfor %}
-WEB FRAMEWORKS:
-{% for wf in web_frameworks %}⇒ {{ wf }}{% endfor %}
-ANALYTICS:
-{% for analytic in analytics %}⇒ {{ analytic }}{% endfor %}
-JS FRAMEWORKS:
-{% for jsf in javascript_frameworks %}⇒ {{ jsf }}{% endfor %}
-TAGS:
-{% for tag in tags %}⇒ {{ tag }}{% endfor %}
-CPE:
-{% for cpe in cpes %}⇒ {{ cpe }}{% endfor %}
-
-BASIC PRE-PENTEST INFORMATION
-
-OPEN PORTS:
-{% for op in ports %}⇒ {{ op }}{% endfor %}
-HOSTNAMES:
-{% for hn in hostnames %}⇒ {{ hn }}{% endfor %}
-POTENTIAL VULNERABILITIES:
-{% for vuln in vulns %}⇒ {{ vuln }}{% endfor %}
-
-DORKING SCAN INFO
-{{ add_dsi | safe }}
-PAGESEARCH RESULTS
-
-SUBDOMAINS FOUND: {{ps_s}}
-EMAILS FOUND: {{ps_e}}
-DOCUMENTS: {{ps_f}}
-COOKIES: {{ps_c}}
-API KEYS: {{ps_a}}
-WEB ELEMENTS: {{ps_w}}
-PASSWORDS: {{ps_p}}
-
-VIRUSTOTAL API SCAN RESULTS
-{{ virustotal_output }}
-SECURITYTRAILS API SCAN RESULTS
-{{ securitytrails_output }}
-HUDSONROCK API SCAN RESULTS
-{{ hudsonrock_output }}
-
-Created by DPULSE (OSINT-TECHNOLOGIES)
-GitHub: https://github.com/OSINT-TECHNOLOGIES
-PyPI: https://pypi.org/project/dpulse/
-
-
-
-
\ No newline at end of file
From a008fd9b4f433c9da29616e12bd7b76a5e1965fd Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:41:54 +0300
Subject: [PATCH 14/45] Added config values support for HTML template selection
---
reporting_modules/html_report_creation.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/reporting_modules/html_report_creation.py b/reporting_modules/html_report_creation.py
index 4a304a7..8fc9932 100644
--- a/reporting_modules/html_report_creation.py
+++ b/reporting_modules/html_report_creation.py
@@ -10,6 +10,7 @@
from api_hudsonrock import hudsonrock_html_prep
from api_virustotal import virustotal_html_prep
from api_securitytrails import securitytrails_html_prep
+from config_processing import read_config
try:
from datetime import datetime
@@ -92,7 +93,6 @@ def report_assembling(short_domain, url, case_comment, data_array, report_info_a
virustotal_output = virustotal_html_prep(virustotal_output)
securitytrails_output = securitytrails_html_prep(securitytrails_output)
- pdf_templates_path = 'service//pdf_report_templates'
if len(ps_emails_return) > 0:
subdomain_mails += ps_emails_return
@@ -116,7 +116,13 @@ def report_assembling(short_domain, url, case_comment, data_array, report_info_a
new_emails = email.split(', ')
subdomain_mails_cleaned.extend(new_emails)
- template_path = pdf_templates_path + '//default_report_temp.html'
+ pdf_templates_path = 'service//pdf_report_templates'
+ config_values = read_config()
+ selected_template = config_values['template']
+ if selected_template.lower() == 'modern':
+ template_path = pdf_templates_path + '//modern_report_template.html'
+ elif selected_template.lower() == 'legacy':
+ template_path = pdf_templates_path + '//legacy_report_template.html'
dorking_results_path = report_folder + '//04-dorking_results.txt'
if os.path.isfile(dorking_results_path):
with open(dorking_results_path, 'r') as f:
From babe4ae099f0fc402983e763de2a534a5e2275f4 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:42:45 +0300
Subject: [PATCH 15/45] Added values for HTML report style selection
---
service/config_processing.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/service/config_processing.py b/service/config_processing.py
index 0dd6177..3801368 100644
--- a/service/config_processing.py
+++ b/service/config_processing.py
@@ -27,7 +27,7 @@ def create_config():
]
config = configparser.ConfigParser()
- config['HTML_REPORTING'] = {'template': 'default'}
+ config['HTML_REPORTING'] = {'template': 'modern'}
config['LOGGING'] = {'log_level': 'info'}
config['CLI VISUAL'] = {'preview_color': 'red', 'font': 'slant'}
config['DORKING'] = {'dorking_delay (secs)': '2', 'delay_step': '5', 'full_path_to_browser': r'path\to\browser\for\dorking'}
From a39dfb0e64cc056ee364413562f01c95d8efe7e5 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:49:02 +0300
Subject: [PATCH 16/45] Remaked Default report template into Legacy
---
.../legacy_report_template.html | 297 ++++++++++++++++++
1 file changed, 297 insertions(+)
create mode 100644 service/pdf_report_templates/legacy_report_template.html
diff --git a/service/pdf_report_templates/legacy_report_template.html b/service/pdf_report_templates/legacy_report_template.html
new file mode 100644
index 0000000..51481f0
--- /dev/null
+++ b/service/pdf_report_templates/legacy_report_template.html
@@ -0,0 +1,297 @@
+
+
+
+ OSINT report for {{ sh_domain }}
+
+
+
+
+Open Source Information Research Report
+{{ org }}
+Table of contents
+1. General scan information
+[BASIC SCAN INFO]
+2. WHOIS information
+3. Social medias links
+4. Subdomains information
+5. DNS & SSL information
+6. Services & frameworks information
+7. Basic pre-pentest information
+[DORKING SCAN INFO]
+8. Dorking queries and results links (if was selected)
+[PAGESEARCH SCAN INFO]
+9. PageSearch results (if was selected)
+[API SCAN INFO]
+10. VirusTotal API scan results (if was selected)
+11. SecurityTrails API scan results (if was selected)
+12. HudsonRock API scan results (if was selected)
+
+GENERAL SCAN INFORMATION
+Total subdomains found: {{ a_tsf }}
+Total social media links found: {{ a_tsm }}
+Status of robots.txt extraction: {{ robots_txt_result }}
+Status of sitemap.xml extraction: {{ sitemap_xml_result }}
+Status of sitemap.xml links extraction: {{ sitemap_links }}
+Google Dorking status: {{ dorking_status }}
+PageSearch conduction: {{ pagesearch_ui_mark }}
+Snapshotting conduction: {{ snapshotting_ui_mark }}
+Report creation time: {{ ctime }}
+
+WHOIS INFORMATION
+Domain: {{ sh_domain }}
+Full URL: {{ full_url }}
+IP address: {{ ip_address }}
+Registrar: {{ registrar }}
+Creation date: {{ creation_date }}
+Expiration date: {{ expiration_date }}
+Organization name: {{ org }}
+Contact e-mails: {{ mails }}
+
+SOCIAL MEDIAS SEARCH RESULTS
+FACEBOOK:
+
+ {% for link in fb_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TWITTER (+ X.com):
+
+ {% for link in tw_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+ {% for link in xcom_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+INSTAGRAM:
+
+ {% for link in inst_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TELEGRAM:
+
+ {% for link in tg_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+TIKTOK:
+
+ {% for link in tt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+LINKEDIN:
+
+ {% for link in li_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+VKONTAKTE:
+
+ {% for link in vk_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+YOUTUBE:
+
+ {% for link in yt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+ODNOKLASSNIKI:
+
+ {% for link in ok_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+WECHAT:
+
+ {% for link in wc_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+SUBDOMAINS ANALYSIS RESULTS
+Found subdomains:
+
+ {% for sd in subdomains %}
+ ⇒ {{ sd }}
+ {% endfor %}
+
+
+Subdomains IP addresses:
+
+ {% for sdip in subdomain_ip %}
+ ⇒ {{ sdip }}
+ {% endfor %}
+
+
+Subdomains e-mails:
+
+ {% for smails in subdomain_mails %}
+ ⇒ {{ smails }}
+ {% endfor %}
+
+
+DNS & SSL INFORMATION
+(DNS) Name servers: {{ name_servers }}
+(DNS) MX addresses: {{ mx_records }}
+(SSL) Issuer: {{ issuer }}
+(SSL) Subject: {{ subject }}
+(SSL) Not before: {{ notBefore }}
+(SSL) Not after: {{ notAfter }}
+(SSL) Certificate name: {{ commonName }}
+(SSL) Certificate serial number: {{ serialNumber }}
+
+SERVICES & FRAMEWORKS INFORMATION
+Web servers:
+
+ {% for ws in web_servers %}
+ ⇒ {{ ws }}
+ {% endfor %}
+
+CMS:
+
+ {% for cm in cms %}
+ ⇒ {{ cm }}
+ {% endfor %}
+
+Used programming languages:
+
+ {% for pl in programming_languages %}
+ ⇒ {{ pl }}
+ {% endfor %}
+
+Used web frameworks:
+
+ {% for wf in web_frameworks %}
+ ⇒ {{ wf }}
+ {% endfor %}
+
+Analytics service:
+
+ {% for analytic in analytics %}
+ ⇒ {{ analytic }}
+ {% endfor %}
+
+Used JavaScript frameworks:
+
+ {% for jsf in javascript_frameworks %}
+ ⇒ {{ jsf }}
+ {% endfor %}
+
+Tags:
+
+ {% for tag in tags %}
+ ⇒ {{ tag }}
+ {% endfor %}
+
+Common Platform Enumeration:
+
+ {% for cpe in cpes %}
+ ⇒ {{ cpe }}
+ {% endfor %}
+
+
+BASIC PRE-PENTEST INFORMATION
+Open ports:
+
+ {% for op in ports %}
+ ⇒ {{ op }}
+ {% endfor %}
+
+Hostnames:
+
+ {% for hn in hostnames %}
+ ⇒ {{ hn }}
+ {% endfor %}
+
+Potential vulnerabilities:
+
+ {% for vuln in vulns %}
+ ⇒ {{ vuln }}
+ {% endfor %}
+
+
+DORKING SCAN INFO
+{{ add_dsi | safe }}
+
+PAGESEARCH RESULTS
+Amount of accessible subdomains: {{ ps_s }}
+Amount of email addresses: {{ ps_e }}
+Amount of found documents: {{ ps_f }}
+Amount of found cookies: {{ ps_c }}
+Amount of found API key: {{ ps_a }}
+Amount of WEB elements found: {{ ps_w }}
+Amount of exposed passwords found: {{ ps_p }}
+PageSearch process listing:
+{{ ps_string }}
+
+VIRUSTOTAL API SCAN RESULTS
+{{ virustotal_output }}
+
+SECURITYTRAILS API SCAN RESULTS
+{{ securitytrails_output }}
+
+HUDSONROCK API SCAN RESULTS
+{{ hudsonrock_output }}
+
+Created using DPULSE software by OSINT-TECHNOLOGIES
+
+Visit our web-pages:
+
+ https://github.com/OSINT-TECHNOLOGIES
+ https://pypi.org/project/dpulse/
+
From 57d419897eed4ceb037286658855ff42a5581980 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:49:17 +0300
Subject: [PATCH 17/45] Delete
service/pdf_report_templates/default_report_temp.html
---
.../default_report_temp.html | 297 ------------------
1 file changed, 297 deletions(-)
delete mode 100644 service/pdf_report_templates/default_report_temp.html
diff --git a/service/pdf_report_templates/default_report_temp.html b/service/pdf_report_templates/default_report_temp.html
deleted file mode 100644
index 51481f0..0000000
--- a/service/pdf_report_templates/default_report_temp.html
+++ /dev/null
@@ -1,297 +0,0 @@
-
-
-
- OSINT report for {{ sh_domain }}
-
-
-
-
-Open Source Information Research Report
-{{ org }}
-Table of contents
-1. General scan information
-[BASIC SCAN INFO]
-2. WHOIS information
-3. Social medias links
-4. Subdomains information
-5. DNS & SSL information
-6. Services & frameworks information
-7. Basic pre-pentest information
-[DORKING SCAN INFO]
-8. Dorking queries and results links (if was selected)
-[PAGESEARCH SCAN INFO]
-9. PageSearch results (if was selected)
-[API SCAN INFO]
-10. VirusTotal API scan results (if was selected)
-11. SecurityTrails API scan results (if was selected)
-12. HudsonRock API scan results (if was selected)
-
-GENERAL SCAN INFORMATION
-Total subdomains found: {{ a_tsf }}
-Total social media links found: {{ a_tsm }}
-Status of robots.txt extraction: {{ robots_txt_result }}
-Status of sitemap.xml extraction: {{ sitemap_xml_result }}
-Status of sitemap.xml links extraction: {{ sitemap_links }}
-Google Dorking status: {{ dorking_status }}
-PageSearch conduction: {{ pagesearch_ui_mark }}
-Snapshotting conduction: {{ snapshotting_ui_mark }}
-Report creation time: {{ ctime }}
-
-WHOIS INFORMATION
-Domain: {{ sh_domain }}
-Full URL: {{ full_url }}
-IP address: {{ ip_address }}
-Registrar: {{ registrar }}
-Creation date: {{ creation_date }}
-Expiration date: {{ expiration_date }}
-Organization name: {{ org }}
-Contact e-mails: {{ mails }}
-
-SOCIAL MEDIAS SEARCH RESULTS
-FACEBOOK:
-
- {% for link in fb_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-TWITTER (+ X.com):
-
- {% for link in tw_links %}
- ⇒ {{ link }}
- {% endfor %}
- {% for link in xcom_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-INSTAGRAM:
-
- {% for link in inst_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-TELEGRAM:
-
- {% for link in tg_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-TIKTOK:
-
- {% for link in tt_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-LINKEDIN:
-
- {% for link in li_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-VKONTAKTE:
-
- {% for link in vk_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-YOUTUBE:
-
- {% for link in yt_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-ODNOKLASSNIKI:
-
- {% for link in ok_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-WECHAT:
-
- {% for link in wc_links %}
- ⇒ {{ link }}
- {% endfor %}
-
-
-SUBDOMAINS ANALYSIS RESULTS
-Found subdomains:
-
- {% for sd in subdomains %}
- ⇒ {{ sd }}
- {% endfor %}
-
-
-Subdomains IP addresses:
-
- {% for sdip in subdomain_ip %}
- ⇒ {{ sdip }}
- {% endfor %}
-
-
-Subdomains e-mails:
-
- {% for smails in subdomain_mails %}
- ⇒ {{ smails }}
- {% endfor %}
-
-
-DNS & SSL INFORMATION
-(DNS) Name servers: {{ name_servers }}
-(DNS) MX addresses: {{ mx_records }}
-(SSL) Issuer: {{ issuer }}
-(SSL) Subject: {{ subject }}
-(SSL) Not before: {{ notBefore }}
-(SSL) Not after: {{ notAfter }}
-(SSL) Certificate name: {{ commonName }}
-(SSL) Certificate serial number: {{ serialNumber }}
-
-SERVICES & FRAMEWORKS INFORMATION
-Web servers:
-
- {% for ws in web_servers %}
- ⇒ {{ ws }}
- {% endfor %}
-
-CMS:
-
- {% for cm in cms %}
- ⇒ {{ cm }}
- {% endfor %}
-
-Used programming languages:
-
- {% for pl in programming_languages %}
- ⇒ {{ pl }}
- {% endfor %}
-
-Used web frameworks:
-
- {% for wf in web_frameworks %}
- ⇒ {{ wf }}
- {% endfor %}
-
-Analytics service:
-
- {% for analytic in analytics %}
- ⇒ {{ analytic }}
- {% endfor %}
-
-Used JavaScript frameworks:
-
- {% for jsf in javascript_frameworks %}
- ⇒ {{ jsf }}
- {% endfor %}
-
-Tags:
-
- {% for tag in tags %}
- ⇒ {{ tag }}
- {% endfor %}
-
-Common Platform Enumeration:
-
- {% for cpe in cpes %}
- ⇒ {{ cpe }}
- {% endfor %}
-
-
-BASIC PRE-PENTEST INFORMATION
-Open ports:
-
- {% for op in ports %}
- ⇒ {{ op }}
- {% endfor %}
-
-Hostnames:
-
- {% for hn in hostnames %}
- ⇒ {{ hn }}
- {% endfor %}
-
-Potential vulnerabilities:
-
- {% for vuln in vulns %}
- ⇒ {{ vuln }}
- {% endfor %}
-
-
-DORKING SCAN INFO
-{{ add_dsi | safe }}
-
-PAGESEARCH RESULTS
-Amount of accessible subdomains: {{ ps_s }}
-Amount of email addresses: {{ ps_e }}
-Amount of found documents: {{ ps_f }}
-Amount of found cookies: {{ ps_c }}
-Amount of found API key: {{ ps_a }}
-Amount of WEB elements found: {{ ps_w }}
-Amount of exposed passwords found: {{ ps_p }}
-PageSearch process listing:
-{{ ps_string }}
-
-VIRUSTOTAL API SCAN RESULTS
-{{ virustotal_output }}
-
-SECURITYTRAILS API SCAN RESULTS
-{{ securitytrails_output }}
-
-HUDSONROCK API SCAN RESULTS
-{{ hudsonrock_output }}
-
-Created using DPULSE software by OSINT-TECHNOLOGIES
-
-Visit our web-pages:
-
- https://github.com/OSINT-TECHNOLOGIES
- https://pypi.org/project/dpulse/
-
From d06e55f2088e1d6535ebd0a44735690472370b85 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Wed, 21 May 2025 01:50:16 +0300
Subject: [PATCH 18/45] Added Modern version (default now)
---
.../modern_report_template.html | 559 ++++++++++++++++++
1 file changed, 559 insertions(+)
create mode 100644 service/pdf_report_templates/modern_report_template.html
diff --git a/service/pdf_report_templates/modern_report_template.html b/service/pdf_report_templates/modern_report_template.html
new file mode 100644
index 0000000..295da3a
--- /dev/null
+++ b/service/pdf_report_templates/modern_report_template.html
@@ -0,0 +1,559 @@
+
+
+
+ OSINT Report for {{ sh_domain }}
+
+ 🌓 Theme
+
+ Inter + Roboto Mono
+ IBM Plex Sans + Mono
+ Fira Sans + Fira Code
+ Space Grotesk + Roboto Mono
+
+
+
+
+
+Open Source Information Research Report
+{{ org }}
+GENERAL SCAN INFORMATION
+Total subdomains found: {{ a_tsf }}
+Total social media links found: {{ a_tsm }}
+Status of robots.txt extraction: {{ robots_txt_result }}
+Status of sitemap.xml extraction: {{ sitemap_xml_result }}
+Status of sitemap.xml links extraction: {{ sitemap_links }}
+Google Dorking status: {{ dorking_status }}
+PageSearch conduction: {{ pagesearch_ui_mark }}
+Snapshotting conduction: {{ snapshotting_ui_mark }}
+Report creation time: {{ ctime }}
+
+WHOIS INFORMATION
+Domain: {{ sh_domain }}
+Full URL: {{ full_url }}
+IP address: {{ ip_address }}
+Registrar: {{ registrar }}
+Creation date: {{ creation_date }}
+Expiration date: {{ expiration_date }}
+Organization name: {{ org }}
+Contact e-mails: {{ mails }}
+
+FACEBOOK:
+
+ {% for link in fb_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+TWITTER (+ X.com):
+
+ {% for link in tw_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+ {% for link in xcom_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+INSTAGRAM:
+
+ {% for link in inst_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+TELEGRAM:
+
+ {% for link in tg_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+TIKTOK:
+
+ {% for link in tt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+LINKEDIN:
+
+ {% for link in li_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+VKONTAKTE:
+
+ {% for link in vk_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+YOUTUBE:
+
+ {% for link in yt_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+ODNOKLASSNIKI:
+
+ {% for link in ok_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+WECHAT:
+
+ {% for link in wc_links %}
+ ⇒ {{ link }}
+ {% endfor %}
+
+
+SUBDOMAINS
+Found subdomains:
+
+ {% for sd in subdomains %}
+ ⇒ {{ sd }}
+ {% endfor %}
+
+
+Subdomains IP addresses:
+
+ {% for sdip in subdomain_ip %}
+ ⇒ {{ sdip }}
+ {% endfor %}
+
+
+Subdomains e-mails:
+
+ {% for smails in subdomain_mails %}
+ ⇒ {{ smails }}
+ {% endfor %}
+
+
+DNS & SSL INFORMATION
+(DNS) Name servers: {{ name_servers }}
+(DNS) MX addresses: {{ mx_records }}
+(SSL) Issuer: {{ issuer }}
+(SSL) Subject: {{ subject }}
+(SSL) Not before: {{ notBefore }}
+(SSL) Not after: {{ notAfter }}
+(SSL) Certificate name: {{ commonName }}
+(SSL) Certificate serial number: {{ serialNumber }}
+
+SERVICES & FRAMEWORKS INFORMATION
+Web servers:
+
+ {% for ws in web_servers %}
+ ⇒ {{ ws }}
+ {% endfor %}
+
+
+CMS:
+
+ {% for cm in cms %}
+ ⇒ {{ cm }}
+ {% endfor %}
+
+
+Used programming languages:
+
+ {% for pl in programming_languages %}
+ ⇒ {{ pl }}
+ {% endfor %}
+
+
+Used web frameworks:
+
+ {% for wf in web_frameworks %}
+ ⇒ {{ wf }}
+ {% endfor %}
+
+
+Analytics service:
+
+ {% for analytic in analytics %}
+ ⇒ {{ analytic }}
+ {% endfor %}
+
+
+Used JavaScript frameworks:
+
+ {% for jsf in javascript_frameworks %}
+ ⇒ {{ jsf }}
+ {% endfor %}
+
+
+Tags:
+
+ {% for tag in tags %}
+ ⇒ {{ tag }}
+ {% endfor %}
+
+
+Common Platform Enumeration:
+
+ {% for cpe in cpes %}
+ ⇒ {{ cpe }}
+ {% endfor %}
+
+
+BASIC PRE-PENTEST INFORMATION
+Open ports:
+
+ {% for op in ports %}
+ ⇒ {{ op }}
+ {% endfor %}
+
+
+Hostnames:
+
+ {% for hn in hostnames %}
+ ⇒ {{ hn }}
+ {% endfor %}
+
+
+Potential vulnerabilities:
+
+ {% for vuln in vulns %}
+ ⇒ {{ vuln }}
+ {% endfor %}
+
+
+DORKING SCAN INFO
+
+ Dorking Results
+ {{ add_dsi | safe }}
+
+
+PAGESEARCH RESULTS
+Accessible subdomains: {{ ps_s }}
+Email addresses: {{ ps_e }}
+Found documents: {{ ps_f }}
+Found cookies: {{ ps_c }}
+Found API keys: {{ ps_a }}
+Found WEB elements: {{ ps_w }}
+Exposed passwords: {{ ps_p }}
+
+ PageSearch Process Listing
+ {{ ps_string }}
+
+
+VIRUSTOTAL API SCAN RESULTS
+
+ VirusTotal Output
+ {{ virustotal_output | safe }}
+
+
+SECURITYTRAILS API SCAN RESULTS
+
+ SecurityTrails Output
+ {{ securitytrails_output | safe }}
+
+
+HUDSONROCK API SCAN RESULTS
+
+ HudsonRock Output
+ {{ hudsonrock_output | safe }}
+
+
+Created using DPULSE software by OSINT-TECHNOLOGIES
+Visit our web-pages:
+
+
+
+
+
\ No newline at end of file
From ec63c81ea3e1d5bd68965e13918084c50ebd95a5 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Fri, 23 May 2025 00:57:41 +0300
Subject: [PATCH 19/45] Added "Print" button (using printer and PDF generation)
---
.../modern_report_template.html | 362 +++++++-----------
1 file changed, 135 insertions(+), 227 deletions(-)
diff --git a/service/pdf_report_templates/modern_report_template.html b/service/pdf_report_templates/modern_report_template.html
index 295da3a..fe0ff97 100644
--- a/service/pdf_report_templates/modern_report_template.html
+++ b/service/pdf_report_templates/modern_report_template.html
@@ -1,202 +1,96 @@
- OSINT Report for {{ sh_domain }}
-
🌓 Theme
+
🖨️ Print
Inter + Roboto Mono
IBM Plex Sans + Mono
@@ -514,46 +408,60 @@ Created using DPULSE software
-
\ No newline at end of file
+
From 6e4a5251783858a290f73cbd1fcb8d908094716b Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Fri, 23 May 2025 01:31:32 +0300
Subject: [PATCH 20/45] Update README.md
---
README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/README.md b/README.md
index 374513f..cba6842 100644
--- a/README.md
+++ b/README.md
@@ -169,9 +169,9 @@ If you have problems with starting installer.sh, you should try to use `dos2unix
# Tasks to complete before new release
-- [x] Add web pages snapshoting (with screenshots)
-- [x] Add web pages snapshoting (with web pages copying as HTML objects)
-- [x] Add web pages snapshoting (with Wayback Machine)
+- [ ] CLI rework (more fancy and user-friendly)
+- [ ] Report storage database rework (more information to store,
+- [ ] HTML report rework (modern style and look; functionality expansion)
# DPULSE mentions in social medias
From 3eaa6e7cffac58fbeee2831bb59a41084db7c86c Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Thu, 29 May 2025 02:18:23 +0300
Subject: [PATCH 21/45] Added extended statistics variables for HTML reporting
---
datagather_modules/data_assembler.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/datagather_modules/data_assembler.py b/datagather_modules/data_assembler.py
index ad6ceff..bf35ad2 100644
--- a/datagather_modules/data_assembler.py
+++ b/datagather_modules/data_assembler.py
@@ -118,6 +118,10 @@ def data_gathering(self, short_domain, url, report_file_type, pagesearch_flag, k
for key in common_socials:
common_socials[key] = list(set(common_socials[key]))
total_socials = sum(len(values) for values in common_socials.values())
+ total_ports = len(ports)
+ total_ips = len(subdomain_ip) + 1
+ total_vulns = len(vulns)
+
print(Fore.LIGHTMAGENTA_EX + "\n[BASIC SCAN END]\n" + Style.RESET_ALL)
if report_file_type == 'xlsx':
if pagesearch_flag.lower() == 'y':
@@ -294,7 +298,7 @@ def data_gathering(self, short_domain, url, report_file_type, pagesearch_flag, k
hostnames, cpes, tags, vulns, common_socials, total_socials, ps_emails_return,
accessible_subdomains, emails_amount, files_counter, cookies_counter, api_keys_counter,
website_elements_counter, exposed_passwords_counter, total_links_counter, accessed_links_counter, keywords_messages_list, dorking_status, dorking_file_path,
- virustotal_output, securitytrails_output, hudsonrock_output, ps_string]
+ virustotal_output, securitytrails_output, hudsonrock_output, ps_string, total_ports, total_ips, total_vulns]
report_info_array = [casename, db_casename, db_creation_date, report_folder, ctime, report_file_type, report_ctime, api_scan_db, used_api_flag]
logging.info(f'### THIS LOG PART FOR {casename} CASE, TIME: {ctime} ENDS HERE')
From bb2358b1a43a7c34c66b3c3a52aa9c2931ee68c9 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Thu, 29 May 2025 02:18:46 +0300
Subject: [PATCH 22/45] Added extended statistics variables for HTML reporting
---
reporting_modules/html_report_creation.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/reporting_modules/html_report_creation.py b/reporting_modules/html_report_creation.py
index 8fc9932..f09f6a2 100644
--- a/reporting_modules/html_report_creation.py
+++ b/reporting_modules/html_report_creation.py
@@ -81,6 +81,9 @@ def report_assembling(short_domain, url, case_comment, data_array, report_info_a
securitytrails_output = data_array[46]
hudsonrock_output = data_array[47]
ps_string = data_array[48]
+ total_ports = data_array[49]
+ total_ips = data_array[50]
+ total_vulns = data_array[51]
casename = report_info_array[0]
db_casename = report_info_array[1]
db_creation_date = report_info_array[2]
@@ -116,6 +119,7 @@ def report_assembling(short_domain, url, case_comment, data_array, report_info_a
new_emails = email.split(', ')
subdomain_mails_cleaned.extend(new_emails)
+ total_mails = len(subdomain_mails_cleaned)
pdf_templates_path = 'service//pdf_report_templates'
config_values = read_config()
selected_template = config_values['template']
@@ -154,7 +158,8 @@ def report_assembling(short_domain, url, case_comment, data_array, report_info_a
'dorking_status': dorking_status,
'add_dsi': add_dsi, 'ps_s': accessible_subdomains, 'ps_e': emails_amount, 'ps_f': files_counter, 'ps_c': cookies_counter, 'ps_a': api_keys_counter,
'ps_w': website_elements_counter, 'ps_p': exposed_passwords_counter, 'ss_l': total_links_counter, 'ss_a': accessed_links_counter, 'hudsonrock_output': hudsonrock_output, "snapshotting_ui_mark": snapshotting_ui_mark,
- 'virustotal_output': virustotal_output, 'securitytrails_output': securitytrails_output, 'ps_string': ps_string}
+ 'virustotal_output': virustotal_output, 'securitytrails_output': securitytrails_output, 'ps_string': ps_string, 'a_tops': total_ports,
+ 'a_temails': total_mails, 'a_tips': total_ips, 'a_tpv': total_vulns}
html_report_name = report_folder + '//' + casename
if generate_report(context, html_report_name, template_path):
From 5a9d3b5578f16321642a85a21ed4de3c0242e453 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Thu, 29 May 2025 02:20:20 +0300
Subject: [PATCH 23/45] Added extended statistics variables for HTML reporting
---
.../modern_report_template.html | 38 +++++++++++--------
1 file changed, 23 insertions(+), 15 deletions(-)
diff --git a/service/pdf_report_templates/modern_report_template.html b/service/pdf_report_templates/modern_report_template.html
index fe0ff97..64f364a 100644
--- a/service/pdf_report_templates/modern_report_template.html
+++ b/service/pdf_report_templates/modern_report_template.html
@@ -79,7 +79,6 @@
.font-select{padding:6px 8px;}
.back-to-top{text-align:right;margin-top:16px;}
- /* details-аккордеоны */
details summary{cursor:pointer;font-weight:bold;margin-bottom:6px;}
details[open]>summary::after{content:" ▲";float:right;}
details>summary::after{content:" ▼";float:right;}
@@ -101,22 +100,23 @@
Open Source Information Research Report
@@ -124,8 +124,6 @@ {{ org }}
GENERAL SCAN INFORMATION
- Total subdomains found: {{ a_tsf }}
-Total social media links found: {{ a_tsm }}
Status of robots.txt extraction: {{ robots_txt_result }}
Status of sitemap.xml extraction: {{ sitemap_xml_result }}
Status of sitemap.xml links extraction: {{ sitemap_links }}
@@ -136,6 +134,16 @@ GENERAL SCAN INFORMATION
GENERAL SCAN STATISTICS
+Total subdomains found: {{ a_tsf }}
+Total social media links found: {{ a_tsm }}
+Total emails found: {{ a_temails }}
+Total associated IP addresses found: {{ a_tips }}
+Total open ports found: {{ a_tops }}
+Total potential vulnerabilities found: {{ a_tpv }}
+
+WHOIS INFORMATION
Domain: {{ sh_domain }}
Full URL: {{ full_url }}
From 4f4bb73b4bc3e77261dd898e2f469a41490ab992 Mon Sep 17 00:00:00 2001
From: OSINT-TECHNOLOGIES
<77023667+OSINT-TECHNOLOGIES@users.noreply.github.com>
Date: Thu, 29 May 2025 03:43:27 +0300
Subject: [PATCH 24/45] Improved statistics paragraph, added statistics diagram
---
.../modern_report_template.html | 99 ++++++++++++++++---
1 file changed, 85 insertions(+), 14 deletions(-)
diff --git a/service/pdf_report_templates/modern_report_template.html b/service/pdf_report_templates/modern_report_template.html
index 64f364a..ab636cc 100644
--- a/service/pdf_report_templates/modern_report_template.html
+++ b/service/pdf_report_templates/modern_report_template.html
@@ -7,6 +7,7 @@
+
