forked from mordax7/flathunter
-
Notifications
You must be signed in to change notification settings - Fork 194
Open
Description
Running in docker-compose, I get this error when starting up the container:
[2025/04/28 23:06:49|config.py |INFO ]: Using config path /usr/src/app/config.yaml
[2025/04/28 23:06:49|chrome_wrapper.py |INFO ]: Initializing Chrome WebDriver for crawler...
[2025/04/28 23:06:49|patcher.py |INFO ]: patching driver executable /root/.local/share/undetected_chromedriver/undetected_chromedriver
Traceback (most recent call last):
File "/usr/src/app/flathunt.py", line 99, in <module>
main()
File "/usr/src/app/flathunt.py", line 95, in main
launch_flat_hunt(config, heartbeat)
File "/usr/src/app/flathunt.py", line 35, in launch_flat_hunt
hunter.hunt_flats()
File "/usr/src/app/flathunter/hunter.py", line 56, in hunt_flats
for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 35, in crawl_for_exposes
return chain(*[try_crawl(searcher, url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 35, in <listcomp>
return chain(*[try_crawl(searcher, url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/hunter.py", line 27, in try_crawl
return searcher.crawl(url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/abstract_crawler.py", line 154, in crawl
return self.get_results(url, max_pages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/crawler/immobilienscout.py", line 93, in get_results
soup = self.get_page(search_url, self.get_driver(), page_no)
^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/crawler/immobilienscout.py", line 68, in get_driver
self.driver = get_chrome_driver(driver_arguments)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/flathunter/chrome_wrapper.py", line 70, in get_chrome_driver
driver = uc.Chrome(version_main=chrome_version, options=chrome_options) # pylint: disable=no-member
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 466, in __init__
super(Chrome, self).__init__(
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
super().__init__(command_executor=executor, options=options)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 241, in __init__
self.start_session(capabilities)
File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 724, in start_session
super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 329, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 384, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: cannot connect to chrome at 127.0.0.1:46215
from chrome not reachable
Stacktrace:
#0 0x55913801d75a <unknown>
#1 0x559137ad0303 <unknown>
#2 0x559137abb3b3 <unknown>
#3 0x559137b0f72c <unknown>
#4 0x559137b05433 <unknown>
#5 0x559137b55f0e <unknown>
#6 0x559137b55436 <unknown>
#7 0x559137b47363 <unknown>
#8 0x559137b13d63 <unknown>
#9 0x559137b149c1 <unknown>
#10 0x559137fe2a6b <unknown>
#11 0x559137fe6951 <unknown>
#12 0x559137fcab62 <unknown>
#13 0x559137fe74c4 <unknown>
#14 0x559137faf13f <unknown>
#15 0x55913800b6f8 <unknown>
#16 0x55913800b8d6 <unknown>
#17 0x55913801c5a6 <unknown>
#18 0x7fa0e811d1f5 <unknown>
[2025/04/28 23:07:49|__init__.py |INFO ]: ensuring close
The relevant parts of my docker-compose.yml are:
services:
flathunter:
....
build: https://github.com/flathunters/flathunter.git#main
shm_size: 512M
command: python /usr/src/app/flathunt.py --heartbeat day
volumes:
- ./config.yaml:/usr/src/app/config.yaml
- ./processed_ids.db:/usr/src/app/processed_ids.db
...
And I am using this in my config:
captcha:
capmonster:
api_key: [...]
driver_arguments:
- "--no-sandbox"
- "--headless"
- "--disable-gpu"
- "--disable-setuid-sandbox"
- "--remote-debugging-port=9222"
- "--disable-dev-shm-usage"
- "window-size=1024,768"
(The --disable-setuid-sandbox
param I got from Google, doesn't seem to make a difference). Is this because of some sort of Chrome update? The local IP it tries to connect to (127.0.0.1:46215
in this case) seem to change port every restart.
Metadata
Metadata
Assignees
Labels
No labels