Immoscout24 block - no Capmonster capture used

[philipempl]

Hi there,

First of all, thanks for creating and maintaining this awesome project! While I was looking for flats in Munich, I came across this project and tried my best to make it run. It works effortlessly with Immowelt and Kleinanzeigen, but when I try to make it work with Immobilienscout24 (IS24), I run into serious issues. Below are the details:

---

Environment

• OS: Ubuntu
• Run Method: Using a systemd service file

---

Config.yaml

---
# Configuration for Flathunter

# Verbose mode for debugging
verbose: true

# Looping settings
loop:
  active: yes
  sleeping_time: 600

# Target URLs
urls:
  - "https://www.immobilienscout24.de/Suche/shape/wohnung-mit-balkon-mieten?shape=e295ZEhpd2VlQXxpQ19ObF9BdXJBaG9BbXFGbFRheUxle0B9Z0FtakJ5ZUF9b0NtSnFBc0JtX0E-b3pEbG5BeWBBam5Be2lAbndEaUNiYEVwcEFkb0V8cEN_Y0M.&numberofrooms=3.0-&price=-1550.0&livingspace=70.0-&exclusioncriteria=swapflat&pricetype=rentpermonth&sorting=2&viewMode=HYBRID_VIEW"
  - "https://www.kleinanzeigen.de/s-wohnung-mieten/muenchen/preis::1800/c203l6411+wohnung_mieten.qm_d:70%2C+wohnung_mieten.swap_s:nein+wohnung_mieten.zimmer_d:3%2C"
  - "https://www.immowelt.de/classified-search?distributionTypes=Rent&estateTypes=House,Apartment&featuresIncluded=Balcony_Terrace&locations=AD09DE419,AD09DE423,AD09DE432,AD09DE431,AD09DE420,AD09DE421,AD09DE429,AD09DE435,AD09DE434,AD09DE430,AD09DE422,AD09DE426,AD09DE424,AD09DE425,AD09DE436&locationsInBuildingExcluded=Groundfloor&numberOfRoomsMin=2.5&priceMax=1600&projectTypes=New_Build,Flatsharing,Short_Time_Rental,Stock&spaceMin=70&order=DateDesc"

# Filters
filters:
  excluded_titles:
    - "Tauschwohnung"
    - "WG"
    - "Wohnungstausch"
    - "WG-Zimmer"
    - "Suche"

notifiers:
  - telegram

# Telegram notification configuration
telegram:
  bot_token: ABC
  receiver_ids:
    - ABC

# Captcha solving service
captcha:
  capmonster:
    api_key: ABC
  driver_arguments:
    - "--no-sandbox"
    - "--headless"
    - "--disable-gpu"
    - "--remote-debugging-port=9222"
    - "--disable-dev-shm-usage"
    - "window-size=1024,768"

# Enable a headless browser for Chrome
browser:
  headless: true

use_proxy_list: True

---

Service Log

Here is the output from running systemctl status flathunter:

root@ubuntu:/opt/flathunter# sudo systemctl status flathunter
× flathunter.service - Flathunter Python Script
     Loaded: loaded (/lib/systemd/system/flathunter.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2025-01-06 11:28:32 UTC; 1s ago
    Process: 3560377 ExecStart=/home/flathunter/.local/bin/pipenv run python /opt/flathunter/flathunt.py --config /opt/flathunter/config.yaml (code=exited, status=1/FAILURE)
   Main PID: 3560377 (code=exited, status=1/FAILURE)
        CPU: 1.934s

Jan 06 11:28:31 ubuntu flathunter[3560377]: #13 0x557f3b442b17 <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: #14 0x557f3b40f6cf <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: #15 0x557f3b4606b8 <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: #16 0x557f3b460880 <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: #17 0x557f3b470776 <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: #18 0x7f1241c48ac3 <unknown>
Jan 06 11:28:31 ubuntu flathunter[3560377]: [2025/01/06 11:28:31|__init__.py             |INFO    ]: ensuring close
Jan 06 11:28:32 ubuntu systemd[1]: flathunter.service: Main process exited, code=exited, status=1/FAILURE
Jan 06 11:28:32 ubuntu systemd[1]: flathunter.service: Failed with result 'exit-code'.
Jan 06 11:28:32 ubuntu systemd[1]: flathunter.service: Consumed 1.934s CPU time

---

Issue

• The script runs perfectly with Immowelt and Kleinanzeigen but fails for IS24.
• The service exits with a failure code without providing clear error messages.

Any help or suggestions would be highly appreciated!

Thanks in advance! 🙏

[philipempl]

Apologies, I realized I forgot to add the -- prefix to window-size=1024,768.

However, when I run the script, it doesn’t seem to utilize Captcha solving. Upon checking my CapMonster balance, there’s no indication of activity. Instead, I receive the following messages:

1. Unable to find IS24 variable in window
2. IS24 bot detection has identified our script as a bot - we've been blocked

[lsbender]

Hi,
I'm running the script in Docker with some changes (The modifications are more on the notification page and I skip Immoscout24 Plus listings and the nearby results (with which is filled up, there are not enough results at Immowelt and Immoscout24)), but that probably won't be the change. Rather, I think it's the proxy. Are you using it from home? Then leave the setting “use_proxy_list: True” setting. This will give you a public IP which is blocked.