[🐛 Bug]: Python - Too-rigid typing on `find_element` and `find_elements`

[perrygoy]

Description

Issue

The typing on find_element and find_elements specify that a selenium.webdriver.remote.webelement.WebElement is returned. This means if another project returns a custom WebElement, the typing for that WebElement is not carried through.

For example, in the Appium project, the create_web_element method of Selenium's webdriver.Remote class is overridden and typed to return an appium.webdriver.webelement.WebElement object. This override is not carried through, so mypy complains with:

error: Incompatible return value type (got "selenium.webdriver.remote.webelement.WebElement", expected "appium.webdriver.webelement.WebElement")  [return-value]
error: Incompatible return value type (got "list[selenium.webdriver.remote.webelement.WebElement]", expected "list[appium.webdriver.webelement.WebElement]")  [return-value]

Requested fix

Please add typing to match the value of create_web_element for the places that currently use selenium.webdriver.remote.webelement.WebElement!

Reproducible Code

# no need to actually run this code, just send it through MyPy to see the error.

from appium.webdriver.common.appiumby import AppiumBy

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from appium.webdriver import Remote
    from appium.webdriver.webelement import WebElement

def find_element(driver: Remote) -> WebElement:
    # should get a mypy error on next line
    return driver.find_element(AppiumBy.ACCESSIBILITY_ID, "demo button")


def find_elements(driver: Remote) -> WebElement:
    # should get a mypy error on next line
    return driver.find_elements(AppiumBy.ACCESSIBILITY_ID, "demo confetti")

Debugging Logs

mypy .
screenpy_appium/target.py:89: error: Incompatible return value type (got "selenium.webdriver.remote.webelement.WebElement", expected "appium.webdriver.webelement.WebElement")  [return-value]
screenpy_appium/target.py:98: error: Incompatible return value type (got "list[selenium.webdriver.remote.webelement.WebElement]", expected "list[appium.webdriver.webelement.WebElement]")  [return-value]
Found 2 errors in 1 file (checked 35 source files)

[selenium-ci]

@perrygoy, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[github-actions]

⚠️ You reported using Selenium version 4.24.0, but the latest release is 4.34.

Please verify that this issue still occurs with the latest version. If it no longer applies, you can close this issue or update your comment.

This issue will be marked "awaiting answer" and may be closed automatically if no response is received.

[perrygoy]

Sorry about that—i have confirmed that, even in the latest version 4.34.2, this issue persists.

[cgoldberg]

I don't know Appium very well, so can you help me understand the issue better?

I don't think your code is a very good example (obviously it has an incompatible return value).

If you call a parent class method but expect a different type, shouldn't you override that method in the subclass with a different type annotation?

In the case of Appium...
appium.webdriver.WebDriver is a subclass of selenium.webdriver.Remote. Since they override create_web_element, should they also override find_element/find_elements with the type they are expecting?

I'm not sure why we should change our typing to accomadate what a subclass might do.

[perrygoy]

I can see the argument for that.

In my own project (the one i encountered this issue on), we had a similar situation where a user was subclassing the Actor class of ours but couldn't get mypy to pass because when our Actor object would return itself it was typed as returning, specifically, Actor, not allowing the subclass they had made. The situation here reminded me of that.

Our solution was to mark the return type as Self instead of Actor, but looking at the code here i don't know exactly how we would capture "return value is whatever create_web_element returns" exactly. Maybe this would be better as an Appium PR instead, as you say.

The reason i suggest the change here is that other folks who are subclassing Remote would have an easier time if they had the option to only override create_web_element and its typing, as the Appium folks have tried to do, rather than override each thing that returns WebElement as well.