Need help running pytest tests from a remote machine

[Abhiram-R]

Hi,

Can someone help me set up the pytest command for remote testing. Based on my current understanding the command that I'm using on my machine is: pytest --lg-env remote.yaml --lg-coordinator username@IP_ADDR test_shell.py

But its showing error:

(venv) root@VIRTU-439654:/home/user2/Downloads/labgridenv# pytest --lg-env ./remote.yaml --lg-coordinator teuser@192.168.65.102 test_shell.py 
============================= test session starts ==============================
platform linux -- Python 3.10.15, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/user2/Downloads/labgridenv
plugins: labgrid-24.1.dev122
collected 1 item                                                               

test_shell.py F                                                          [100%]

=================================== FAILURES ===================================
_________________________________ test_target __________________________________

self = Serial<id=0x7fdfa2d35fc0, open=False>(port='rfc2217://teuservbox:41171?ign_set_control&timeout=3.0', baudrate=1843200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)

    def open(self):
        """\
        Open port with current settings. This may throw a SerialException
        if the port cannot be opened.
        """
        self.logger = None
        self._ignore_set_control_answer = False
        self._poll_modem_state = False
        self._network_timeout = 3
        if self._port is None:
            raise SerialException("Port must be configured before it can be used.")
        if self.is_open:
            raise SerialException("Port is already open.")
        try:
>           self._socket = socket.create_connection(self.from_url(self.portstr), timeout=5)  # XXX good value?

venv/lib/python3.10/site-packages/serial/rfc2217.py:420: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.10/socket.py:836: in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

host = 'teuservbox', port = 41171, family = 0
type = <SocketKind.SOCK_STREAM: 1>, proto = 0, flags = 0

    def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        """Resolve host and port into list of address info entries.
    
        Translate the host/port argument into a sequence of 5-tuples that contain
        all the necessary arguments for creating a socket connected to that service.
        host is a domain name, a string representation of an IPv4/v6 address or
        None. port is a string service name such as 'http', a numeric port number or
        None. By passing None as the value of host and port, you can pass NULL to
        the underlying C API.
    
        The family, type and proto arguments can be optionally specified in order to
        narrow the list of addresses returned. Passing zero as a value for each of
        these arguments selects the full range of results.
        """
        # We override this function since we want to translate the numeric family
        # and socket type values to enum constants.
        addrlist = []
>       for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E       socket.gaierror: [Errno -2] Name or service not known

/usr/lib/python3.10/socket.py:967: gaierror

During handling of the above exception, another exception occurred:

self = SerialDriver(target=Target(name='main', env=Environment(config_file='./remote.yaml')), name=None, state=<BindingState.bound: 1>, txdelay=0.0, timeout=3.0)

    def open(self):
        """Opens the serialport, does nothing if it is already open"""
        if not self.status:
            try:
>               self.serial.open()

venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Serial<id=0x7fdfa2d35fc0, open=False>(port='rfc2217://teuservbox:41171?ign_set_control&timeout=3.0', baudrate=1843200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)

    def open(self):
        """\
        Open port with current settings. This may throw a SerialException
        if the port cannot be opened.
        """
        self.logger = None
        self._ignore_set_control_answer = False
        self._poll_modem_state = False
        self._network_timeout = 3
        if self._port is None:
            raise SerialException("Port must be configured before it can be used.")
        if self.is_open:
            raise SerialException("Port is already open.")
        try:
            self._socket = socket.create_connection(self.from_url(self.portstr), timeout=5)  # XXX good value?
            self._socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        except Exception as msg:
            self._socket = None
>           raise SerialException("Could not open port {}: {}".format(self.portstr, msg))
E           serial.serialutil.SerialException: Could not open port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: [Errno -2] Name or service not known

venv/lib/python3.10/site-packages/serial/rfc2217.py:424: SerialException

The above exception was the direct cause of the following exception:

target = Target(name='main', env=Environment(config_file='./remote.yaml'))

    def test_target(target):
>       barebox = target.get_driver("CommandProtocol")

test_shell.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.10/site-packages/labgrid/target.py:234: in get_driver
    return self._get_driver(cls, name=name, resource=resource, activate=activate)
venv/lib/python3.10/site-packages/labgrid/target.py:208: in _get_driver
    self.activate(found[0])
venv/lib/python3.10/site-packages/labgrid/target.py:467: in activate
    self.activate(supplier)
venv/lib/python3.10/site-packages/labgrid/target.py:471: in activate
    client.on_activate()
venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:51: in on_activate
    self.open()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = SerialDriver(target=Target(name='main', env=Environment(config_file='./remote.yaml')), name=None, state=<BindingState.bound: 1>, txdelay=0.0, timeout=3.0)

    def open(self):
        """Opens the serialport, does nothing if it is already open"""
        if not self.status:
            try:
                self.serial.open()
            except serial.SerialException as e:
>               raise serial.SerialException(
                    f"Could not open serial port {self.serial.port}: {str(e)}") from e
E               serial.serialutil.SerialException: Could not open serial port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: Could not open port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: [Errno -2] Name or service not known

venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:104: SerialException
=========================== short test summary info ============================
FAILED test_shell.py::test_target - serial.serialutil.SerialException: Could not open serial port rfc2217://teu...
============================== 1 failed in 0.75s ===============================

my remote.yaml file:

targets:
  main:
    resources:
      RemotePlace:
        name: example-place
    drivers:
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'

and test_shell.py:

def test_target(target):
    barebox = target.get_driver("CommandProtocol")
    target.activate(barebox)

    stdout, stderr, returncode = barebox.run("version")
    assert returncode == 0
    assert stdout
    assert not stderr
    assert "barebox" in "\n".join(stdout)

    stdout, stderr, returncode = barebox.run("false")
    assert returncode == 1
    assert not stdout
    assert not stderr

I am using the Labgrid 24.1.dev123 release.

On one machine I have both coordinator and exporter running.

The exporter configuration:

example-group:
  location: example-location
  USBSerialPort:
    match:
      ID_SERIAL_SHORT: '002124'
      ID_USB_INTERFACE_NUM: '00'
    port: /dev/ttyUSB0
    speed: 1843200

I am able to access my hardware's terminal from my machine using the command:
labgrid-client -P username@IP_ADDR -p example-place acquire

and then using:
labgrid-client -P username@IP_ADDR -p example-place console

I think my test_shell.py file is incorrect or there is some change to be made to the remote.yaml file. Can someone help me on fixing this issue.

[Bastian-Krause]

Sorry for the late response.

Can someone help me set up the pytest command for remote testing. Based on my current understanding the command that I'm using on my machine is: pytest --lg-env remote.yaml --lg-coordinator username@IP_ADDR test_shell.py

But its showing error:

(venv) root@VIRTU-439654:/home/user2/Downloads/labgridenv# pytest --lg-env ./remote.yaml --lg-coordinator teuser@192.168.65.102 test_shell.py 

From pytest --help:

  --lg-coordinator=COORDINATOR_ADDRESS
                        labgrid coordinator HOST[:PORT].

In practice the username is currently ignored, but you shouldn't rely on that.

============================= test session starts ==============================
platform linux -- Python 3.10.15, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/user2/Downloads/labgridenv
plugins: labgrid-24.1.dev122
collected 1 item                                                               

test_shell.py F                                                          [100%]

=================================== FAILURES ===================================
_________________________________ test_target __________________________________

self = Serial<id=0x7fdfa2d35fc0, open=False>(port='rfc2217://teuservbox:41171?ign_set_control&timeout=3.0', baudrate=1843200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)

    def open(self):
        """\
        Open port with current settings. This may throw a SerialException
        if the port cannot be opened.
        """
        self.logger = None
        self._ignore_set_control_answer = False
        self._poll_modem_state = False
        self._network_timeout = 3
        if self._port is None:
            raise SerialException("Port must be configured before it can be used.")
        if self.is_open:
            raise SerialException("Port is already open.")
        try:
>           self._socket = socket.create_connection(self.from_url(self.portstr), timeout=5)  # XXX good value?

venv/lib/python3.10/site-packages/serial/rfc2217.py:420: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.10/socket.py:836: in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

host = 'teuservbox', port = 41171, family = 0
type = <SocketKind.SOCK_STREAM: 1>, proto = 0, flags = 0

    def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        """Resolve host and port into list of address info entries.
    
        Translate the host/port argument into a sequence of 5-tuples that contain
        all the necessary arguments for creating a socket connected to that service.
        host is a domain name, a string representation of an IPv4/v6 address or
        None. port is a string service name such as 'http', a numeric port number or
        None. By passing None as the value of host and port, you can pass NULL to
        the underlying C API.
    
        The family, type and proto arguments can be optionally specified in order to
        narrow the list of addresses returned. Passing zero as a value for each of
        these arguments selects the full range of results.
        """
        # We override this function since we want to translate the numeric family
        # and socket type values to enum constants.
        addrlist = []
>       for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E       socket.gaierror: [Errno -2] Name or service not known

teuservbox can't be resolved.

/usr/lib/python3.10/socket.py:967: gaierror

During handling of the above exception, another exception occurred:

self = SerialDriver(target=Target(name='main', env=Environment(config_file='./remote.yaml')), name=None, state=<BindingState.bound: 1>, txdelay=0.0, timeout=3.0)

    def open(self):
        """Opens the serialport, does nothing if it is already open"""
        if not self.status:
            try:
>               self.serial.open()

venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Serial<id=0x7fdfa2d35fc0, open=False>(port='rfc2217://teuservbox:41171?ign_set_control&timeout=3.0', baudrate=1843200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)

    def open(self):
        """\
        Open port with current settings. This may throw a SerialException
        if the port cannot be opened.
        """
        self.logger = None
        self._ignore_set_control_answer = False
        self._poll_modem_state = False
        self._network_timeout = 3
        if self._port is None:
            raise SerialException("Port must be configured before it can be used.")
        if self.is_open:
            raise SerialException("Port is already open.")
        try:
            self._socket = socket.create_connection(self.from_url(self.portstr), timeout=5)  # XXX good value?
            self._socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        except Exception as msg:
            self._socket = None
>           raise SerialException("Could not open port {}: {}".format(self.portstr, msg))
E           serial.serialutil.SerialException: Could not open port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: [Errno -2] Name or service not known

venv/lib/python3.10/site-packages/serial/rfc2217.py:424: SerialException

The above exception was the direct cause of the following exception:

target = Target(name='main', env=Environment(config_file='./remote.yaml'))

    def test_target(target):
>       barebox = target.get_driver("CommandProtocol")

test_shell.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.10/site-packages/labgrid/target.py:234: in get_driver
    return self._get_driver(cls, name=name, resource=resource, activate=activate)
venv/lib/python3.10/site-packages/labgrid/target.py:208: in _get_driver
    self.activate(found[0])
venv/lib/python3.10/site-packages/labgrid/target.py:467: in activate
    self.activate(supplier)
venv/lib/python3.10/site-packages/labgrid/target.py:471: in activate
    client.on_activate()
venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:51: in on_activate
    self.open()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = SerialDriver(target=Target(name='main', env=Environment(config_file='./remote.yaml')), name=None, state=<BindingState.bound: 1>, txdelay=0.0, timeout=3.0)

    def open(self):
        """Opens the serialport, does nothing if it is already open"""
        if not self.status:
            try:
                self.serial.open()
            except serial.SerialException as e:
>               raise serial.SerialException(
                    f"Could not open serial port {self.serial.port}: {str(e)}") from e
E               serial.serialutil.SerialException: Could not open serial port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: Could not open port rfc2217://teuservbox:41171?ign_set_control&timeout=3.0: [Errno -2] Name or service not known

venv/lib/python3.10/site-packages/labgrid/driver/serialdriver.py:104: SerialException
=========================== short test summary info ============================
FAILED test_shell.py::test_target - serial.serialutil.SerialException: Could not open serial port rfc2217://teu...
============================== 1 failed in 0.75s ===============================

my remote.yaml file:

targets:
  main:
    resources:
      RemotePlace:
        name: example-place
    drivers:
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'

and test_shell.py:

def test_target(target):
    barebox = target.get_driver("CommandProtocol")
    target.activate(barebox)

If you expect an instance of the BareboxDriver, request that instead of the generic CommandProtocol.

    stdout, stderr, returncode = barebox.run("version")
    assert returncode == 0
    assert stdout
    assert not stderr
    assert "barebox" in "\n".join(stdout)

    stdout, stderr, returncode = barebox.run("false")
    assert returncode == 1
    assert not stdout
    assert not stderr

I am using the Labgrid 24.1.dev123 release.

On one machine I have both coordinator and exporter running.

The exporter configuration:

example-group:
  location: example-location
  USBSerialPort:
    match:
      ID_SERIAL_SHORT: '002124'
      ID_USB_INTERFACE_NUM: '00'
    port: /dev/ttyUSB0
    speed: 1843200

The port is only required for a RawSerialPort. See https://labgrid.readthedocs.io/en/latest/configuration.html#usbserialport.

I am able to access my hardware's terminal from my machine using the command: labgrid-client -P username@IP_ADDR -p example-place acquire

and then using: labgrid-client -P username@IP_ADDR -p example-place console

Note that the proxy value is meant to contain only the hostname. The "username@" prefix might currently work, but that's only an implementation detail you should not rely on. Better configure these things in your SSH client config.

I think my test_shell.py file is incorrect or there is some change to be made to the remote.yaml file. Can someone help me on fixing this issue.