Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs_espressif/en/additionalfeatures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Additional IDE Features
Project Configuration Editor<additionalfeatures/project-configuration>
QEMU<additionalfeatures/qemu>
ESP RainMaker<additionalfeatures/rainmaker>
Unit Testing<additionalfeatures/unit-testing>
Unit Testing with Unity<additionalfeatures/unit-testing>
Working with Multiple Projects<additionalfeatures/multiple-projects>
Using WSL in Windows<additionalfeatures/wsl>
Web Extension<additionalfeatures/web-extension>
90 changes: 48 additions & 42 deletions docs_espressif/en/additionalfeatures/unit-testing.rst
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
.. _unit testing:

ESP-IDF Unit testing with Unity
===================================
Unit Testing with Unity
=======================

When you are developing an application using ESP-IDF and you are considering adding unit testing for your components functions, this extension can help to discover and execute tests on your device based on Unity as described in `Unit Testing in ESP32 <https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/unit-tests.html>`_ documentation.
When developing an application using ESP-IDF and considering unit testing for your component functions, this extension helps discover and execute tests on your device based on Unity, as described in `Unit Testing in ESP32 <https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/unit-tests.html>`_.

The extension explores tests in your current project workspace folders that follow the convension in the former documentation, this is, all tests files that follow the glob pattern specified in the **idf.unitTestFilePattern** configuration setting (default: ``**/test/test_*.c``) in your current workspace folders. The tests cases are parsed with the ``TEST_CASE\\(\"(.*)\",\\s*\"(.*)\"\\)`` regular expression matching the following test file format:
The extension scans the workspace folders in your current project for test files that follow the convention described in the previous documentation. Specifically, it looks for files that match the glob pattern defined by the ``idf.unitTestFilePattern`` configuration setting (default: ``/test/test_*.c``). Within these files, the extension identifies test cases using the regular expression ``TEST_CASE\("(.*)",\s*"(.*)"\)``, which matches the following test file format:

.. code-block:: C

TEST_CASE("test name", "[module name]")
{
// Add test here
}
TEST_CASE("test name", "[module name]")
{
// Add test here
}


Configure the ESP-IDF Project to enable unit tests in the extension
-------------------------------------------------------------------------
Configure the ESP-IDF Project to Enable Unit Tests in the Extension
-------------------------------------------------------------------

Let's say you have a ESP-IDF project with the following structure:
Consider an ESP-IDF project with the following structure:

.. code-block::

Expand All @@ -33,64 +33,70 @@ Let's say you have a ESP-IDF project with the following structure:
* mean.c - Component source file


Inside the ``testable`` component, unit tests are added into ``test`` directory. ``test`` directory contains source files of the tests and the component makefile (component.mk / CMakeLists.txt).
Inside the ``testable`` component, unit tests are added to the ``test`` directory, which contains the test source files and the component makefile (``component.mk``/``CMakeLists.txt``).

If you want to add tests for a ``testable`` component, just need to define a ``test`` subdirectory and add ``test_name.c`` files with the different test cases to run.
To add tests for a ``testable`` component, define a ``test`` subdirectory and add ``test_name.c`` files with different test cases to run.

This is the structure from the `ESP-IDF unit_test example <https://github.com/espressif/esp-idf/tree/master/examples/system/unit_test>`_ which can serve as reference.
This structure is from the `ESP-IDF unit_test example <https://github.com/espressif/esp-idf/tree/master/examples/system/unit_test>`_, which can serve as a reference.

.. note::
You can customize the test file discovery pattern by modifying the **idf.unitTestFilePattern** setting in your VS Code settings. This allows you to use different naming conventions or directory structures for your test files.

PyTest Embedded Services Configuration
You can customize the test file discovery pattern by modifying the ``idf.unitTestFilePattern`` setting in your VS Code settings. This allows you to use different naming conventions or directory structures for your test files.

pytest Embedded Services Configuration
--------------------------------------

The extension uses `pytest-embedded <https://docs.espressif.com/projects/pytest-embedded/en/latest/index.html>`_ to run tests on ESP-IDF devices. The **idf.pyTestEmbeddedServices** configuration setting allows you to specify which embedded services to use when running pytest commands.
The extension uses `pytest-embedded <https://docs.espressif.com/projects/pytest-embedded/en/latest/index.html>`_ to run tests on ESP-IDF devices. The ``idf.pyTestEmbeddedServices`` configuration setting allows you to specify which embedded services to use when running pytest commands.

By default, the extension uses ``["esp", "idf"]`` as the embedded services. These services provide the following functionality:

* **esp**: Enables Espressif-specific functionality including automatic target detection and port confirmation using `esptool`
* **idf**: Provides ESP-IDF project support including automatic flashing of built binaries and parsing of binary information
* **esp**: Enables Espressif-specific functionality, including automatic target detection and port confirmation using ``esptool``.
* **idf**: Provides ESP-IDF project support, including automatic flashing of built binaries and parsing of binary information.

You can customize the embedded services by modifying the **idf.pyTestEmbeddedServices** setting in your VS Code settings. For example, you might want to add additional services like:
You can customize the embedded services by modifying the ``idf.pyTestEmbeddedServices`` setting in your VS Code settings. For example, you might want to add following additional services:

* **serial**: For basic serial port communication
* **jtag**: For OpenOCD/GDB utilities
* **qemu**: For running tests on QEMU instead of real hardware
* **wokwi**: For running tests on Wokwi simulation platform
* **serial**: For basic serial port communication.
* **jtag**: For OpenOCD/GDB utilities.
* **qemu**: For running tests on QEMU instead of real hardware.
* **wokwi**: For running tests on the Wokwi simulation platform.

For a complete list of available services and their capabilities, refer to the `pytest-embedded Services Documentation <https://docs.espressif.com/projects/pytest-embedded/en/latest/concepts/services.html>`_.

.. note::
The embedded services you choose will affect the pytest command that gets executed. Make sure the services you specify are compatible with your testing environment and requirements.

The embedded services you choose will affect the pytest command that gets executed. Make sure the services you specify are compatible with your testing environment and requirements.

Running the tests
--------------------------------------------
Running the Tests
-----------------

When you click on the Testing Tab in the `Visual Studio Code Activity bar <https://code.visualstudio.com/docs/getstarted/userinterface>`_, the extension will try to find all test files and test cases and save the list of test components to add later in step 3.
When you click the ``Testing`` Tab in the `Visual Studio Code Activity bar <https://code.visualstudio.com/docs/getstarted/userinterface>`_, the extension will try to find all test files and test cases and save the list of test components to add later in step 3.

.. note::
User needs to install ESP-IDF PyTest python requirements by selecting menu **View** > **Command Palette** and type **ESP-IDF Unit Test: Install ESP-IDF PyTest requirements**. Select the command and see the pytest package installation output.

When it press the run button on a test, it will configure the current project before the tests as follows:
User needs to install ESP-IDF pytest Python requirements by selecting menu ``View`` > ``Command Palette`` and typing ``Unit Test: Install ESP-IDF pytest requirements``. Select the command and see the pytest package installation output.

1. Check that PyTest requirements from ESP-IDF are satisfied.
When you press the ``run`` button on a test, it will configure the current project before the tests as follows:

.. note::
Unit tests in this extension requires `ESP-IDF PyTest requirements <https://github.com/espressif/esp-idf/blob/master/tools/requirements/requirements.pytest.txt>`_ to be installed in your Python virtual environment.
1. Check that pytest requirements from ESP-IDF are satisfied.

2. Install ESP-IDF PyTest requirements if they are not found in the python current virtual environment specified in **idf.toolsPath** configuration setting in settings.json.
.. note::

3. Copy the unity-app from the extension template and add the test components to the main CMakeLists.txt ``TEST_COMPONENTS`` cmake variable. The extension unity-app is a basic ESP-IDF application with a unity menu that will be built and flashed on the current **idf.port** serial device with all test cases that were found during exploration step.
Unit tests in this extension requires `ESP-IDF pytest requirements <https://github.com/espressif/esp-idf/blob/master/tools/requirements/requirements.pytest.txt>`_ to be installed in your Python virtual environment.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @brianignacio5 This link leads to a 404 page, pls help fix, thx~

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems ESP-IDF changed the way pytest is installed and defined. This would require a bigger change on the vscode extension.


.. note::
You can also create, build and flash the unity test application using the **ESP-IDF Unit Test: Install ESP-IDF PyTest requirements** extension command, which will copy build and flash to your device the generated unit testing application.
2. Install ESP-IDF pytest requirements if they are not found in the python current virtual environment specified in ``idf.toolsPath`` configuration setting in ``settings.json``.

4. Runs `pytest-embedded <https://docs.espressif.com/projects/pytest-embedded/en/latest/index.html>`_ a plugin that extends PyTest to run on esp-idf devices and output the results as XML file in the unity-app directory. This is executed as an extension task and the output shown in the terminal (similar to Build and Flash tasks). The pytest command uses the embedded services specified in the **idf.pyTestEmbeddedServices** configuration setting (default: ``["esp", "idf"]``).
3. Copy the ``unity-app`` from the extension template, and add the required test components to the ``TEST_COMPONENTS`` CMake variable in the main ``CMakeLists.txt``. The ``unity-app`` provided by the extension is a simple ESP-IDF application that includes a unity test menu. It will be built and flashed to the serial device defined by ``idf.port``, together with all the test cases discovered during the exploration step.

.. note::
You can customize the embedded services used by pytest by modifying the **idf.pyTestEmbeddedServices** setting in your VS Code settings. This allows you to specify different services or add additional ones as needed for your testing environment.
.. note::

You can also create, build and flash the unity test application using the ``Unit Test: Install ESP-IDF pytest requirements`` extension command, which will copy, build, and flash the generated unit testing application to your device.

4. Run `pytest-embedded <https://docs.espressif.com/projects/pytest-embedded/en/latest/index.html>`_, a plugin that extends pytest to run on ESP-IDF devices and output the results as an XML file in the ``unity-app`` directory. This is executed as an extension task, and the output is shown in the terminal (similar to build and flash tasks). The pytest command uses the embedded services specified in the ``idf.pyTestEmbeddedServices`` configuration setting (default: ``["esp", "idf"]``).

.. note::

You can customize the embedded services used by pytest by modifying the ``idf.pyTestEmbeddedServices`` setting in your VS Code settings. This allows you to specify different services or add additional ones as needed for your testing environment.

5. The XML results file is parsed and test results are updated in the Testing tab with test duration.
5. The XML results file is parsed, and test results are updated in the ``Testing`` tab with test duration.

6. You can refresh the tests and build the unity-app again with the ``Refresh Tests`` button from the Testing tab.
6. You can refresh the tests and build the ``unity-app`` again with the ``Refresh Tests`` button from the ``Testing`` tab.
2 changes: 1 addition & 1 deletion docs_espressif/zh_CN/additionalfeatures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
项目配置编辑器<additionalfeatures/project-configuration>
QEMU 模拟器<additionalfeatures/qemu>
ESP RainMaker<additionalfeatures/rainmaker>
单元测试<additionalfeatures/unit-testing>
使用 Unity 框架进行单元测试<additionalfeatures/unit-testing>
处理多个项目<additionalfeatures/multiple-projects>
在 Windows 系统中使用 WSL<additionalfeatures/wsl>
Web Extension<additionalfeatures/web-extension>
Loading