[REQ] Auto generated test samples, unit test, integration test

[OmarBouslama]

Is your feature request related to a problem? Please describe.

We would like to auto generate tests from open API specs. I can see there testCase holder in this test API file : https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/main/resources/python/api_test.handlebars#L40
but it is not clear how this works.

I found in the code

openapi-generator/modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultCodegen.java

Line 95 in 574a70c

private static final String xSchemaTestExamplesRefPrefix = "#/components/x-schema-test-examples/";

that this may come from x-schema-test-examples but it is not clear how to add this to open API yaml

Could please add some example of open API specs that allow tests generation or explain this in the documentation

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[spacether]

Please read the PR that implemented it here:
https://github.com/OpenAPITools/openapi-generator/pull/12619/files
This feature currently only allows for autogenerated tests of:

• models
• endpoints with only request bodies
• endpoints with only response bodies

when positive and negative test samples are included in the source spec in a vendor extension components section.

It could be improved in the future to provide thorough testing of all inputs for a given endpoint.
For now it is only used to test one input at a time in very limited conditions.

[spacether]

This is the source test spec w/ positive and negative test cases:
https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/test/resources/3_0/unit_test_spec/3_0_3_unit_test_spec.yaml

Written by this file:
https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/test/resources/3_0/unit_test_spec/spec_writer.py

Which generates this python client sample:
https://github.com/OpenAPITools/openapi-generator/tree/master/samples/openapi3/client/3_0_3_unit_test/python

Which includes these autogenerated tests:
https://github.com/OpenAPITools/openapi-generator/tree/master/samples/openapi3/client/3_0_3_unit_test/python/test

One can see them passing in CI here:
https://app.circleci.com/pipelines/github/OpenAPITools/openapi-generator/20487/workflows/cb50fa9a-bbfe-40fa-aaff-196a202588d6/jobs/53333
with 81% coverage

Note:
Auto generated endpoints are tagged by json schema/openapi feature here:
https://github.com/OpenAPITools/openapi-generator/tree/master/samples/openapi3/client/3_0_3_unit_test/python/docs/apis/tags