chore: Support LRO value from response in autogen resources #3829

lantoli · 2025-10-30T12:48:45Z

Description

Support LRO value from response in autogen resources.

Support LRO for attributes not known in advance, e.g. index_is, for Get and Delete operations from Create
Generate search_index_api resource
Common style log messages: INFO, WARN or ERROR
Pop up errors which were ignored before, e.g. error that is surfaced now:

2025/10/30 13:30:37 [INFO] Generating resource model: search_index_api
2025/10/30 13:30:37 [ERROR] An error occurred while generating codespec.Model: unable to map to code spec model for search_index_api: failed to process create request attributes for search_index_api: request attributes could not be mapped (OperationId: createClusterSearchIndex): invalid schema. no values for schema.Type found and type cannot be inferred (schema: BaseSearchIndexCreateRequestDefinition)

Link to any related issue(s): CLOUDP-352320

Type of change:

Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR. A migration guide must be created or updated if the new feature will go in a major version.
Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR. A migration guide must be created or updated.
This change requires a documentation update
Documentation fix/enhancement

Required Checklist:

I have signed the MongoDB CLA
I have read the contributing guides
I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
I have added any necessary documentation (if appropriate)
I have run make fmt and formatted my code
If changes include deprecations or removals I have added appropriate changelog entries.
If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Further comments

Copilot

Pull Request Overview

This PR improves the autogenerated resource code generation toolchain to support LRO (Long Running Operations) value from response and enhances error handling. It also generates a new search_index_api resource based on the MongoDB Atlas Search Index API.

Enhanced code generation with better error handling and surface previously ignored errors
Updated log message format to be consistent with INFO/WARN/ERROR prefixes
Improved LRO handling by making CallParams functions accept generic models instead of specific types

Reviewed Changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/codegen/models/search_index_api.yaml	New autogenerated resource model for search index API
tools/codegen/main.go	Updated log messages to use standardized ERROR/WARN/INFO prefixes
tools/codegen/codespec/api_to_provider_spec_mapper.go	Enhanced error handling and propagation for better debugging
tools/codegen/codespec/api_spec.go	Added schema type inference for objects without explicit types
tools/codegen/gofilegen/resource/testdata/	Updated generated code to use function-based CallParams
tools/codegen/gofilegen/codetemplate/resource-file.go.tmpl	Modified template to generate function-based CallParams handlers
internal/serviceapi/searchindexapi/	New generated search index API resource implementation
internal/common/autogen/handle_operations.go	Updated LRO handling to support function-based CallParams

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tools/codegen/codespec/api_spec.go

tools/codegen/config.yml

manupedrozo · 2025-10-31T09:31:35Z

tools/codegen/config.yml

+      method: DELETE
+    version_header: application/vnd.atlas.2024-05-30+json
+    schema:
+      use_custom_nested_types: false # TODO: investigate, prevents a panic in basetypes.ListValue.ToTerraformValue while creating the index


Fixed in: #3833

enabled here: d0c8298

EspenAlbert · 2025-10-31T09:40:17Z

internal/common/autogen/handle_operations.go

 		MinTimeout: time.Duration(wait.MinTimeoutSeconds) * time.Second,
 		Delay:      time.Duration(wait.DelaySeconds) * time.Second,
-		Refresh:    refreshFunc(ctx, wait, client),
+		Refresh:    refreshFunc(ctx, wait, client, model),


Nice. Wonder if we will ever need to expose retry.StateChangeConf{ContinuousTargetOccurence: {CUSTOM_VALUE}} in the future. I can imagine it will improve some of our acceptance tests

not sure I understand, can you explain in more detail?

EspenAlbert · 2025-10-31T09:41:53Z

internal/common/autogen/handle_operations.go


 type WaitReq struct {
-	CallParams        *config.APICallParams
+	CallParams        func(model any) *config.APICallParams


I guess this any cannot be replaced with TFModel, but will always be TFModel? Maybe worth documenting

it can't be replaced because handle operations are generic code for all resources.

We're following this pattern across autogen, e.g. passing plan or state so I think it's not needed to add a code comment here, but feel free to propose one and I'll add it.

AgustinBettati

LGTM!

AgustinBettati · 2025-10-31T10:05:00Z

tools/codegen/config.yml

+      aliases:
+        indexId: IndexID # path param name does not match the API response property


manupedrozo · 2025-10-31T10:01:46Z

tools/codegen/codespec/api_spec.go

+		// Infer object type when type is not explicitly defined but properties exist.
+		// This handles cases like BaseSearchIndexCreateRequestDefinition and BaseSearchIndexResponseLatestDefinition which have properties but no explicit type.
+		// This case can be removed after CLOUDP-355777 is done.


manupedrozo · 2025-10-31T10:05:16Z

tools/codegen/codespec/api_to_provider_spec_mapper.go

 }

-func opRequestToAttributes(op *high.Operation, useCustomNestedTypes bool) Attributes {
+func opRequestToAttributes(op *high.Operation, useCustomNestedTypes bool) (Attributes, error) {


Good! We need to keep changing all these warns to errors and just fail the generation.

yes, see the PR description for more info

* master: Add support for typed maps in autogen (#3833) Add support for typed nested maps in autogen (#3830) doc: Clarify known limitation associated to creation of duplicate entries in `project_ip_access_list` (#3823)

lantoli added 19 commits October 30, 2025 13:25

infer object type

d7915fe

generate search_index

08f865a

wait for Create, Update

7ba13e0

unify log formatting and pop errors up

ffaa788

acc test config

6320619

use indexId endpoints

72f763f

test

1ee7cf2

alias indexID

0a0ff27

ignore dynamic field

ea735ff

use_custom_nested_types=false

770e7b9

state property

7c8fa80

wait callParams as function

48c164f

update golden files

de93854

DeleteReq as a func

3bff748

delete not LRO

5caa4bf

move to autogen_slow

452985c

revert provider

b3427c1

readAPICallParams condition

7417442

ImportStateVerifyIdentifierAttribute

525d809

lantoli marked this pull request as ready for review October 31, 2025 08:01

lantoli requested a review from a team as a code owner October 31, 2025 08:01

Copilot AI review requested due to automatic review settings October 31, 2025 08:01

Copilot AI reviewed Oct 31, 2025

View reviewed changes

tools/codegen/codespec/api_spec.go Show resolved Hide resolved

tools/codegen/config.yml Show resolved Hide resolved

manupedrozo reviewed Oct 31, 2025

View reviewed changes

ImportStateVerifyIdentifierAttribute

2e3150a

EspenAlbert reviewed Oct 31, 2025

View reviewed changes

AgustinBettati approved these changes Oct 31, 2025

View reviewed changes

manupedrozo approved these changes Oct 31, 2025

View reviewed changes

Merge branch 'master' into CLOUDP-352320_autogen_lro_followup

1cc4d84

* master: Add support for typed maps in autogen (#3833) Add support for typed nested maps in autogen (#3830) doc: Clarify known limitation associated to creation of duplicate entries in `project_ip_access_list` (#3823)

custom types for search index

d0c8298

lantoli merged commit 1be1ac3 into master Oct 31, 2025
49 checks passed

lantoli deleted the CLOUDP-352320_autogen_lro_followup branch October 31, 2025 11:01

		aliases:
		indexId: IndexID # path param name does not match the API response property

chore: Support LRO value from response in autogen resources #3829

chore: Support LRO value from response in autogen resources #3829

Uh oh!

Conversation

lantoli commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change:

Required Checklist:

Further comments

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AgustinBettati left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lantoli commented Oct 30, 2025 •

edited

Loading