object.collect return LIMIT ?

[dikobrazsd]

After version 0.32 object.collect returns MAX 100(200 lines) items, how to increase it? I can't find any argument for pagination or smth.

Example:

$ ./govc version
govc 0.38.0
$ ./govc object.collect -type m  / name runtime.powerState | wc -l
200

$ ./govc032 version
govc 0.32.0
$ ./govc033 object.collect -type m  / name runtime.powerState | wc -l
5756

[dougm]

Can you share your vCenter build number and add -trace (./govc object.collect -trace ... 2> trace.log), sharing the trace.log if possible? Wondering if it could be this issue: #3455
Workaround: #3455 (comment)

We did change to collecting in batches (#3408), but it is expected you'll see the same set of results. With exception of the known issue linked above.

[dikobrazsd]

will try to get trace.log

here is version and build number:

FullName:     VMware vCenter Server 7.0.3 build-24026615
Name:         VMware vCenter Server
Vendor:       VMware, Inc.
Version:      7.0.3
Build:        24026615
OS type:      linux-x64
API type:     VirtualCenter
API version:  7.0.3.0
Product ID:   vpx

[dikobrazsd]

Adding trace.log

https://gist.github.com/dikobrazsd/d8167cc354cf920df507b53b39ca978e

[dougm]

Thanks @dikobrazsd , can see in the log:

<WaitForUpdatesExResponse xmlns="urn:vim25">
 <returnval>
  <version>0_1</version>
...
  <truncated>true</truncated>
 </returnval>
</WaitForUpdatesExResponse>

When truncated=true, we are supposed to continue calling WaitForUpdatesEx, but there's only 1 response:

% grep -c /WaitForUpdatesExResponse gistfile1.txt 
1

When truncated=true, govc object.collect's callback returns false, which would trigger another call to WaitForUpdatesEx:

govmomi/govc/object/collect.go

Lines 467 to 469 in fc7056a

	if filter.Truncated {
	return false // vCenter truncates updates if > 100
	}

The value of filter.Truncated is supposed to be propagated from WaitForUpdatesEx response to the caller here:

govmomi/property/collector.go

Lines 312 to 314 in fc7056a

	if set.Truncated != nil {
	opts.Truncated = *set.Truncated
	}

I bisected this to PR #3331 - in that refactor, WaitOptions was changed to pass by value, rather than reference. Patch below fixes.
Before opening a PR, I want add vcsim support for truncated=true so we can have the proper test coverage in place for this.

diff --git a/property/collector.go b/property/collector.go
index 263621a0..9ee78d0d 100644
--- a/property/collector.go
+++ b/property/collector.go
@@ -276,7 +276,7 @@ func (p *Collector) RetrieveOne(ctx context.Context, obj types.ManagedObjectRefe
 // propagation.
 func (p *Collector) WaitForUpdatesEx(
 	ctx context.Context,
-	opts WaitOptions,
+	opts *WaitOptions,
 	onUpdatesFn func([]types.ObjectUpdate) bool) error {
 
 	if !p.mu.TryLock() {
diff --git a/property/wait.go b/property/wait.go
index 07ea3cb5..dae7bf38 100644
--- a/property/wait.go
+++ b/property/wait.go
@@ -123,7 +123,7 @@ func WaitForUpdates(
 		return err
 	}
 
-	return pc.WaitForUpdatesEx(ctx, filter.WaitOptions, onUpdatesFn)
+	return pc.WaitForUpdatesEx(ctx, &filter.WaitOptions, onUpdatesFn)
 }
 
 // WaitForUpdates waits for any of the specified properties of the specified
@@ -166,5 +166,5 @@ func WaitForUpdatesEx(
 
 	}()
 
-	return pc.WaitForUpdatesEx(ctx, filter.WaitOptions, onUpdatesFn)
+	return pc.WaitForUpdatesEx(ctx, &filter.WaitOptions, onUpdatesFn)
 }

[dougm]

@dikobrazsd fix included in the v0.40 release

[dikobrazsd]

thank you for update.

now it works correctly.:)

# ./govc040 object.collect -type m  / name runtime.powerState | wc -l
5820