fix: Strict Projection Object Typing #15327

pshaddel · 2025-03-22T12:11:43Z

Summary
Fixes this issue: #13840

This was a PR which was merged and then reverted for releasing of 8.1

Checkout the PR, as this is already once reviewed and discussed: #13993

I just rebased the branch and pushed a commit to fix the TS tests.

…queries

…tion projection

pshaddel · 2025-04-03T10:27:31Z

@vkarpov15 Hey, can you take a look at this? :)

Copilot

Pull Request Overview

This PR fixes strict projection object typing issues as described in the related GitHub issue. Key changes include updating the chaining type test for schema plugins and enhancing TS tests for projection validations in queries.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

File	Description
test/types/schema.test.ts	Updated plugin chaining test to validate strict type preservation.
test/types/queries.test.ts	Added and updated TS assertions for projection types and query validations.

vkarpov15 · 2025-04-23T15:52:34Z

test/types/queries.test.ts

+Test.find({}, { child: 1 }); // Dot notation should be able to use a combination with objects
+Test.find({}, { 'docs.profiles': { name: 1 } }); // should support a combination of dot notation and objects
+expectError(Test.find({}, { 'docs.profiles': { name: 'aa' } })); // should support a combination of dot notation and objects
+expectError(Test.find({}, { endDate: { toString: 1 } }));


I'd love to see a test of using as ProjectionType<> to explicitly cast in case the type checking is incorrect for some reason (or if user is dealing with 5-level deep path)

@vkarpov15
something like this you mean?

Test.find({}, { docs: { unknownParams: 1 }} as ProjectionType<ITest>);

hasezoey

Looks OK to me, but makes typegoose's types not work anymore. Some of the errors i could workaround, but the last one i couldnt find the cause of, also bisecting this PR only caused errors everywhere.

As a side question, why change some unknown's to anys?

Typegoose Errors in more detail

Errors without any modifications:

$ tsc -p tsconfig.buildTests.json
src/typegoose.ts:103:9 - error TS2322: Type 'Model<ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>>, ... 4 more ..., Schema<...>>' is not assignable to type 'Model<any, {}, {}, {}, any, any>'.
  The types returned by 'findById(...).exec()' are incompatible between these types.
    Type 'Promise<(ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>> extends any[] ? Default__v<...>[] : Default__v<...>) | null>' is not assignable to type 'Promise<(FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; })[] | (FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; }) | null>'.
      Type '(ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>> extends any[] ? Default__v<...>[] : Default__v<...>) | null' is not assignable to type '(FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; })[] | (FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; }) | null'.

103   const compiledModel: mongoose.Model<any> = modelFn(name, buildSchema(cl, mergedOptions));
            ~~~~~~~~~~~~~

src/typeguards.ts:21:12 - error TS2677: A type predicate's type must be assignable to its parameter's type.
  Type 'Array<DocumentType<NonNullable<T>>>' is not assignable to type 'Array<Ref<T, S>>'.
    Type 'DocumentType<NonNullable<T>>' is not assignable to type 'Ref<T, S>'.
      Type 'DocumentType<NonNullable<T>>' is not assignable to type 'DocumentType<T, BeAnObject>'.
        Type 'DocumentType<NonNullable<T>>' is not assignable to type 'Document<unknown, BeAnObject, T>'.

21 ): docs is mongoose.Types.Array<DocumentType<NonNullable<T>>>;
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

src/typeguards.ts:27:102 - error TS2677: A type predicate's type must be assignable to its parameter's type.
  Type 'DocumentType<NonNullable<T>>[]' is not assignable to type 'Ref<T, S>[]'.
    Type 'DocumentType<NonNullable<T>>' is not assignable to type 'Ref<T, S>'.
      Type 'DocumentType<NonNullable<T>>' is not assignable to type 'DocumentType<T, BeAnObject>'.
        Type 'DocumentType<NonNullable<T>>' is not assignable to type 'Document<unknown, BeAnObject, T>'.

27 export function isDocumentArray<T, S extends RefType>(docs: Ref<T, S>[] | null | undefined): docs is DocumentType<NonNullable<T>>[];
                                                                                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Found 3 errors in 2 files.

Errors  Files
     1  src/typegoose.ts:103
     2  src/typeguards.ts:21
error Command failed with exit code 2.

Errors with modifictions:

$ tsc -p tsconfig.buildTests.json
src/typegoose.ts:103:9 - error TS2322: Type 'Model<ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>>, ... 4 more ..., Schema<...>>' is not assignable to type 'Model<any, {}, {}, {}, any, any>'.
  The types returned by 'findById(...).exec()' are incompatible between these types.
    Type 'Promise<(ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>> extends any[] ? Default__v<...>[] : Default__v<...>) | null>' is not assignable to type 'Promise<(FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; })[] | (FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; }) | null>'.
      Type '(ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>> extends any[] ? Default__v<...>[] : Default__v<...>) | null' is not assignable to type '(FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; })[] | (FlattenMaps<any> & Required<{ _id: unknown; }> & { __v: number; }) | null'.

103   const compiledModel: mongoose.Model<any> = modelFn(name, buildSchema(cl, mergedOptions));
            ~~~~~~~~~~~~~


Found 1 error in src/typegoose.ts:103

error Command failed with exit code 2.

Modifications:

diff --git a/src/typeguards.ts b/src/typeguards.ts
index 52886460..7e62e66c 100644
--- a/src/typeguards.ts
+++ b/src/typeguards.ts
@@ -18,13 +18,13 @@ export function isDocument<T, S extends RefType>(doc: Ref<T, S> | null | undefin
  */
 export function isDocumentArray<T, S extends RefType>(
   docs: mongoose.Types.Array<Ref<T, S>> | null | undefined
-): docs is mongoose.Types.Array<DocumentType<NonNullable<T>>>;
+): docs is mongoose.Types.Array<DocumentType<T>>;
 /**
  * Check if the given array is fully populated
  * Only returns "true" if all members in the array are populated
  * @param docs The Array of Refs with uncertain type
  */
-export function isDocumentArray<T, S extends RefType>(docs: Ref<T, S>[] | null | undefined): docs is DocumentType<NonNullable<T>>[];
+export function isDocumentArray<T, S extends RefType>(docs: Ref<T, S>[] | null | undefined): docs is DocumentType<T>[];
 export function isDocumentArray(docs: Ref<any, any>[] | null | undefined): unknown {
   // its "any" & "unknown" because this is not listed as an overload
   return Array.isArray(docs) && docs.every((v) => isDocument(v));

Note that everything runs fine without this PR.

Also finally, with this PR has compile times of ~30s and also kinda high resource (cpu, ram) usage, though this might just be because its taking error paths.
Normal time is around ~7s for typegoose (before this PR) and way lower resource usage.

Typegoose at b3ad7940b96baf9b3a46573bb417479be8bf94d8.

I dont really see how just changing QueryOptions projection could cause those errors. (i already tried reverting some of the other changes regarding unknown -> any to rule them out)

As a final note, i think some of those types should maybe live in their own file to be better organized than in big type files, though that might not be in scope for this PR.

EDIT: sorry, i had some issues while posting this, so the original message was incomplete and i didnt know it was posted, hence my next (hidden) message

test/types/schema.test.ts

types/index.d.ts

hasezoey · 2025-04-25T13:43:01Z

types/query.d.ts

@@ -128,7 +128,7 @@ declare module 'mongoose' {
    updatedAt?: boolean;
  }

-  interface QueryOptions<DocType = unknown> extends
+  interface QueryOptions<DocType = any> extends


Isnt DocType now unused here again?

That is true. QueryOptions is used in lots of places.

The projection is passed like this, I am not sure why the projection is used in this interface(or if anyone uses this, instead of passing it as the second - because you are already passing the projection as one argument). it causes some circular reference as well.

@hasezoey Just added a new commit. leaving the projection in the QueryOptions as it was before(not using ProjectionType).

I am trying to find out why Typegoose is facing errors, the one that I get is this one:

Type 'Model<ObtainDocumentType<any, DocumentType<InstanceType<U & Func>>, ResolveSchemaOptions<DefaultSchemaOptions>>, ... 4 more ..., Schema<...>>' is not assignable to type 'Model<any, {}, {}, {}, any, any>'. The types returned by 'findById(...).exec()' are incompatible between these types.

Yes that is the error i could not figure out why it was happening.
I mean i know that typegoose is not 100% correct with mongoose types, but it worked before this PR (and on master) and this error just seems unrelated.

Co-authored-by: hasezoey <hasezoey@gmail.com>

pshaddel · 2025-04-25T18:55:04Z

@vkarpov15 After pulling master, the number of instantiation exceeds.
I expected some more instantiations, because of the dot support and handling of nested objects to depth 4.

TypeScript Diagnostics Comparison

Metric	Master	This Branch
Files	193	193
Lines of Library	38,658	38,658
Lines of Definitions	77,536	77,639
Lines of TypeScript	20	20
Lines of JavaScript	0	0
Lines of JSON	0	0
Lines of Other	0	0
Identifiers	103,996	104,211
Symbols	254,594	244,474
Types	84,529	85,807
Instantiations	266,799	279,715
Memory used	272,070K	269,809K
Assignability cache size	3,247	3,293
Identity cache size	171	184
Subtype cache size	2	2
Strict subtype cache size	4	4
I/O Read time	0.02s	0.03s
Parse time	0.46s	0.49s
ResolveModule time	0.05s	0.05s
ResolveTypeReference time	0.01s	0.01s
ResolveLibrary time	0.02s	0.01s
Program time	0.60s	0.63s
Bind time	0.24s	0.28s
Check time	2.68s	2.81s
transformTime time	0.01s	0.01s
commentTime time	0.00s	0.00s
I/O Write time	0.00s	0.00s
printTime time	0.01s	0.01s
Emit time	0.01s	0.02s
Total time	3.54s	3.73s

For building the Projection, we need the interface, not the DocType

…ations

pshaddel and others added 11 commits March 21, 2025 19:11

Projection Type Should handle all exclusion and inclusions

2780189

fix tests

2ce0ce2

remove comment

8e482ef

remove test

6ad9293

fields which are not in schema should be allowed

df08fc7

support $slice and $elemMatch

83fc8f6

$slice and $elemMatch should be allowed with inclusion and exclusion …

5b11b44

…queries

feat: ✨ support dot notation and a combination of object and dot nota…

9daf0ca

…tion projection

should not project methods inside Date type

e94956b

use replacer type

0c1d13a

use generic to fix Types.Subdocument type

66665e3

pshaddel mentioned this pull request Mar 22, 2025

fix: Strict Projection Object Typing #13993

Merged

vkarpov15 added this to the 8.14 milestone Apr 23, 2025

vkarpov15 requested review from hasezoey and Copilot April 23, 2025 15:48

Copilot AI reviewed Apr 23, 2025

View reviewed changes

vkarpov15 reviewed Apr 23, 2025

View reviewed changes

vkarpov15 modified the milestones: 8.14, 8.15 Apr 25, 2025

hasezoey reviewed Apr 25, 2025

View reviewed changes

This comment was marked as outdated.

Sign in to view

pshaddel and others added 5 commits April 25, 2025 19:23

Update types/index.d.ts

3ff4f4d

Co-authored-by: hasezoey <hasezoey@gmail.com>

Merge branch 'Automattic:master' into fix-projection-nested-objects-new

3f43612

feat: add tsodc for recursive projector type

05e2a45

use explicit type

ed36d2d

add a sample of manual casting using ProjectionType

19b490d

Poorshad added 2 commits April 25, 2025 23:17

leave the queryOptions projection as it is

ac66c7b

For building the Projection, we need the interface, not the DocType

reduce maxDotNotationDepth to improve performance and reduce instanti…

d94f1b7

…ations

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Strict Projection Object Typing #15327

fix: Strict Projection Object Typing #15327

pshaddel commented Mar 22, 2025 •

edited

Loading

pshaddel commented Apr 3, 2025

Copilot AI left a comment

vkarpov15 Apr 23, 2025

pshaddel Apr 24, 2025

pshaddel Apr 25, 2025

hasezoey left a comment •

edited

Loading

hasezoey Apr 25, 2025

pshaddel Apr 25, 2025 •

edited

Loading

pshaddel Apr 25, 2025

hasezoey Apr 27, 2025

This comment was marked as outdated.

pshaddel commented Apr 25, 2025

fix: Strict Projection Object Typing #15327

Are you sure you want to change the base?

fix: Strict Projection Object Typing #15327

Conversation

pshaddel commented Mar 22, 2025 • edited Loading

pshaddel commented Apr 3, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

vkarpov15 Apr 23, 2025

Choose a reason for hiding this comment

pshaddel Apr 24, 2025

Choose a reason for hiding this comment

pshaddel Apr 25, 2025

Choose a reason for hiding this comment

hasezoey left a comment • edited Loading

Choose a reason for hiding this comment

hasezoey Apr 25, 2025

Choose a reason for hiding this comment

pshaddel Apr 25, 2025 • edited Loading

Choose a reason for hiding this comment

pshaddel Apr 25, 2025

Choose a reason for hiding this comment

hasezoey Apr 27, 2025

Choose a reason for hiding this comment

This comment was marked as outdated.

pshaddel commented Apr 25, 2025

TypeScript Diagnostics Comparison

pshaddel commented Mar 22, 2025 •

edited

Loading

hasezoey left a comment •

edited

Loading

pshaddel Apr 25, 2025 •

edited

Loading