Skip to content

operation_data usability #396

@DifferentialOrange

Description

@DifferentialOrange

*_many operation provides operation_data with each error so it would be possible to

  • track which exact record had been failed and
  • retry failed records afterward.

But operation_data that may be returned is inconsistent, may lose some info or contain additional values that may change the behavior (like bucket_id).

./doc/playground.lua
tarantool> crud.insert_many('customers', {{'asd'}})
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:179:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for ['asd'] is ['asd', 1550].

tarantool> crud.insert_object_many('customers', {{id = 'asd'}})
2023-11-10 17:17:50.891 [1571729] main/103/playground.lua/crud.common.schema schema.lua:108 W> Number of attempts to reload schema has been ended: 1
---
- null
- - line: 314
    class_name: InsertManyError
    err: 'Failed to flatten object: FlattenError: Object is specified in bad format:
      FlattenError: Field "name" isn''t nullable (set skip_nullability_check_on_flatten
      option to true to skip check)'
    file: '...y/Development/github/tarantool/crud/crud/insert_many.lua'
    operation_data:
      id: asd
    str: 'InsertManyError: Failed to flatten object: FlattenError: Object is specified
      in bad format: FlattenError: Field "name" isn''t nullable (set skip_nullability_check_on_flatten
      option to true to skip check)'
...

Operation data for {id = 'asd'} is {id = 'asd'}.

tarantool> crud.insert_object_many('customers', {{id = 'asd', name = 'asd', age = 2}})
2023-11-10 17:20:02.788 [1571729] main/103/playground.lua/crud.common.schema schema.lua:108 W> Number of attempts to reload schema has been ended: 1
---
- null
- - line: 58
    class_name: CallError
    space_schema_hash: 247485528
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:179:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function 'wrap_func_reload'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:331:
      in function <...y/Development/github/tarantool/crud/crud/insert_many.lua:280>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550, 'asd', 2]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for {id = 'asd', name = 'asd', age = 2} is ['asd', 1550, 'asd', 2].

tarantool> crud.upsert_many('customers', { { {'asd'}, {{'=', 'age', 2}} } })
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/upsert_many.lua:193:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for { {'asd'}, {{'=', 'age', 2}} } is ['asd', 1550].

tarantool> crud.upsert_many('customers', { { {1, box.NULL, 'name', 2}, {{'=!', 'age', 2}} } })
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Unknown UPDATE operation #1: "=!"'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/upsert_many.lua:193:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: [1, 477, 'name', 2]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Unknown UPDATE operation #1: "=!"'
...

Operation data for { {1, box.NULL, 'name', 2}, {{'=!', 'age', 2}} } is [1, 477, 'name', 2], even though it's update operation that has been failed.

It seems that we need to properly consider the potential usability of operation_data and rework current implementation to satisfy these potentials.

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions