feat(iota, iota-move): add implicit system package dependencies #7886
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove dep from sui-e2e-tests -> sui (#21422) · MystenLabs/sui@988531b
…_coin (#7084) New_coin API should take the type_tag of the coin directly, instead of allowing arbitrary move types. According to [changes](MystenLabs/sui@8a28a6e).
…d duplicate "the the" (#7132) Removed duplicate "the the”. According to [changes](MystenLabs/sui@3b36106). Fixes #7070 . - Documentation Fix
…o canonicalize a type (#7218) # Description of change Add a function to the package resolver to canonicalize a type (ensure all the package IDs it mentions are defining IDs) According to [changes](MystenLabs/sui@778e78a#diff-000c960638af936148493636d55c8ea62676d2f051774239fc73e7fc1468bf32). ## Links to any relevant issues Fixes #7078 . ## How the change has been tested Run ``` cargo nextest run -p iota-package-resolver ```
…7213) # Description of change Consistently use the canonical display format with prefix in error messages from the package resolver. According to [changes](MystenLabs/sui@83f6f17#diff-b327546aaf3e64d31d006c18c86f8f9c478930d7cd6d69389ae0069f35b758f3). Any snapshot changes haven’t been detected.. ## Links to any relevant issues Fixes #7062 . ## How the change has been tested Run tests: ``` cargo nextest run \ -p iota-graphql-e2e-tests \ -p iota-package-resolver ```
…s` (#7216) Generalize `Resolver::function_parameters` so that it returns the fully resolved function signature. Previously this function was used to infer transaction input layouts, but we also need roughly the same logic for returning a normalized move function. According to [changes](MystenLabs/sui@5024b1f#diff-76de243b84e1bcb523b9725574d3ae44530b9a49db102035ad19f2d8d6bb54c3). Fixes #7060 . Run ``` cargo nextest run -p iota-package-resolver ```
…nsactional test adapter (#7180) [run-ci] According to [changes](MystenLabs/sui@ae367c4#diff-7a132b89ef988a881b2c5d33b4cee85acb20e6be9c7fc645d26806d7ae307290). I as see we don’t have iota-indexer-alt-e2e tests and iota-mvr-graphql-e2e, so changes only touch iota-graphql-e2e- tests. Only these changes differ from Sui’s commit. I assume it’s because of the version differences in fastcrypto or async_graphql. To comply with how iota-graphql-rpc decodes the JSON cursor … this traces back to async_graphql = 7.0.7, which uses no padding for encoding/decoding. `Ok(Base64::encode(cursor)); ` This likely defaults to: - STANDARD base64 - with padding (=) - and possibly using the general character set (+ / /). But the decoder expects: - URL_SAFE_NO_PAD - no padding, using - and _ - matching what async_graphql and the RPC layer parse. ``` return Ok(base64::Engine::encode( &base64::engine::general_purpose::URL_SAFE_NO_PAD, cursor, )); ``` Fixes #7059 . - Enhancement (a non-breaking change which adds functionality) For iota-graphql-e2e- tests run: `cargo nextest run --features pg_integration` --------- Co-authored-by: Istvan Davorin Acs <istvan.acs@iota.org>
# Description of change In [commit](058b493#diff-17cdfbdc54b5e6dd0eff37a911fbf44167d77ac8342b54ff6f41b8cccfca2671) some test names weren't following the changes from the original [changes list](MystenLabs/sui@778e78a#diff-000c960638af936148493636d55c8ea62676d2f051774239fc73e7fc1468bf32). To avoid any future confusion, these were fixed. ## Links to any relevant issues Adjusts #7078 .
Based on [changes](MystenLabs/sui@6d8ceed). (For original description see the ticket.) Most APIs, command line tools simply do not allow the user to construct syntactically incorrect transaction, but there are APIs which do. Namely ones in which users can send in an already prepared transaction as a byte sequence. ``` Usage: iota client execute-signed-tx [OPTIONS] --tx-bytes <TX_BYTES> ``` or the GraphQL API: ``` [executeTransactionBlock](https://graphql.testnet.iota.cafe/#)( txBytes: [String](https://graphql.testnet.iota.cafe/#)! signatures: [[String](https://graphql.testnet.iota.cafe/#)!]! ): [ExecutionResult](https://graphql.testnet.iota.cafe/#)! ``` or the Json RPC API: ``` iota_executeTransactionBlock ``` In terms of this change the cases in which an invalid **module** or **function** identifier manages to sneak through the checks the system would have deemed it a valid transaction, but it would have failed during execution. Because of this it would have cost gas, forcing it the invalid transaction to be stored on-chain. This would cause downstream interfaces to fail while trying to access the invalid transaction. For example calling this is a correct call for the clock timestamp: `cargo run -p iota -- client call --package 0x2 --module clock --function timestamp_ms --args 0x6` While this would be incorrect: `cargo run -p iota -- client call --package 0x2 --module _ --function timestamp_ms --args 0x6` Now if you tried to execute the second function you would notice that the call fails as the tool properly protects the user. But if you manage to construct a byte sequence containing the same function call with an underscore as a module name, you could sign it and send it in (through the raw byte interfaces), and get the above described issue. This change aims to mitigate these issues. Now the **module** and **function** identifiers are checked for being syntactically valid identifiers, so after the new protocol version, no more transactions can be logged on-chain. Furthermore the GraphQL API has been modified so that it won't crash while trying to parse such an invalid entry. We have added an e2e test for checking the modified behavior. fixes #7081 Choose a type of change, and delete any options that are not relevant. - Bug fix (a non-breaking change which fixes an issue) Regenerate crates/iota-core/tests/staged/iota.yaml: `cargo run -p iota-core --example generate-format -- print > crates/iota-core/tests/staged/iota.yaml` Regenerate crates/iota-graphql-rpc/tests/snapshots/snapshot_tests__schema_sdl_export.snap `cargo nextest run -p iota-graphql-rpc --features pg_integration --no-fail-fast --test-threads 1` Regenerate/update iota-protocol-config snapshots: `cargo nextest run -p iota-protocol-config` Run iota-graphql-e2e-tests: `cargo nextest run -p iota-graphql-e2e-tests --features pg_integration` The new e2e tests: `cargo nextest run -p iota-e2e-tests --test tx_bytes_validity_check` @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. <!-- Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. --> - [x] Protocol: Transactions that include invalid types will not be signed by validators from protocol 10 onwards. - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: `MoveType` can fail to provide a layout or abilities in cases where it represents an invalid type. - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: Ashok Menon <ashok@mystenlabs.com> Co-authored-by: Marc Espin <mespinsanz@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 6 Skipped Deployments
|
iota-ci
added
the
sc-platform
…nsactional test adapter (#7180) [run-ci] According to [changes](MystenLabs/sui@ae367c4#diff-7a132b89ef988a881b2c5d33b4cee85acb20e6be9c7fc645d26806d7ae307290). I as see we don’t have iota-indexer-alt-e2e tests and iota-mvr-graphql-e2e, so changes only touch iota-graphql-e2e- tests. Only these changes differ from Sui’s commit. I assume it’s because of the version differences in fastcrypto or async_graphql. To comply with how iota-graphql-rpc decodes the JSON cursor … this traces back to async_graphql = 7.0.7, which uses no padding for encoding/decoding. `Ok(Base64::encode(cursor)); ` This likely defaults to: - STANDARD base64 - with padding (=) - and possibly using the general character set (+ / /). But the decoder expects: - URL_SAFE_NO_PAD - no padding, using - and _ - matching what async_graphql and the RPC layer parse. ``` return Ok(base64::Engine::encode( &base64::engine::general_purpose::URL_SAFE_NO_PAD, cursor, )); ``` Fixes #7059 . - Enhancement (a non-breaking change which adds functionality) For iota-graphql-e2e- tests run: `cargo nextest run --features pg_integration` --------- Co-authored-by: Istvan Davorin Acs <istvan.acs@iota.org>
Based on [changes](MystenLabs/sui@6d8ceed). (For original description see the ticket.) Most APIs, command line tools simply do not allow the user to construct syntactically incorrect transaction, but there are APIs which do. Namely ones in which users can send in an already prepared transaction as a byte sequence. ``` Usage: iota client execute-signed-tx [OPTIONS] --tx-bytes <TX_BYTES> ``` or the GraphQL API: ``` [executeTransactionBlock](https://graphql.testnet.iota.cafe/#)( txBytes: [String](https://graphql.testnet.iota.cafe/#)! signatures: [[String](https://graphql.testnet.iota.cafe/#)!]! ): [ExecutionResult](https://graphql.testnet.iota.cafe/#)! ``` or the Json RPC API: ``` iota_executeTransactionBlock ``` In terms of this change the cases in which an invalid **module** or **function** identifier manages to sneak through the checks the system would have deemed it a valid transaction, but it would have failed during execution. Because of this it would have cost gas, forcing it the invalid transaction to be stored on-chain. This would cause downstream interfaces to fail while trying to access the invalid transaction. For example calling this is a correct call for the clock timestamp: `cargo run -p iota -- client call --package 0x2 --module clock --function timestamp_ms --args 0x6` While this would be incorrect: `cargo run -p iota -- client call --package 0x2 --module _ --function timestamp_ms --args 0x6` Now if you tried to execute the second function you would notice that the call fails as the tool properly protects the user. But if you manage to construct a byte sequence containing the same function call with an underscore as a module name, you could sign it and send it in (through the raw byte interfaces), and get the above described issue. This change aims to mitigate these issues. Now the **module** and **function** identifiers are checked for being syntactically valid identifiers, so after the new protocol version, no more transactions can be logged on-chain. Furthermore the GraphQL API has been modified so that it won't crash while trying to parse such an invalid entry. We have added an e2e test for checking the modified behavior. fixes #7081 Choose a type of change, and delete any options that are not relevant. - Bug fix (a non-breaking change which fixes an issue) Regenerate crates/iota-core/tests/staged/iota.yaml: `cargo run -p iota-core --example generate-format -- print > crates/iota-core/tests/staged/iota.yaml` Regenerate crates/iota-graphql-rpc/tests/snapshots/snapshot_tests__schema_sdl_export.snap `cargo nextest run -p iota-graphql-rpc --features pg_integration --no-fail-fast --test-threads 1` Regenerate/update iota-protocol-config snapshots: `cargo nextest run -p iota-protocol-config` Run iota-graphql-e2e-tests: `cargo nextest run -p iota-graphql-e2e-tests --features pg_integration` The new e2e tests: `cargo nextest run -p iota-e2e-tests --test tx_bytes_validity_check` @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. <!-- Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. --> - [x] Protocol: Transactions that include invalid types will not be signed by validators from protocol 10 onwards. - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: `MoveType` can fail to provide a layout or abilities in cases where it represents an invalid type. - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: Ashok Menon <ashok@mystenlabs.com> Co-authored-by: Marc Espin <mespinsanz@gmail.com>
…nsactional test adapter (#7180) [run-ci] According to [changes](MystenLabs/sui@ae367c4#diff-7a132b89ef988a881b2c5d33b4cee85acb20e6be9c7fc645d26806d7ae307290). I as see we don’t have iota-indexer-alt-e2e tests and iota-mvr-graphql-e2e, so changes only touch iota-graphql-e2e- tests. Only these changes differ from Sui’s commit. I assume it’s because of the version differences in fastcrypto or async_graphql. To comply with how iota-graphql-rpc decodes the JSON cursor … this traces back to async_graphql = 7.0.7, which uses no padding for encoding/decoding. `Ok(Base64::encode(cursor)); ` This likely defaults to: - STANDARD base64 - with padding (=) - and possibly using the general character set (+ / /). But the decoder expects: - URL_SAFE_NO_PAD - no padding, using - and _ - matching what async_graphql and the RPC layer parse. ``` return Ok(base64::Engine::encode( &base64::engine::general_purpose::URL_SAFE_NO_PAD, cursor, )); ``` Fixes #7059 . - Enhancement (a non-breaking change which adds functionality) For iota-graphql-e2e- tests run: `cargo nextest run --features pg_integration` --------- Co-authored-by: Istvan Davorin Acs <istvan.acs@iota.org>
…lib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies.
Dkwcs
force-pushed
the
vm-lang/add-implicit-system-package-dependencies
branch
from
67cf03d to
bd89423
Compare
…-1.43.1-1.45.3' into vm-lang/add-implicit-system-package-dependencies
miker83z
reviewed
Jul 23, 2025
Thoralf-M
reviewed
Jul 24, 2025
crates/iota/tests/shell_tests/iota_move_new_tests/new_uppercase_name.sh
Outdated
Show resolved
Hide resolved
crates/iota/tests/snapshots/shell_tests__iota_move_new_tests__new_files.sh.snap
Outdated
Show resolved
Hide resolved
On-chain you only publish the bytes of a CompiledModule. In that case, the dependency "links" (address + name) stored there are the only ones that you declare at the incipit of the module. This change affects the CompiledPackage, adding all the framework packages automatically to the package dependencies. This means that if a CompiledModule declares a dependency it must be present in the CompiledPackage dependency field. However, instances of CompiledPackage are only handled off-chain, so no impact on the gas. |
crates/iota-move/tests/snapshots/cli_tests__new_tests__new_then_build.sh.snap
Outdated
Show resolved
Hide resolved
thibault-martinez
approved these changes
Jul 24, 2025
DaughterOfMars
approved these changes
Jul 25, 2025
miker83z
approved these changes
Jul 25, 2025
Dkwcs
merged commit 488326e
into
vm-lang/upstream-changes/mainnet-1.43.1-1.45.3
36 of 38 checks passed
miker83z
pushed a commit
that referenced
this pull request
Jul 28, 2025
This PR enables you to use system packages (e.g. `MoveStdlib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies. During build, the latest known version is used. `Bridge` examples have been replaced with `Kiosk`. According to the [changes](MystenLabs/sui@46c727e). Fixes #7074 Run for iota crate: `cargo test -p iota --test shell_tests ` `cargo test -p iota --test ptb_tests `
miker83z
pushed a commit
that referenced
this pull request
Jul 28, 2025
This PR enables you to use system packages (e.g. `MoveStdlib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies. During build, the latest known version is used. `Bridge` examples have been replaced with `Kiosk`. According to the [changes](MystenLabs/sui@46c727e). Fixes #7074 Run for iota crate: `cargo test -p iota --test shell_tests ` `cargo test -p iota --test ptb_tests ` - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Implicitly added dependencies on system packages (MoveStdLib, Iota, System, Stardust) if they are not included in Move.toml. - [ ] Rust SDK: - [ ] REST API:
miker83z
pushed a commit
that referenced
this pull request
Jul 28, 2025
This PR enables you to use system packages (e.g. `MoveStdlib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies. During build, the latest known version is used. `Bridge` examples have been replaced with `Kiosk`. According to the [changes](MystenLabs/sui@46c727e). Fixes #7074 Run for iota crate: `cargo test -p iota --test shell_tests ` `cargo test -p iota --test ptb_tests ` - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Implicitly added dependencies on system packages (MoveStdLib, Iota, System, Stardust) if they are not included in Move.toml. - [ ] Rust SDK: - [ ] REST API:
miker83z
pushed a commit
that referenced
this pull request
Jul 28, 2025
This PR enables you to use system packages (e.g. `MoveStdlib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies. During build, the latest known version is used. `Bridge` examples have been replaced with `Kiosk`. According to the [changes](MystenLabs/sui@46c727e). Fixes #7074 Run for iota crate: `cargo test -p iota --test shell_tests ` `cargo test -p iota --test ptb_tests ` - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Implicitly added dependencies on system packages (MoveStdLib, Iota, System, Stardust) if they are not included in Move.toml. - [ ] Rust SDK: - [ ] REST API:
miker83z
pushed a commit
that referenced
this pull request
Jul 29, 2025
This PR enables you to use system packages (e.g. `MoveStdlib`, `Iota`, `IotaSystem`, `Stardust`) without explicitly adding them to your `Move.toml`. It works by querying the network during publication / upgrade for the protocol version and using that information to determine the correct version of the system dependencies. During build, the latest known version is used. `Bridge` examples have been replaced with `Kiosk`. According to the [changes](MystenLabs/sui@46c727e). Fixes #7074 Run for iota crate: `cargo test -p iota --test shell_tests ` `cargo test -p iota --test ptb_tests ` - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Implicitly added dependencies on system packages (MoveStdLib, Iota, System, Stardust) if they are not included in Move.toml. - [ ] Rust SDK: - [ ] REST API:
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
This PR enables you to use system packages (e.g.
MoveStdlib,Iota,IotaSystem,Stardust) without explicitly adding them to yourMove.toml. It worksby querying the network during publication / upgrade for the protocol
version and using that information to determine the correct version of
the system dependencies. During build, the latest known version is used.
Bridgeexamples have been replaced withKiosk.According to the changes.
Links to any relevant issues
Fixes #7074
How the change has been tested
Run for iota crate:
cargo test -p iota --test shell_testscargo test -p iota --test ptb_tests