Improve error handling for HTTP status code 0 errors

README.md

@@ -20,7 +20,6 @@ Then you can run `ap-avs` binary. We make an effort to use pure Go so you can al
 
 Check how to run an [operator docs](docs/operator.md)
 
-
 ### Run aggregator
 
 To run the aggregator, use the following command:
@@ -35,7 +34,6 @@ Note: The Ava Protocol team currently manages the aggregator, and the communicat
 
 <table><tr><td bgcolor='white'><img src="docs/highlevel-diagram.png"/></td></tr></table>
 
-
 ## User wallet
 
 For each owner we deploy a ERC6900 wallet to schedule task and approve spending
@@ -71,7 +69,6 @@ in the operator config file.
 - aggregator.avaprotocol.org:2206
 - [https://api-explorer.avaprotocol.org/](https://api-explorer.avaprotocol.org/)
 
-
 ## Operators
 
 Operators communicates with aggregators through RPC. It requests task data from aggregator, it performs condition execution to check whether a task can be trigger. The result is then sent back to aggregator.
@@ -85,7 +82,7 @@ Currently, Ava Protocol has deployed our operator on the testnet. Community memb
 
 ### Testnet
 
-- Ava Protocol's operator: [0x997e5d40a32c44a3d93e59fc55c4fd20b7d2d49d](https://holesky.eigenlayer.xyz/operator/0x997e5d40a32c44a3d93e59fc55c4fd20b7d2d49d). 
+- Ava Protocol's operator: [0x997e5d40a32c44a3d93e59fc55c4fd20b7d2d49d](https://holesky.eigenlayer.xyz/operator/0x997e5d40a32c44a3d93e59fc55c4fd20b7d2d49d).
 
 ### Mainnet
 
@@ -110,10 +107,52 @@ View docs/development.md
 
 ## Testing
 
-The commands to run tests locally are found in the Makefile:
-* `go test -race -buildvcs -vet=off ./...` (default)
-* `go test -v -race -buildvcs ./...` (verbose)
+### Standard Tests
+
+The Makefile includes two primary test configurations:
+
+```bash
+# Default test suite
+go test -race -buildvcs -vet=off ./...
+
+# Verbose test output
+go test -v -race -buildvcs ./...
+```
+
+### Enhanced Test Output
+
+For improved test result formatting, use `gotestfmt`:
+
+1. Install the formatter:
+
+   ```bash
+   go install github.com/gotesttools/gotestfmt/v2/cmd/gotestfmt@latest
+   ```
+
+2. Run once in the current terminal session to make Bash scripts more robust and error-aware.
+
+   ```bash
+   set -euo pipefail
+   ```
+
+3. Run tests with formatted output:
+
+   Run all tests with complete output:
+
+   ```base
+   go test -json ./... | gotestfmt
+   ```
+
+   or, run selected test cases:
+
+   ```bash
+   go test -json -run ^TestRestRequestErrorHandling$ ./... 2>&1 | gotestfmt --hide=all
+   ```
+
+   The `--hide=all` flag suppresses output for skipped and successful tests, showing only failures. For more output configuration options, see the [gotestfmt documentation](https://github.com/GoTestTools/gotestfmt?tab=readme-ov-file#how-do-i-make-the-output-less-verbose).
+
 =======
+
 ## Linting and Code Quality
 
 ### Running the linter
@@ -136,7 +175,6 @@ make audit
 - Include linting in CI/CD pipelines to enforce code quality standards
 - Fix linting issues as they arise rather than letting them accumulate
 
-
 ## Dependencies
 
 ### EigenLayer CLI
@@ -163,18 +201,20 @@ Install the Foundry toolchain with the following commands:
 curl -L https://foundry.paradigm.xyz | bash
 foundryup
 ```
+
 ### Protobuf Compiler
 
 ```
 go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
 ```
 
 ### Generate API Docs from Proto Files
+
 1. Install the `protoc-gen-doc` plugin for `protoc`.
    Install via Go:
    ```
    go install github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc@latest
-   ```  
+   ```
    Ensure the plugin is in your PATH:
    ```
    export PATH="$PATH:$(go env GOPATH)/bin"
@@ -184,20 +224,20 @@ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
    which protoc-gen-doc
    ```
 2. Generate API references in HTML format
-    ```
-    protoc --doc_out=./docs --doc_opt=html,docs.html ./protobuf/avs.proto
-    ```
 
-3. Alternatively, generate API references in Markdown format
-    ```
-    protoc --doc_out=./docs --doc_opt=markdown,docs.md ./protobuf/avs.proto
-    ```
-    This command will generate a markdown version of the gRPC API documentation. To enhance the clarity of the generated documentation, the following improvements should be made to the .proto file:
-    - **Group Definitions by Command**: Organize the API methods and their descriptions by command categories to make it easier for users to find relevant information.
-    - **Elaborate on Input Fields**: Provide detailed descriptions for each input field, including data types, expected values, and any constraints or special considerations.
-    - **Add Examples**: Include usage examples for each API method to demonstrate how to construct requests and interpret responses.
-    - **Link to Related Resources**: Where applicable, link to additional resources or documentation that provide further context or implementation details.
+   ```
+   protoc --doc_out=./docs --doc_opt=html,docs.html ./protobuf/avs.proto
+   ```
 
+3. Alternatively, generate API references in Markdown format
+   ```
+   protoc --doc_out=./docs --doc_opt=markdown,docs.md ./protobuf/avs.proto
+   ```
+   This command will generate a markdown version of the gRPC API documentation. To enhance the clarity of the generated documentation, the following improvements should be made to the .proto file:
+   - **Group Definitions by Command**: Organize the API methods and their descriptions by command categories to make it easier for users to find relevant information.
+   - **Elaborate on Input Fields**: Provide detailed descriptions for each input field, including data types, expected values, and any constraints or special considerations.
+   - **Add Examples**: Include usage examples for each API method to demonstrate how to construct requests and interpret responses.
+   - **Link to Related Resources**: Where applicable, link to additional resources or documentation that provide further context or implementation details.
 
 ## Getting started
 
@@ -207,31 +247,29 @@ Coming soon
 
 ### Holesky Testnet
 
-| Name                     | Address                                                                                                                 		|
-| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------	|
-| ProxyAdmin              | [`0x26CF7A7DF7d1E00D83A5Ca24385f697a3ca4577d`](https://holesky.etherscan.io/address/0x26CF7A7DF7d1E00D83A5Ca24385f697a3ca4577d) 	|
-| ServiceManager          | [`0xEA3E82F9Ae371A6a372A6DCffB1a9bD17e0608eF`](https://holesky.etherscan.io/address/0xEA3E82F9Ae371A6a372A6DCffB1a9bD17e0608eF) 	|
-| RegistryCoordinator     | [`0x90c6d6f2A78d5Ce22AB8631Ddb142C03AC87De7a`](https://holesky.etherscan.io/address/0x90c6d6f2A78d5Ce22AB8631Ddb142C03AC87De7a) 	|
-| BLSApkRegistry          | [`0x6752F8BeeE5BF45c9d11FDBC4F8aFfF879925585`](https://holesky.etherscan.io/address/0x6752F8BeeE5BF45c9d11FDBC4F8aFfF879925585) 	|
-| IndexRegistry           | [`0x298a5d3C8F8Db30E8292C9e2BF92292de469C8FF`](https://holesky.etherscan.io/address/0x298a5d3C8F8Db30E8292C9e2BF92292de469C8FF) 	|
-| OperatorStateRetriever  | [`0xb7bb920538e038DFFEfcB55caBf713652ED2031F`](https://holesky.etherscan.io/address/0xb7bb920538e038DFFEfcB55caBf713652ED2031F) 	|
-| PauserRegistry          | [`0x3A8ea6e4202CdDe4a9e0cCE19c4Dc1739ba2cF0b`](https://holesky.etherscan.io/address/0x3A8ea6e4202CdDe4a9e0cCE19c4Dc1739ba2cF0b) 	|
-| StakeRegistry           | [`0x7BacD5dd5A7C3acf8bf1a3c88fB0D00B68EE626A`](https://holesky.etherscan.io/address/0x7BacD5dd5A7C3acf8bf1a3c88fB0D00B68EE626A) 	|
-| ApConfig                | [`0xb8abbb082ecaae8d1cd68378cf3b060f6f0e07eb`](https://holesky.etherscan.io/address/0xb8abbb082ecaae8d1cd68378cf3b060f6f0e07eb) 	|
-
-
+| Name                   | Address                                                                                                                         |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
+| ProxyAdmin             | [`0x26CF7A7DF7d1E00D83A5Ca24385f697a3ca4577d`](https://holesky.etherscan.io/address/0x26CF7A7DF7d1E00D83A5Ca24385f697a3ca4577d) |
+| ServiceManager         | [`0xEA3E82F9Ae371A6a372A6DCffB1a9bD17e0608eF`](https://holesky.etherscan.io/address/0xEA3E82F9Ae371A6a372A6DCffB1a9bD17e0608eF) |
+| RegistryCoordinator    | [`0x90c6d6f2A78d5Ce22AB8631Ddb142C03AC87De7a`](https://holesky.etherscan.io/address/0x90c6d6f2A78d5Ce22AB8631Ddb142C03AC87De7a) |
+| BLSApkRegistry         | [`0x6752F8BeeE5BF45c9d11FDBC4F8aFfF879925585`](https://holesky.etherscan.io/address/0x6752F8BeeE5BF45c9d11FDBC4F8aFfF879925585) |
+| IndexRegistry          | [`0x298a5d3C8F8Db30E8292C9e2BF92292de469C8FF`](https://holesky.etherscan.io/address/0x298a5d3C8F8Db30E8292C9e2BF92292de469C8FF) |
+| OperatorStateRetriever | [`0xb7bb920538e038DFFEfcB55caBf713652ED2031F`](https://holesky.etherscan.io/address/0xb7bb920538e038DFFEfcB55caBf713652ED2031F) |
+| PauserRegistry         | [`0x3A8ea6e4202CdDe4a9e0cCE19c4Dc1739ba2cF0b`](https://holesky.etherscan.io/address/0x3A8ea6e4202CdDe4a9e0cCE19c4Dc1739ba2cF0b) |
+| StakeRegistry          | [`0x7BacD5dd5A7C3acf8bf1a3c88fB0D00B68EE626A`](https://holesky.etherscan.io/address/0x7BacD5dd5A7C3acf8bf1a3c88fB0D00B68EE626A) |
+| ApConfig               | [`0xb8abbb082ecaae8d1cd68378cf3b060f6f0e07eb`](https://holesky.etherscan.io/address/0xb8abbb082ecaae8d1cd68378cf3b060f6f0e07eb) |
 
 ### Ethereum Mainnet
 
-| Name                     | Address                                                                                                                 		|
-| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------	|
-| ProxyAdmin              | [`0x5989934D31f7f397511f105B7E4175a06B7A517F`](https://etherscan.io/address/0x5989934D31f7f397511f105B7E4175a06B7A517F) 	|
-| ServiceManager          | [`0x18343Aa10e3D2F3A861e5649627324aEAD987Adf`](https://etherscan.io/address/0x18343Aa10e3D2F3A861e5649627324aEAD987Adf) 	|
-| RegistryCoordinator     | [`0x8DE3Ee0dE880161Aa0CD8Bf9F8F6a7AfEeB9A44B`](https://etherscan.io/address/0x8DE3Ee0dE880161Aa0CD8Bf9F8F6a7AfEeB9A44B) 	|
-| BLSApkRegistry          | [`0xB58687fF303C8e92C28a484342755d3228081d45`](https://etherscan.io/address/0xB58687fF303C8e92C28a484342755d3228081d45) 	|
-| IndexRegistry           | [`0xc6A464e39d4fA5013D61295501c7cCd050d76612`](https://etherscan.io/address/0xc6A464e39d4fA5013D61295501c7cCd050d76612) 	|
-| OperatorStateRetriever  | [`0xb3af70D5f72C04D1f490ff49e5aB189fA7122713`](https://etherscan.io/address/0xb3af70D5f72C04D1f490ff49e5aB189fA7122713) 	|
-| PauserRegistry          | [`0xeec585186c37c517030ba371deac5c17e728c135`](https://etherscan.io/address/0xeec585186c37c517030ba371deac5c17e728c135) 	|
-| StakeRegistry           | [`0x363b3604fE8c2323a98c00906115c8b87a512a12`](https://etherscan.io/address/0x363b3604fE8c2323a98c00906115c8b87a512a12) 	|
-| TaskManager             | [`0x940f62f75cbbbd723d37c9171dc681dfba653b49`](https://etherscan.io/address/0x940f62f75cbbbd723d37c9171dc681dfba653b49) 	|
-| ApConfig                | [`0x9c02dfc92eea988902a98919bf4f035e4aaefced`](https://etherscan.io/address/0x9c02dfc92eea988902a98919bf4f035e4aaefced) 	|
+| Name                   | Address                                                                                                                 |
+| ---------------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| ProxyAdmin             | [`0x5989934D31f7f397511f105B7E4175a06B7A517F`](https://etherscan.io/address/0x5989934D31f7f397511f105B7E4175a06B7A517F) |
+| ServiceManager         | [`0x18343Aa10e3D2F3A861e5649627324aEAD987Adf`](https://etherscan.io/address/0x18343Aa10e3D2F3A861e5649627324aEAD987Adf) |
+| RegistryCoordinator    | [`0x8DE3Ee0dE880161Aa0CD8Bf9F8F6a7AfEeB9A44B`](https://etherscan.io/address/0x8DE3Ee0dE880161Aa0CD8Bf9F8F6a7AfEeB9A44B) |
+| BLSApkRegistry         | [`0xB58687fF303C8e92C28a484342755d3228081d45`](https://etherscan.io/address/0xB58687fF303C8e92C28a484342755d3228081d45) |
+| IndexRegistry          | [`0xc6A464e39d4fA5013D61295501c7cCd050d76612`](https://etherscan.io/address/0xc6A464e39d4fA5013D61295501c7cCd050d76612) |
+| OperatorStateRetriever | [`0xb3af70D5f72C04D1f490ff49e5aB189fA7122713`](https://etherscan.io/address/0xb3af70D5f72C04D1f490ff49e5aB189fA7122713) |
+| PauserRegistry         | [`0xeec585186c37c517030ba371deac5c17e728c135`](https://etherscan.io/address/0xeec585186c37c517030ba371deac5c17e728c135) |
+| StakeRegistry          | [`0x363b3604fE8c2323a98c00906115c8b87a512a12`](https://etherscan.io/address/0x363b3604fE8c2323a98c00906115c8b87a512a12) |
+| TaskManager            | [`0x940f62f75cbbbd723d37c9171dc681dfba653b49`](https://etherscan.io/address/0x940f62f75cbbbd723d37c9171dc681dfba653b49) |
+| ApConfig               | [`0x9c02dfc92eea988902a98919bf4f035e4aaefced`](https://etherscan.io/address/0x9c02dfc92eea988902a98919bf4f035e4aaefced) |

core/taskengine/engine.go

@@ -603,7 +603,7 @@ func (n *Engine) TriggerTask(user *model.User, payload *avsproto.UserTriggerTask
 			}, nil
 		}
 
-		return nil, grpcstatus.Errorf(codes.Code(avsproto.Error_TaskTriggerError), fmt.Sprintf("error trigger task: %s", err.Error()))
+		return nil, grpcstatus.Errorf(codes.Code(avsproto.Error_TaskTriggerError), "Error trigger task: %v", err)
 	}
 
 	data, err := json.Marshal(queueTaskData)

core/taskengine/executor.go

@@ -171,5 +171,5 @@ func (x *TaskExecutor) RunTask(task *model.Task, queueData *QueueExecutionData)
 		x.logger.Info("succesfully executing task", "task_id", task.Id, "triggermark", triggerMetadata)
 		return execution, nil
 	}
-	return execution, fmt.Errorf("Error executing task %s %v", task.Id, runTaskErr)
+	return execution, fmt.Errorf("Error executing task %s: %v", task.Id, runTaskErr)
 }

core/taskengine/vm_runner_contract_write.go

@@ -1,12 +1,15 @@
 package taskengine
 
 import (
+	"context"
+	"encoding/json"
 	"fmt"
 	"math/big"
 	"strings"
 	"time"
 
 	"github.com/ethereum/go-ethereum/common"
+	"github.com/ethereum/go-ethereum/core/types"
 	"github.com/ethereum/go-ethereum/ethclient"
 
 	"github.com/AvaProtocol/ap-avs/core/chainio/aa"
@@ -77,11 +80,11 @@ func (r *ContractWriteProcessor) Execute(stepID string, node *avsproto.ContractW
 	log.WriteString("\nsend userops to bundler rpc\n")
 
 	total, _ := r.vm.db.GetCounter(ContractWriteCounterKey(r.owner), 0)
-	
+
 	var paymasterRequest *preset.VerifyingPaymasterRequest
 	// TODO: move to config
 	if total < 10 {
-		paymasterRequest = preset.GetVerifyingPaymasterRequestForDuration(r.smartWalletConfig.PaymasterAddress, 15 * time.Minute)
+		paymasterRequest = preset.GetVerifyingPaymasterRequestForDuration(r.smartWalletConfig.PaymasterAddress, 15*time.Minute)
 	}
 
 	userOp, txReceipt, err := preset.SendUserOp(
@@ -97,17 +100,6 @@ func (r *ContractWriteProcessor) Execute(stepID string, node *avsproto.ContractW
 	}
 	r.vm.db.IncCounter(ContractWriteCounterKey(r.owner), 0)
 
-	var bloom []byte
-	if txReceipt != nil {
-		bloom, _ = txReceipt.Bloom.MarshalText()
-	}
-
-	blobGasPrice := uint64(0)
-
-	if txReceipt != nil && txReceipt.BlobGasPrice != nil {
-		blobGasPrice = uint64(txReceipt.BlobGasPrice.Int64())
-	}
-
 	outputData := &avsproto.Execution_Step_ContractWrite{
 		ContractWrite: &avsproto.ContractWriteNode_Output{
 			UserOp: &avsproto.Evm_UserOp{
@@ -123,31 +115,63 @@ func (r *ContractWriteProcessor) Execute(stepID string, node *avsproto.ContractW
 				PaymasterAndData:     common.Bytes2Hex(userOp.PaymasterAndData),
 				Signature:            common.Bytes2Hex(userOp.Signature),
 			},
-
-			TxReceipt: &avsproto.Evm_TransactionReceipt{
-				Hash:        txReceipt.TxHash.Hex(),
-				BlockHash:   txReceipt.BlockHash.Hex(),
-				BlockNumber: uint64(txReceipt.BlockNumber.Int64()),
-				// TODO: Need to fetch this, it isn't available
-				//From:            txReceipt.From.Hex(),
-				//To:              txReceipt.To.Hex(),
-				GasUsed:           txReceipt.GasUsed,
-				GasPrice:          uint64(txReceipt.EffectiveGasPrice.Int64()),
-				CumulativeGasUsed: txReceipt.CumulativeGasUsed,
-				// Fee:                 txReceipt.Fee,
-				ContractAddress: txReceipt.ContractAddress.Hex(),
-				Index:           uint64(txReceipt.TransactionIndex),
-				// TODO: convert raw log
-				//Logs:                txReceipt.Logs,
-				LogsBloom:    common.Bytes2Hex(bloom),
-				Root:         common.Bytes2Hex(txReceipt.PostState),
-				Status:       uint32(txReceipt.Status),
-				Type:         uint32(txReceipt.Type),
-				BlobGasPrice: blobGasPrice,
-				BlobGasUsed:  uint64(txReceipt.BlobGasUsed),
-			},
 		},
 	}
+
+	// Only add TxReceipt if it exists
+	if txReceipt != nil {
+		var bloom []byte
+		bloom, _ = txReceipt.Bloom.MarshalText()
+
+		blobGasPrice := uint64(0)
+		if txReceipt.BlobGasPrice != nil {
+			blobGasPrice = uint64(txReceipt.BlobGasPrice.Int64())
+		}
+
+		// Get the transaction to access From and To fields
+		tx, _, err := r.client.TransactionByHash(context.Background(), txReceipt.TxHash)
+		if err != nil {
+			return nil, fmt.Errorf("failed to get transaction: %w", err)
+		}
+
+		// Get the sender address using the newer method
+		signer := types.LatestSignerForChainID(tx.ChainId())
+		from, err := types.Sender(signer, tx)
+		if err != nil {
+			return nil, fmt.Errorf("failed to get sender from transaction: %w", err)
+		}
+
+		outputData.ContractWrite.TxReceipt = &avsproto.Evm_TransactionReceipt{
+			Hash:              txReceipt.TxHash.Hex(),
+			BlockHash:         txReceipt.BlockHash.Hex(),
+			BlockNumber:       uint64(txReceipt.BlockNumber.Int64()),
+			From:              from.Hex(),
+			To:                tx.To().Hex(),
+			GasUsed:           txReceipt.GasUsed,
+			GasPrice:          uint64(txReceipt.EffectiveGasPrice.Int64()),
+			CumulativeGasUsed: txReceipt.CumulativeGasUsed,
+			Fee:               uint64(txReceipt.GasUsed * txReceipt.EffectiveGasPrice.Uint64()),
+			ContractAddress:   txReceipt.ContractAddress.Hex(),
+			Index:             uint64(txReceipt.TransactionIndex),
+			Logs:              make([]string, len(txReceipt.Logs)),
+			LogsBloom:         common.Bytes2Hex(bloom),
+			Root:              common.Bytes2Hex(txReceipt.PostState),
+			Status:            uint32(txReceipt.Status),
+			Type:              uint32(txReceipt.Type),
+			BlobGasPrice:      blobGasPrice,
+			BlobGasUsed:       uint64(txReceipt.BlobGasUsed),
+		}
+
+		// Convert logs to JSON strings for storage in the protobuf message
+		for i, log := range txReceipt.Logs {
+			logBytes, err := json.Marshal(log)
+			if err != nil {
+				return nil, fmt.Errorf("failed to marshal log: %w", err)
+			}
+			outputData.ContractWrite.TxReceipt.Logs[i] = string(logBytes)
+		}
+	}
+
 	s.OutputData = outputData
 	r.SetOutputVarForStep(stepID, map[string]any{
 		"userOp":    outputData.ContractWrite.UserOp,