Debug mode

[alfonso-paella]

Is your feature request related to a problem? Please describe it.

Sometimes my agent is getting errors executing a tool, and it would be helpful to get a log that shows what the LLM actually executed, so that the tool can be fixed

Describe the solution you'd like

Modify getTools, or some env var, to set debug true. And then i would get debug logs for all tools. Ideally this works in a way that all tools get this capability, so I think it should probably be implemented by adding this at the wallet layer + HTTP library layer

Describe alternatives you've considered

None

Additional context

None

[devin-ai-integration]

Debug Mode Design Document

Problem Statement

Currently, when agents execute tools through GOAT SDK, there's no way to debug issues or track what the LLM actually executed. This makes troubleshooting difficult when tools fail or behave unexpectedly.

Proposed Solution

Implement a debug mode that can be enabled via configuration to provide detailed logging of tool executions at both the wallet and HTTP library layers.

Design Overview

1. Configuration

export type GetToolsParams<TWalletClient extends WalletClientBase> = {
    wallet: TWalletClient;
    plugins?: (PluginBase<TWalletClient> | PluginBase<WalletClientBase>)[];
    debug?: boolean;  // New parameter
};

2. Debug Context

// New file: src/utils/DebugContext.ts
export class DebugContext {
    private static instance: DebugContext;
    private isEnabled: boolean = false;
    private logs: string[] = [];

    static getInstance(): DebugContext {
        if (!DebugContext.instance) {
            DebugContext.instance = new DebugContext();
        }
        return DebugContext.instance;
    }

    enable() {
        this.isEnabled = true;
    }

    log(scope: string, message: string, data?: any) {
        if (!this.isEnabled) return;
        
        const timestamp = new Date().toISOString();
        const logEntry = {
            timestamp,
            scope,
            message,
            data: data || null
        };
        
        console.debug(JSON.stringify(logEntry));
        this.logs.push(JSON.stringify(logEntry));
    }

    getLogs(): string[] {
        return this.logs;
    }
}

3. Integration Points

a. Tool Execution Layer

// Modify ToolBase.ts
export abstract class ToolBase<TParameters extends z.ZodSchema = z.ZodSchema, TResult = any> {
    // ... existing code ...

    async execute(parameters: z.infer<TParameters>): Promise<TResult> {
        const debug = DebugContext.getInstance();
        debug.log('tool', `Executing tool: ${this.name}`, {
            parameters,
            description: this.description
        });

        try {
            const result = await this._execute(parameters);
            debug.log('tool', `Tool execution completed: ${this.name}`, {
                result
            });
            return result;
        } catch (error) {
            debug.log('tool', `Tool execution failed: ${this.name}`, {
                error: error.message,
                stack: error.stack
            });
            throw error;
        }
    }

    protected abstract _execute(parameters: z.infer<TParameters>): TResult | Promise<TResult>;
}

b. Wallet Layer

// Modify WalletClientBase.ts
export abstract class WalletClientBase {
    // ... existing code ...

    async signMessage(message: string): Promise<Signature> {
        const debug = DebugContext.getInstance();
        debug.log('wallet', 'Signing message', {
            message,
            address: this.getAddress(),
            chain: this.getChain()
        });

        const result = await this._signMessage(message);
        
        debug.log('wallet', 'Message signed', {
            signature: result.signature
        });
        
        return result;
    }

    protected abstract _signMessage(message: string): Promise<Signature>;
}

c. HTTP Layer

// New file: src/utils/debugFetch.ts
export const debugFetch = async (input: RequestInfo, init?: RequestInit) => {
    const debug = DebugContext.getInstance();
    const requestId = Math.random().toString(36).substring(7);
    
    debug.log('http', 'Making HTTP request', {
        requestId,
        url: typeof input === 'string' ? input : input.url,
        method: init?.method || 'GET',
        headers: init?.headers
    });

    try {
        const response = await fetch(input, init);
        const responseData = await response.clone().text();
        
        debug.log('http', 'HTTP response received', {
            requestId,
            status: response.status,
            statusText: response.statusText,
            headers: Object.fromEntries(response.headers.entries()),
            body: responseData
        });

        return response;
    } catch (error) {
        debug.log('http', 'HTTP request failed', {
            requestId,
            error: error.message,
            stack: error.stack
        });
        throw error;
    }
};

4. Initialization

// Modify getTools.ts
export async function getTools<TWalletClient extends WalletClientBase>({
    wallet,
    plugins = [],
    debug = false
}: GetToolsParams<TWalletClient>) {
    if (debug) {
        DebugContext.getInstance().enable();
    }
    
    // ... rest of existing code ...
}

Usage Example

const tools = await getOnChainTools({
    wallet: viem(walletClient),
    plugins: [sendETH(), erc20({ tokens: [USDC] })],
    debug: true  // Enable debug mode
});

Implementation Plan

1. Core Changes

   • Add DebugContext singleton
   • Modify ToolBase to include debug logging
   • Update WalletClientBase with debug hooks
   • Create debugFetch utility

2. Documentation

   • Update README with debug mode documentation
   • Add examples of debug output format
   • Document environment variable support

3. Testing

   • Add unit tests for DebugContext
   • Add integration tests for debug logging
   • Test across different plugins and wallets

Considerations

1. Performance Impact

   • Debug logging is only enabled when explicitly requested
   • Logs are stored in memory with optional console output
   • Consider log rotation for long-running sessions

2. Security

   • Sensitive data (private keys, signatures) should be redacted in logs
   • Debug mode should be disabled by default
   • Consider adding log sanitization utilities

3. Extensibility

   • Plugin authors can access DebugContext for custom logging
   • Format is standardized but extensible
   • Support for custom log handlers can be added later

Future Enhancements

1. Log Storage

   • Option to write logs to file
   • Integration with external logging services

2. Log Analysis

   • Tools for parsing and analyzing debug logs
   • Visualization of tool execution flows

3. Real-time Monitoring

   • WebSocket support for real-time log streaming
   • Debug UI for monitoring tool executions

[devin-ai-integration]

Debug Mode Design Document

Problem Statement

Currently, when agents execute tools through GOAT SDK, there's no way to debug issues or track what the LLM actually executed. This makes troubleshooting difficult when tools fail or behave unexpectedly.

Proposed Solution

Implement a debug mode that can be enabled via configuration to provide detailed logging of tool executions at both the wallet and HTTP library layers.

Design Overview

1. Configuration

export type GetToolsParams<TWalletClient extends WalletClientBase> = {
    wallet: TWalletClient;
    plugins?: (PluginBase<TWalletClient> | PluginBase<WalletClientBase>)[];
    debug?: boolean;  // New parameter
};

2. Debug Context

// New file: src/utils/DebugContext.ts
export class DebugContext {
    private static instance: DebugContext;
    private isEnabled: boolean = false;
    private logs: string[] = [];

    static getInstance(): DebugContext {
        if (!DebugContext.instance) {
            DebugContext.instance = new DebugContext();
        }
        return DebugContext.instance;
    }

    enable() {
        this.isEnabled = true;
    }

    log(scope: string, message: string, data?: any) {
        if (!this.isEnabled) return;
        
        const timestamp = new Date().toISOString();
        const logEntry = {
            timestamp,
            scope,
            message,
            data: data || null
        };
        
        console.debug(JSON.stringify(logEntry));
        this.logs.push(JSON.stringify(logEntry));
    }

    getLogs(): string[] {
        return this.logs;
    }
}

3. Integration Points

a. Tool Execution Layer

// Modify ToolBase.ts
export abstract class ToolBase<TParameters extends z.ZodSchema = z.ZodSchema, TResult = any> {
    // ... existing code ...

    async execute(parameters: z.infer<TParameters>): Promise<TResult> {
        const debug = DebugContext.getInstance();
        debug.log('tool', `Executing tool: ${this.name}`, {
            parameters,
            description: this.description
        });

        try {
            const result = await this._execute(parameters);
            debug.log('tool', `Tool execution completed: ${this.name}`, {
                result
            });
            return result;
        } catch (error) {
            debug.log('tool', `Tool execution failed: ${this.name}`, {
                error: error.message,
                stack: error.stack
            });
            throw error;
        }
    }

    protected abstract _execute(parameters: z.infer<TParameters>): TResult | Promise<TResult>;
}

b. Wallet Layer

// Modify WalletClientBase.ts
export abstract class WalletClientBase {
    // ... existing code ...

    async signMessage(message: string): Promise<Signature> {
        const debug = DebugContext.getInstance();
        debug.log('wallet', 'Signing message', {
            message,
            address: this.getAddress(),
            chain: this.getChain()
        });

        const result = await this._signMessage(message);
        
        debug.log('wallet', 'Message signed', {
            signature: result.signature
        });
        
        return result;
    }

    protected abstract _signMessage(message: string): Promise<Signature>;
}

c. HTTP Layer

// New file: src/utils/debugFetch.ts
export const debugFetch = async (input: RequestInfo, init?: RequestInit) => {
    const debug = DebugContext.getInstance();
    const requestId = Math.random().toString(36).substring(7);
    
    // Parse URL and query parameters
    const url = typeof input === 'string' ? new URL(input) : new URL(input.url);
    const queryParams = Object.fromEntries(url.searchParams.entries());
    
    // Parse request body if present
    let requestBody = null;
    if (init?.body) {
        try {
            requestBody = typeof init.body === 'string' 
                ? JSON.parse(init.body) 
                : init.body instanceof FormData 
                    ? Object.fromEntries(init.body.entries())
                    : init.body;
        } catch {
            requestBody = init.body; // Keep as string if not JSON
        }
    }
    
    debug.log('http', 'Making HTTP request', {
        requestId,
        url: url.toString(),
        method: init?.method || 'GET',
        headers: init?.headers,
        queryParams,
        body: requestBody
    });

    try {
        const response = await fetch(input, init);
        const responseData = await response.clone().text();
        let parsedResponseData = responseData;
        
        try {
            parsedResponseData = JSON.parse(responseData);
        } catch {
            // Keep as string if not JSON
        }
        
        debug.log('http', 'HTTP response received', {
            requestId,
            status: response.status,
            statusText: response.statusText,
            headers: Object.fromEntries(response.headers.entries()),
            body: parsedResponseData
        });

        return response;
    } catch (error) {
        debug.log('http', 'HTTP request failed', {
            requestId,
            error: error.message,
            stack: error.stack
        });
        throw error;
    }
};

Plugin Integration Example

// Example plugin using debugFetch
import { debugFetch } from '../utils/debugFetch';
import { PluginBase } from '../classes/PluginBase';
import { WalletClientBase } from '../classes/WalletClientBase';
import { createTool } from '../classes/ToolBase';
import { z } from 'zod';

export class ExampleAPIPlugin extends PluginBase<WalletClientBase> {
    name = 'example-api';
    
    async getPrice(symbol: string): Promise<number> {
        // Use debugFetch instead of regular fetch
        const response = await debugFetch(
            `https://api.example.com/v1/prices?symbol=${symbol}`,
            {
                method: 'GET',
                headers: {
                    'Content-Type': 'application/json'
                }
            }
        );
        
        const data = await response.json();
        return data.price;
    }
    
    async createOrder(symbol: string, amount: number): Promise<string> {
        // Example POST request with body
        const response = await debugFetch(
            'https://api.example.com/v1/orders',
            {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json'
                },
                body: JSON.stringify({
                    symbol,
                    amount
                })
            }
        );
        
        const data = await response.json();
        return data.orderId;
    }
    
    getTools(wallet: WalletClientBase) {
        return [
            createTool(
                {
                    name: 'get_price',
                    description: 'Get the current price of an asset',
                    parameters: z.object({
                        symbol: z.string()
                    })
                },
                async (params) => this.getPrice(params.symbol)
            ),
            createTool(
                {
                    name: 'create_order',
                    description: 'Create a new order',
                    parameters: z.object({
                        symbol: z.string(),
                        amount: z.number()
                    })
                },
                async (params) => this.createOrder(params.symbol, params.amount)
            )
        ];
    }
}

When debug mode is enabled, the above plugin will automatically log:

1. API requests with query parameters (get_price)
2. POST requests with JSON body (create_order)
3. Response data and status codes
4. Any HTTP errors that occur

Example debug output for getPrice:

{
    "timestamp": "2024-01-20T10:30:00.000Z",
    "scope": "http",
    "message": "Making HTTP request",
    "data": {
        "requestId": "abc123",
        "url": "https://api.example.com/v1/prices?symbol=ETH",
        "method": "GET",
        "headers": {
            "Content-Type": "application/json"
        },
        "queryParams": {
            "symbol": "ETH"
        }
    }
}

Example debug output for createOrder:

{
    "timestamp": "2024-01-20T10:31:00.000Z",
    "scope": "http",
    "message": "Making HTTP request",
    "data": {
        "requestId": "def456",
        "url": "https://api.example.com/v1/orders",
        "method": "POST",
        "headers": {
            "Content-Type": "application/json"
        },
        "body": {
            "symbol": "ETH",
            "amount": 1.5
        }
    }
}

Future Enhancements

1. Log Storage

   • Option to write logs to file
   • Integration with external logging services

2. Log Analysis

   • Tools for parsing and analyzing debug logs
   • Visualization of tool execution flows

3. Real-time Monitoring

   • WebSocket support for real-time log streaming
   • Debug UI for monitoring tool executions