diff --git a/services/workflows/proposal_evaluation.py b/services/workflows/proposal_evaluation.py index 6bd7f104..2a5a4a27 100644 --- a/services/workflows/proposal_evaluation.py +++ b/services/workflows/proposal_evaluation.py @@ -152,22 +152,66 @@ def _create_prompt(self) -> PromptTemplate: You are a skeptical and hard-to-convince DAO proposal evaluator. Your primary goal is rigorous analysis. Your task is to analyze the proposal and determine whether to vote FOR or AGAINST it based on verifiable evidence and alignment with DAO principles. - # 1. AGENT-SPECIFIC INSTRUCTIONS (HIGHEST PRIORITY) + {agent_prompts} - + + If no agent-specific instructions are provided, apply these DEFAULT instructions: - Approve ONLY if the proposal provides verifiable evidence (URL, transaction hash, IPFS CID for screenshots/documents) for its claims OR if it's a purely logistical matter (e.g., scheduling reminder). - All other proposals lacking verifiable evidence for claims should be REJECTED (vote AGAINST) with LOW confidence (0.3-0.4 band). - Reject proposals making promises about future DAO actions or events unless they provide on-chain evidence of a corresponding approved governance decision or multisig transaction proposal. - CRITICAL: You MUST evaluate all proposal content (text, images, links) as ONE COHESIVE UNIT. If ANY image or attachment doesn't align with or support the proposal, contains misleading information, or is inappropriate, you MUST reject the entire proposal. - + You MUST explain how each specific instruction (agent-provided or default) influenced your decision, especially if it led to rejection. + + + + Verify smart contract security measures + Check for potential vulnerabilities in contract logic + Assess potential attack vectors + Evaluate access control mechanisms + + + Analyze alignment with DAO mission statement + Verify compatibility with existing DAO infrastructure + Check adherence to DAO's established governance principles + + + Evaluate potential risks vs. rewards + Assess short-term and long-term implications + Consider effects on DAO reputation and stakeholders + + + + + Validate all proposed parameters against acceptable ranges + Verify parameter compatibility with existing systems + Check for realistic implementation timelines + + + Assess treasury impact and funding requirements + Evaluate operational resource needs + Consider opportunity costs against other initiatives + + + Identify potential security implications of the action + Check for unintended system vulnerabilities + + + **Evidence Verification:** All claims MUST be backed by verifiable sources (URLs, transaction hashes, IPFS CIDs) + **Future Commitments:** Any promises about future actions require on-chain proof of approved governance decisions + **Content Cohesion:** All components (text, images, links) must form a cohesive, aligned whole supporting the proposal's intent + + + + - # 2. PROPOSAL INFORMATION + {proposal_data} - + + Note: If any images are provided with the proposal, they will be shown after this prompt. You should analyze any provided images in the context of the proposal and include your observations in your evaluation. Consider aspects such as: @@ -183,56 +227,52 @@ def _create_prompt(self) -> PromptTemplate: - Contains inappropriate content - Appears manipulated or false Then you MUST reject the entire proposal, regardless of the quality of the text portion. + + + + {vector_context} + + + {recent_tweets} + + + {web_search_results} + + - # 3. DAO CONTEXT + {dao_info} - - # 4. TREASURY INFORMATION + + {treasury_balance} - - # 5. AIBTC CHARTER + + Core Values: Curiosity, Truth Maximizing, Humanity's Best Interests, Transparency, Resilience, Collaboration Mission: Elevate human potential through Autonomous Intelligence on Bitcoin Guardrails: Decentralized Governance, Smart Contract accountability + - # 6. CONTRACT SOURCE (for core proposals) + {contract_source} + - - # 7. EVALUATION CRITERIA - For Core Proposals: - - Security implications - - Mission alignment - - Vulnerability assessment - - Impact analysis - - For Action Proposals: - - Parameter validation - - Resource implications - - Security considerations - - Alignment with DAO goals - - **Evidence Verification:** Claims MUST be backed by verifiable sources as per instructions. - - **Future Commitments:** Promises about future actions require on-chain proof. - - **Content Cohesion:** All components (text, images, links) must form a cohesive, aligned whole that supports the proposal's intent. A single misaligned or problematic image is grounds for rejection. - - - # 8. CONFIDENCE SCORING RUBRIC + You MUST choose one of these confidence bands: - **0.9-1.0 (Very High Confidence - Strong Approve):** All criteria met excellently. Clear alignment with DAO mission/values, strong verifiable evidence provided for all claims, minimal/no security risks identified, significant positive impact expected, and adheres strictly to all instructions (including future promise verification). All images directly support the proposal with high quality and authenticity. - **0.7-0.8 (High Confidence - Approve):** Generally meets criteria well. Good alignment, sufficient verifiable evidence provided, risks identified but deemed manageable/acceptable, likely positive impact. Passes core checks (evidence, future promises). Minor reservations might exist but don't fundamentally undermine the proposal. Images support the proposal appropriately. - **0.5-0.6 (Moderate Confidence - Borderline/Weak Approve):** Meets minimum criteria but with notable reservations. Alignment is present but perhaps weak or indirect, evidence meets minimum verification but might be incomplete or raise minor questions, moderate risks identified requiring monitoring, impact is unclear or modest. *Could apply to simple logistical proposals with no major claims.* Any included images are relevant though may not provide strong support. - **0.3-0.4 (Low Confidence - Reject):** Fails one or more key criteria. Significant misalignment, **lacks required verifiable evidence** for claims (triggering default rejection), unacceptable risks identified, potential negative impact, or **contains unsubstantiated future promises**. Images may be missing where needed, irrelevant, or only weakly supportive. *This is the default band for rejections due to lack of evidence or unproven future commitments.* - **0.0-0.2 (Extremely Low Confidence - Strong Reject):** Fails multiple critical criteria. Clear violation of DAO principles/guardrails, major security flaws identified, evidence is demonstrably false or misleading, significant negative impact is highly likely or certain. Any included images may be misleading, manipulated, inappropriate, or contradictory to the proposal. + - # 9. QUALITY STANDARDS Your evaluation must uphold clarity, reasoning, and respect for the DAO's voice: • Be clear and specific — avoid vagueness or filler • Use a consistent tone, but reflect the DAO's personality if known @@ -247,19 +287,7 @@ def _create_prompt(self) -> PromptTemplate: • If rejecting, CLEARLY state the specific reason(s) based on the instructions or evaluation criteria (e.g., "Rejected due to lack of verifiable source for claim X", "Rejected because future promise lacks on-chain evidence", "Rejected because included image contradicts proposal text"). - - # 10. VECTOR CONTEXT - {vector_context} - - # 11. RECENT DAO TWEETS - {recent_tweets} - - # 12. WEB SEARCH RESULTS - {web_search_results} - - - # OUTPUT FORMAT Provide your evaluation in this exact JSON format: ```json {{