Question on Terminating Condition

[diffunity]

Hi! Thank you for the innovative work! I had a few questions regarding the terminating conditions of your algorithm.

The paper states two conditions for terminating the iteration of $Q_i$

This iterative process continues until it reaches a maximum number $D$, which is assigned by the depth of the first generated graph $\mathcal{G}_0$ to prevent infinite decomposition.

To optimize test-time efficiency, AoT incorporates an automated termination mechanism that uses LLM evaluation to assess solution quality through answer comparison ... If this synthesized answer demonstrates consistency with the answer produced by $Q_{i+1}$, the iterative process continues.

My understanding was that AoT iterates through $Q_i$ until the minimum of either of the two conditions is reached. Therefore, later on in Figure 3, when the paper mentions "as many problems achieve satisfactory solutions at earlier depths", it is referring to the fact that a new iteration was not necessary because the second condition in the above two conditions was met before reaching the maximum iteration (in this case, it was explicitly set to 5 for the purpose of the experiment)

My first question is: is my understanding correct?

As a follow-up, my additional questions are:

1. Does the term "demonstrates consistency" here mean that the answers to $Q_i$ and $Q_{i+1}$ are the same? (and hence in that condition, AoT continues the next cycles of decomposition-contraction?)

---

2. How is this consistency measured in the code?

atom/experiment/module.py

Line 128 in c9be7ac

async def atom(question: str, contexts: str=None, direct_result=None, decompose_result=None, depth=None, log=None):

This function seems to be the one that runs a single cycle of decomposition-contraction. However, I failed to find the code that compares the results between each iteration of $Q_i$

---

3. How is the depth used in the code?

atom/experiment/module.py

Line 144 in c9be7ac

depth = depth if depth else min(ATOM_DEPTH, calculate_depth(decompose_result["sub-questions"]))

Here, the variable depth is defined but I failed to find where this variable is used.

---

Thank you in advance!