Weird cyclic error

1. What is the issue? Please be detailed.

when I validate a form I am getting the famous

XForm is invalid.
: Cycle detected in form's relevant and calculation logic!

But that should not be possible because that form is build with tricc that stoip in case of cycle

2. What steps can we take to reproduce this issue?

Simply use that form

3. What have you tried to fix the issue?

this is a super complex form so I ask LLM to make a mermaid visualisation of the cycle

import pandas as pd
import re

# ========================= CONFIG =========================
file_path = "/mnt/data/Development/tricc-mage/output/adult.xlsx"
sheet_name = "survey"

# ←←← YOUR PREFERENCES
only_core_nodes = False          # False = full expanded graph (recommended)
                                 # True  = ONLY your 9 core nodes

# ←←← NEW: SPACING CONTROLS (this is what you asked for!)
node_spacing = 180               # ← Increase this for MORE horizontal space between nodes
rank_spacing = 220               # ← Increase this for MORE vertical space between levels
                                 #    (try 200–300 if your graph feels crowded)


core_names = [
"flag_dehydration_ap",
"ask_ooUbYRn_3wWAAqequ05F_0_Vv_3",
"flag_dehydration_bp"
]
# =========================================================

df = pd.read_excel(file_path, sheet_name=sheet_name)

relevant_col = next((c for c in df.columns if str(c).lower() in ['relevant', 'relevance']), 'relevant')
calculation_col = next((c for c in df.columns if str(c).lower() in ['calculation', 'calculate']), 'calculation')
constraint_col = next((c for c in df.columns if str(c).lower() in ['constraint', 'constrains']), 'constraint')
def extract_refs(text):
    if pd.isna(text) or not isinstance(text, str):
        return []
    return re.findall(r'\$\{([^\}]+)\}', text)

core_set = set(core_names)

if only_core_nodes:
    all_names = core_set
    nodes_df = df[df['name'].isin(core_set)].copy()
    mode = "CORE ONLY"
else:
    all_names = set(core_names)
    for _, row in df[df['name'].isin(core_names)].iterrows():
        for col in [relevant_col, calculation_col]:
            for ref in extract_refs(row.get(col, '')):
                if ref in df['name'].values:
                    all_names.add(ref)
    nodes_df = df[df['name'].isin(all_names)].copy()
    mode = "EXPANDED"

print(f"βœ… Mode: {mode} β†’ {len(nodes_df)} nodes")

# ========================= MERMAID GENERATION =========================
mermaid_lines = [
    # ←←← SPACING CONFIG (this ensures nodes are more spaced)
    f"%%{{init: {{'flowchart': {{'nodeSpacing': {node_spacing}, 'rankSpacing': {rank_spacing}}}}}}}%%",
    "flowchart TD"
]

# Nodes with two styles
label_col = next((c for c in df.columns if str(c).lower().startswith('label')), None)
for _, row in nodes_df.iterrows():
    name = row['name']
    raw_label = row.get(label_col, '') if label_col else ''
    display = str(raw_label).strip()[:120] if pd.notna(raw_label) and str(raw_label).strip() else name
    display = display.replace('"', '\\"').replace('\n', ' ')
    node_text = f"{name}\\n{display}" if display != name else name
    
    style = ":::core" if name in core_set else ":::dep"
    mermaid_lines.append(f'    {name}["{node_text}"]{style}')

# Edges (no duplicates)
seen = set()
for _, row in nodes_df.iterrows():
    target = row['name']
    
    if pd.notna(row.get(relevant_col)):
        for source in extract_refs(row[relevant_col]):
            if source in all_names and source != target:
                key = (source, target, 'r')
                if key not in seen:
                    seen.add(key)
                    mermaid_lines.append(f'    {source} --r--> {target}')
    
    if pd.notna(row.get(calculation_col)):
        for source in extract_refs(row[calculation_col]):
            if source in all_names and source != target:
                key = (source, target, 'c')
                if key not in seen:
                    seen.add(key)
                    mermaid_lines.append(f'    {source} --c--> {target}')
    if pd.notna(row.get(calculation_col)):
        for source in extract_refs(row[constraint_col]):
            if source in all_names and source != target:
                key = (source, target, 'x')
                if key not in seen:
                    seen.add(key)
                    mermaid_lines.append(f'    {source} --x--> {target}')


# Style definitions
mermaid_lines.append('')
mermaid_lines.append('classDef core fill:#ffeb3b,stroke:#d32f2f,stroke-width:4px,color:#000,font-weight:bold;')
mermaid_lines.append('classDef dep  fill:#bbdefb,stroke:#1976d2,stroke-width:2px,color:#333;')

mermaid_code = '\n'.join(mermaid_lines)

print("\n" + "="*70)
print(f"MERMAID CODE ({mode} – {len(nodes_df)} nodes, {len(seen)} edges)")
print("="*70)
print(mermaid_code)
print("="*70)

# Save
output_path = "/mnt/data/Development/tricc-mage/output/dependency_graph.mmd"
with open(output_path, "w", encoding="utf-8") as f:
    f.write(mermaid_code)

print(f"βœ… Saved to: {output_path}")
print("   Paste into https://mermaid.live")
print("\n🎨 Spacing is now controlled by the two numbers at the top.")
print("   Want even more space? Just increase node_spacing / rank_spacing and run again!")

this gives me that result : NO CYCLE DISPLAYED

Is it possible that I reach the limitation of the validator , if not do you have tool to investigate ?

4. Upload any forms or screenshots you can share publicly below.

adult.xlsx (131.4 KB)

1 Like

Ciao @delcroip

I went through the form, and indeed, there are many nodes referring to themselves (loop cycles) in both relevant and calculation. I have to say the naming convention in the form is very complex, which makes debugging somewhat difficult (I assume the data analysis as well, depending on target platform), nonetheless, you might find below tips useful:

  • relevant: Many nodes has 1 as relevance condition, this might not be necessary, as by default a lack of relevant means the question will appear for collector.
  • calculation: many nodes had calculations despite being required and not read-only, if this had worked, user inputs would be reset upon form validation. Remove the calculation and make these read-only or populate desired default values if that was the intention

In this version of your form adult.xlsx I removed most of the problematic calculation and relevant conditions and the validations worked. Please give it a try and do a backward test/edit to match the originally intended controls.

As for the validation tools, unfortunately, I am not familiar with tricc. You can use ODK online validator https://getodk.org/xlsform/.

Best regards,

Jules R

1 Like

Hi,

Thanks you for taking the time to check.

Calculation on an input form works because they have β€˜.’ as first coalesce parameters, so one the data is enter it stays.

The form is generated by a script (TRICC) that should prohibit loops by design, I did not find a field that refers to itself, but often to a previous version (version number comes after _vV_ ) this is required when you capture the same concept in different places so at the end you have only one variable that has the consolidated results. I will check your edits to check which calculate you removed to ensure there is no tricc bug; I have run several python scripts to find loops but non manage to find real loops but those could have been flawed too.

In my research I found the javarosa bug

I did change the javarosa to add a mermaid output that show the cycle (LLM generated; poke @LN in case if interest other, it was super helpful in my case )

mermaid_cycle.txt (3.1 KB)

it showed the root cause: coalesce(.,XXXX) in calculation while it works well on enketo (I use that all the time in CHT; this idea even came from the CHT team)

is there the –skip-validate equivalent for https://getodk.org/xlsform/. ?

thanks in advance

1 Like

I did manage to show the form on a local enketo without errors; so neither from a logical point of view (any systems needs to track prev/current version of a fields) nor from an implementation point of view this should be blocking

1 Like

Thanks for sharing that Mermaid output patch, it's a nice idea! It nudged me to add mermaid diagram support to the docs which is something I'd been wanting to do for some time.

Currently, self-references are not allowed by Collect or Validate (https://github.com/getodk/javarosa/issues/458). We don't currently offer the option of skipping Validate because it exactly matches Collect's behavior. I understand it could be useful for folks using purely web-based forms, we'll consider it.

It looks from a quick look at your form like your goal with that usage of coalesce is to set a dynamic default for a field. The way we document for doing this is to put the default expression in the calculation column and to explicitly specify one or more references that should trigger recomputation in the trigger column: https://docs.getodk.org/form-logic/#dynamic-defaults-from-form-data Otherwise, the existing value is maintained. This should work across all clients.

2 Likes

I was also inspired by @delcroip and spent some time using an llm to create a script that would ingest a form and output a page with TOC style navigation for (nested) groups/repeats and styling for question types. Still needs some tweaking but it looks like it'll be a nice way to represent form content.

2 Likes

@Jiayu_Wang is doing that I believe

Yes, I’m working on visualising the ODK XLSForm as an interactive HTML. @ahblake We could exchange ideas if you’re interested as well. Honestly it is more complex than I initially thought, but it seems like a very useful approach for developers to debug forms and to visualise skip logic, nested groups, and other structures that are otherwise hidden within the XLSForm.

Absolutely!

I'd be happy to chat more about it, though i feel you're far more advanced than i am regarding this.

1 Like

I started exploring it a few months ago and have been working with @aurdipas and @Thalie. You can find an example here. We could have a quick meeting to exchange ideas if you’d like :smiling_face:

I recall that there was a (poster?) presentation at the 2023 ODK Summit in London that covered visualizing form flow. But now I cant seem to find any links to it... Perhaps @LN or @Aly_Blenkin recalls?

It was @Dalerhoda who shared a poster of their form flow. Here is the forum post.

We did an Insiders call on a similar topic as well.

2 Likes

Thanks @Aly_Blenkin for sharing the post and I’m pleased that the links still work.

Our handsome posters were constructed by hand in PowerPoint by @cclary, who uses them to lead conversations with the client about implications of thoughtful changes to the form.

After London we thought for a while about a program to read .xlsform and construct at least a very simple .pptx that has a box for each question laid out in top-to-bottom or left-to-right order with maybe some elementary formatting. One might start that document to design a bespoke visualization, which would surely involve moving those elements around and abbreviating and formatting to fit a specific purpose. To do more … to try to make a polished representation … we quickly set aside as impractical because with the same form we might make different viz choices depending on its purpose. So we considered a simple goal of basic export to a crude layout but alas even those thoughts had to be set aside in order to prioritize other work.

We will follow with interest to see what related efforts and conversations produce.

-Dale

3 Likes

Yes, I have thought about @Dalerhoda 's poster a lot, and had some discussion a while back with @Thalie about form visualisation which led to her bump of that thread.

I am keen to explore it further, and will contact you @Jiayu_Wang to bounce some ideas back and forth. Hopefully there is a way that approaches the utility of a bespoke diagram without the high time cost.

3 Likes