# Include non-relevant groups and fields in ODK Central API responses

**URL:** <https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536>\
**Category:** Development\
**Tags:** odk-central\
**Created:** [May 12, 2021, 11:51am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536 "2021-05-12T11:51:25Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![vlehn](https://getodk.b-cdn.net/letter_avatar_proxy/v4/letter/v/9dc877/32.png) [@vlehn](https://forum.getodk.org/u/vlehn)\
**Post date:** [May 12, 2021, 11:51am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/1 "2021-05-12T11:51:25Z")

</div>

Dear ODK community,

we recently encountered some unexpected behaviour for ODK Central's OData and SubmissionXML APIs. Depending on whether `relevant` was false for a group or field when filling in the form, they will be included or not included in inconsistent ways in the API responses. See below for the different cases.

 ![grafik](https://getodk.b-cdn.net/uploads/default/original/2X/c/cb8ff772ef5d7f6d6c57eb178156fba63d2a92cf.png)

The submission XML appearantly never includes groups or fields when they were not relevant. However, the OData submission response includes non-relevant fields with null-values but does not include non-relevant groups and their inner-fields.

What behaviour should be expected? From the OData API, we'd expect that all fields are always given, even if they were not relevant. This means that non-relevant groups would be included in responses, just like non-relevant normal fields already are.

Below you can find sample responses and an XlsForm to reproduce.

Thanks for you thoughts!

Best,  
Vitus

**Addition:** We noticed the issue because the behaviour was different on ODK Aggregate, if I remember correctly. In ODK Aggregate's JSON publishing mechanism, all fields were always included, independent of whether they were relevant or not.

OData responses with different relevant-options:

```json
{
	"value": [
		{
			"__id": "uuid:38c44ef9-1b4f-4305-8b06-9a525520b604",
			"show_group": "yes",
			"show_field": "yes",
			"number": null,
			"__system": {},
			"group": {
				"inner_number": null
			},
			"meta": {
				"instanceID": "uuid:38c44ef9-1b4f-4305-8b06-9a525520b604"
			}
		},
		{
			"__id": "uuid:f77aa888-8cfd-4695-b261-292333477106",
			"show_group": "yes",
			"show_field": "no",
			"number": null,
			"__system": {},
			"group": {
				"inner_number": null
			},
			"meta": {
				"instanceID": "uuid:f77aa888-8cfd-4695-b261-292333477106"
			}
		},
		{
			"__id": "uuid:30f81702-0967-4e6f-bf97-383f7e4221c8",
			"show_group": "no",
			"show_field": "no",
			"number": null,
			"__system": {},
			"meta": {
				"instanceID": "uuid:30f81702-0967-4e6f-bf97-383f7e4221c8"
			}
		}
	],
	"@odata.context": "https://example.com/v1/projects/4/forms/field_conditions_minimal_dv2.svc/$metadata#Submissions",
	"@odata.count": 3
}

```

Different Submission XMLs:

```xml
<!-- group is relevant, field is not-->
<data xmlns:jr="http://openrosa.org/javarosa" xmlns:orx="http://openrosa.org/xforms" id="field_conditions_minimal_dv2" version="1">
    <show_group>yes</show_group>
    <show_field>no</show_field>
    <group>
        <inner_number/>
    </group>
    <meta>
        <instanceID>uuid:f77aa888-8cfd-4695-b261-292333477106</instanceID>
    </meta>
</data>
<!-- group and field are not relevant-->
<data xmlns:jr="http://openrosa.org/javarosa" xmlns:orx="http://openrosa.org/xforms" id="field_conditions_minimal_dv2" version="1">
    <show_group>no</show_group>
    <show_field>no</show_field>
    <meta>
        <instanceID>uuid:30f81702-0967-4e6f-bf97-383f7e4221c8</instanceID>
    </meta>
</data>
<!-- group and field are relevant-->
<data xmlns:jr="http://openrosa.org/javarosa" xmlns:orx="http://openrosa.org/xforms" id="field_conditions_minimal_dv2" version="1">
    <show_group>yes</show_group>
    <show_field>yes</show_field>
    <group>
        <inner_number/>
    </group>
    <number/>
    <meta>
        <instanceID>uuid:38c44ef9-1b4f-4305-8b06-9a525520b604</instanceID>
    </meta>
</data>

```

XlsForm:  
[central\_nullvalues.xlsx](https://forum.getodk.org/uploads/short-url/mrfQbWvZUymuFRSXGcXZosCX2t1.xlsx) (12.4 KB)

---

<div class="post-metadata">

**Author:** ![Matthew\_White](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/matthew_white/32/1654_2.png) [@Matthew\_White](https://forum.getodk.org/u/Matthew_White)\
**Post date:** [May 12, 2021, 1:12pm UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/2 "2021-05-12T13:12:25Z")

</div>

Hi @vlehn, and welcome to the ODK forum! When you have a chance, I'd encourage you to [introduce yourself here](https://forum.getodk.org/t/-/6671). Doing so helps build community!

Thank you for this report! This sounds similar to the issue discussed [here](https://github.com/getodk/central-backend/issues/356). We'll be changing that behavior as part of v1.2, so the OData should include non-relevant fields. We're hoping to release v1.2 later this month.

---

<div class="post-metadata">

**Author:** ![Matthew\_White](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/matthew_white/32/1654_2.png) [@Matthew\_White](https://forum.getodk.org/u/Matthew_White)\
**Post date:** [May 12, 2021, 1:17pm UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/3 "2021-05-12T13:17:24Z")

</div>

And now I'm realizing you were probably part of that discussion on GitHub! Have you had a chance to see whether you encounter this same behavior for the latest commit to `central-backend`? As far as I know, ODK Central doesn't consider `relevant` when generating the OData, so I'm hoping it's the same issue.

---

<div class="post-metadata">

**Author:** ![vlehn](https://getodk.b-cdn.net/letter_avatar_proxy/v4/letter/v/9dc877/32.png) [@vlehn](https://forum.getodk.org/u/vlehn)\
**Post date:** [May 12, 2021, 2:29pm UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/4 "2021-05-12T14:29:32Z")

</div>

Hi Matthew, thanks for your quick reaction!

Indeed, I also reported the other issue. I tested the recent case with and without that commit, and the behaviour is the same. So the commit did not solve that one, I'm afraid.

Depending on what responses are expected, I can also file a bug report for it.

P.S: Yes, will introduce myself soon!

---

<div class="post-metadata">

**Author:** ![Matthew\_White](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/matthew_white/32/1654_2.png) [@Matthew\_White](https://forum.getodk.org/u/Matthew_White)\
**Post date:** [May 12, 2021, 2:37pm UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/5 "2021-05-12T14:37:30Z")

</div>

@issa, just wanted to make sure you saw this topic. What do you think is the expected behavior in this case?

---

<div class="post-metadata">

**Author:** ![vlehn](https://getodk.b-cdn.net/letter_avatar_proxy/v4/letter/v/9dc877/32.png) [@vlehn](https://forum.getodk.org/u/vlehn)\
**Post date:** [May 18, 2021, 11:07am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/6 "2021-05-18T11:07:47Z")

</div>

Hi @Matthew_White and @issa

So it seems like developing a position regarding this issue is not as simple. 😉 Still, is the topic something that you'd consider part of the current development roadmap (independent of what the outcome might be)? Could it even find its way into v1.2?

As stated, we would expect to always receive the full field set via OData JSON API, as then there would be fewer "surprises" in those individual submissions.

Would really appreciate if you can give me some perspective.

Thank you in any case!  
Vitus

---

<div class="post-metadata">

**Author:** ![mathieubossaert](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/mathieubossaert/32/6131_2.png) [@mathieubossaert](https://forum.getodk.org/u/mathieubossaert)\
**Post date:** [May 18, 2021, 11:50am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/7 "2021-05-18T11:50:58Z")

</div>

Hi to all,  
using my [central2pg](https://forum.getodk.org/t/postgresql-set-of-functions-to-get-data-from-central/33350) set of PostgreSQL function with new forms, I am facing the same problem.  
I am using the api result to automatically generate database tables and I realized that questions are missing. It would be great to get all fields, including nulls in submission data.

As a temporary workaround, is it possible to query the API to get all fields for each table instead of the whole form ?  
The given json doe not help to know wich table is concerned by each attribute?

> **[ODK Central API · Apiary](https://odkcentral.docs.apiary.io/#reference/forms-and-submissions/'-individual-form/getting-form-schema-fields?console=1)**
>
> A place where APIs are kept.

Here is the form (sorry it is not the simplest one :  
[suivis\_mc\_costieres\_V2.xls](https://forum.getodk.org/uploads/short-url/5H5tJpQBh0DDkhAAUrJNlNGFTAY.xls) (27.5 KB)) and here a submission for the "releves\_indicateurs" table

```json
{"__id":"9f397e9780e26dedf9e63c7313bd7f66ee4b9375",
"__Submissions-id":"uuid:2b633df3-63c5-4f61-8625-7ea0e78585d9",
"infos_generales_cen_occitanie":{"suivi_controle":"controle_simple",
"espece":null,
"saison":null,
"maitrise_fonciere":"convention_mae"},
"infos_releve_ug":{"now_releve_ug":"2021-05-06T09:14:29.142+02:00",
"date_time_releve_ug":"2021-05-05T09:05:00.000+02:00",
"id_ug":2305},
"conformite":{"controle_interne_conformite":"conforme",
"controle_interne_description":null},
"previsionnel":{"previsionnel_a_faire":null,
"previsionnel_autre":null},
"observations":{"indices_presence":"non",
"presence_individus":"non",
"remarques_mae":null,
"remarques_ppte":null}
}

```

Thanks a lot 🙂

---

<div class="post-metadata">

**Author:** ![issa](https://getodk.b-cdn.net/letter_avatar_proxy/v4/letter/i/87869e/32.png) [@issa](https://forum.getodk.org/u/issa)\
**Post date:** [August 18, 2021, 1:41am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/8 "2021-08-18T01:41:51Z")

</div>

okay, once i wrapped my head back around all this stuff i saw a way to fix it. should be coming in the next release.

---

<div class="post-metadata">

**Author:** ![Matthew\_White](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/matthew_white/32/1654_2.png) [@Matthew\_White](https://forum.getodk.org/u/Matthew_White)\
**Post date:** [October 1, 2021, 12:47am UTC](https://forum.getodk.org/t/include-non-relevant-groups-and-fields-in-odk-central-api-responses/33536/9 "2021-10-01T00:47:59Z")

</div>

As of [v1.3](https://forum.getodk.org/t/odk-central-v1-3/35255), we now include all groups in OData, even if they are not relevant. Thank you for reporting this issue!