What are the options when XML files (instances) are corrupted? Root cause?

For some reason, a small minority of XML instances are corrupted. (see
attached one of them). I tried recovering using ODK Briefcase but it
throws a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML)

Anyone knows the possible cause of this corruption. Could it be from a low
battery shutdown, file permissions, file system or android version?

I see my device is currently using ODK Collect 1.4.4 and the OS is Android
4.2.2.

Regards,
piqo

corrupted.xml (3.67 KB)

Piqo,

All those are good guesses. This is the first time I've heard of an
issue like this, so you'll have to do some investigation.

Is it one device or all devices? Is it one form or all forms? Did this
start happening or has it always happened? If this is a new issue,
what changed recently? Do you have very cheap SD cards? Does it happen
with one enumerator or all of them? Do you have any special software
(app lock) that might be causing this issue? Do any other apps have
problems? Does ODK crash/force close before a file is corrupted? Etc,
etc, etc.

Once you've done a bit of investigation to find a common variable or
reproducible steps, report back with what you've found.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Mon, Sep 22, 2014 at 4:25 AM, E Piqo piqoni@gmail.com wrote:

For some reason, a small minority of XML instances are corrupted. (see
attached one of them). I tried recovering using ODK Briefcase but it throws
a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML)

Anyone knows the possible cause of this corruption. Could it be from a low
battery shutdown, file permissions, file system or android version?

I see my device is currently using ODK Collect 1.4.4 and the OS is Android
4.2.2.

Regards,
piqo

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

And do you have device encryption enabled, or might you have enabled
encryption while ODK Collect was open?

··· On Mon, Sep 22, 2014 at 8:29 AM, Yaw Anokwa wrote:

Piqo,

All those are good guesses. This is the first time I've heard of an
issue like this, so you'll have to do some investigation.

Is it one device or all devices? Is it one form or all forms? Did this
start happening or has it always happened? If this is a new issue,
what changed recently? Do you have very cheap SD cards? Does it happen
with one enumerator or all of them? Do you have any special software
(app lock) that might be causing this issue? Do any other apps have
problems? Does ODK crash/force close before a file is corrupted? Etc,
etc, etc.

Once you've done a bit of investigation to find a common variable or
reproducible steps, report back with what you've found.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Mon, Sep 22, 2014 at 4:25 AM, E Piqo piqoni@gmail.com wrote:

For some reason, a small minority of XML instances are corrupted. (see
attached one of them). I tried recovering using ODK Briefcase but it
throws
a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML)

Anyone knows the possible cause of this corruption. Could it be from a
low
battery shutdown, file permissions, file system or android version?

I see my device is currently using ODK Collect 1.4.4 and the OS is
Android
4.2.2.

Regards,
piqo

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Yaw and Mitch, thanks for your replies.

The issue happens on all devices, and only one kind of form so far has this
issue (but we haven't done other forms with these devices yet so I guess I
have to wait to find out more), One thing that bothers me is that the
instance name on ODK Collect for this form is just a dash (-), but probably
unrelated. Though the directory of instances has full names of the project
concatenated with the date the form was filled.

We do use AppLock, but odk collect is not locked, and we have used Applock
before (with other devices and surveys) and didn't have issues like this.

The tablet uses its internal memory, no SD card, but yeah I think they are
cheap ones.

Mitch, no device encryption, as far as I know.

If I find something, I'll report back.

Thanks for your ideas.
piqo

··· On Monday, September 22, 2014 5:30:36 PM UTC+2, Yaw Anokwa wrote: > > Piqo, > > All those are good guesses. This is the first time I've heard of an > issue like this, so you'll have to do some investigation. > > Is it one device or all devices? Is it one form or all forms? Did this > start happening or has it always happened? If this is a new issue, > what changed recently? Do you have very cheap SD cards? Does it happen > with one enumerator or all of them? Do you have any special software > (app lock) that might be causing this issue? Do any other apps have > problems? Does ODK crash/force close before a file is corrupted? Etc, > etc, etc. > > Once you've done a bit of investigation to find a common variable or > reproducible steps, report back with what you've found. > > Yaw > -- > Need ODK services? http://nafundi.com provides form design, server > setup, professional support, and software development for ODK. > > On Mon, Sep 22, 2014 at 4:25 AM, E Piqo <piq...@gmail.com > wrote: > > For some reason, a small minority of XML instances are corrupted. (see > > attached one of them). I tried recovering using ODK Briefcase but it > throws > > a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML) > > > > Anyone knows the possible cause of this corruption. Could it be from a > low > > battery shutdown, file permissions, file system or android version? > > > > I see my device is currently using ODK Collect 1.4.4 and the OS is > Android > > 4.2.2. > > > > Regards, > > piqo > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google > Groups > > "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to opendatakit...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. >

Piqo,

And if you can, share the form you are using (XLS and XForm) and I'm
glad to take a quick look to make sure there is nothing obviously
wrong with it.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Tue, Sep 23, 2014 at 12:50 AM, El Piqo piqoni@gmail.com wrote:

Yaw and Mitch, thanks for your replies.

The issue happens on all devices, and only one kind of form so far has this
issue (but we haven't done other forms with these devices yet so I guess I
have to wait to find out more), One thing that bothers me is that the
instance name on ODK Collect for this form is just a dash (-), but probably
unrelated. Though the directory of instances has full names of the project
concatenated with the date the form was filled.

We do use AppLock, but odk collect is not locked, and we have used Applock
before (with other devices and surveys) and didn't have issues like this.

The tablet uses its internal memory, no SD card, but yeah I think they are
cheap ones.

Mitch, no device encryption, as far as I know.

If I find something, I'll report back.

Thanks for your ideas.
piqo

On Monday, September 22, 2014 5:30:36 PM UTC+2, Yaw Anokwa wrote:

Piqo,

All those are good guesses. This is the first time I've heard of an
issue like this, so you'll have to do some investigation.

Is it one device or all devices? Is it one form or all forms? Did this
start happening or has it always happened? If this is a new issue,
what changed recently? Do you have very cheap SD cards? Does it happen
with one enumerator or all of them? Do you have any special software
(app lock) that might be causing this issue? Do any other apps have
problems? Does ODK crash/force close before a file is corrupted? Etc,
etc, etc.

Once you've done a bit of investigation to find a common variable or
reproducible steps, report back with what you've found.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Mon, Sep 22, 2014 at 4:25 AM, E Piqo piq...@gmail.com wrote:

For some reason, a small minority of XML instances are corrupted. (see
attached one of them). I tried recovering using ODK Briefcase but it
throws
a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML)

Anyone knows the possible cause of this corruption. Could it be from a
low
battery shutdown, file permissions, file system or android version?

I see my device is currently using ODK Collect 1.4.4 and the OS is
Android
4.2.2.

Regards,
piqo

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thnx for your availability. I just found the XLS form, and noticed there
was a mistake in the settings tab, instance_name was defined as:
concat(SERIALID,'-', intname)

As you can see, these two ill-defined variables have been used without the
necessary '${}', that's why the form names on ODK collect consisted of a
singe dash.
Not sure if this is related to the issue, or how can this wrong
instance_name affect the filesystem
.

Though, from 1000 surveys, there were corrupted about 15 of them, so I tend
to believe it's not related with the above issue (which seems more
deterministic, and not random).

If I find something worth sharing, I'll keep you posted.

Thanks again,
take care,
Piqo

··· On Wednesday, September 24, 2014 5:15:34 PM UTC+2, Yaw Anokwa wrote: > > Piqo, > > And if you can, share the form you are using (XLS and XForm) and I'm > glad to take a quick look to make sure there is nothing obviously > wrong with it. > > Yaw > -- > Need ODK services? http://nafundi.com provides form design, server > setup, professional support, and software development for ODK. > > On Tue, Sep 23, 2014 at 12:50 AM, El Piqo <piq...@gmail.com > wrote: > > Yaw and Mitch, thanks for your replies. > > > > The issue happens on all devices, and only one kind of form so far has > this > > issue (but we haven't done other forms with these devices yet so I guess > I > > have to wait to find out more), One thing that bothers me is that the > > instance name on ODK Collect for this form is just a dash (-), but > probably > > unrelated. Though the directory of instances has full names of the > project > > concatenated with the date the form was filled. > > > > We do use AppLock, but odk collect is not locked, and we have used > Applock > > before (with other devices and surveys) and didn't have issues like > this. > > > > The tablet uses its internal memory, no SD card, but yeah I think they > are > > cheap ones. > > > > Mitch, no device encryption, as far as I know. > > > > If I find something, I'll report back. > > > > Thanks for your ideas. > > piqo > > > > On Monday, September 22, 2014 5:30:36 PM UTC+2, Yaw Anokwa wrote: > >> > >> Piqo, > >> > >> All those are good guesses. This is the first time I've heard of an > >> issue like this, so you'll have to do some investigation. > >> > >> Is it one device or all devices? Is it one form or all forms? Did this > >> start happening or has it always happened? If this is a new issue, > >> what changed recently? Do you have very cheap SD cards? Does it happen > >> with one enumerator or all of them? Do you have any special software > >> (app lock) that might be causing this issue? Do any other apps have > >> problems? Does ODK crash/force close before a file is corrupted? Etc, > >> etc, etc. > >> > >> Once you've done a bit of investigation to find a common variable or > >> reproducible steps, report back with what you've found. > >> > >> Yaw > >> -- > >> Need ODK services? http://nafundi.com provides form design, server > >> setup, professional support, and software development for ODK. > >> > >> On Mon, Sep 22, 2014 at 4:25 AM, E Piqo wrote: > >> > For some reason, a small minority of XML instances are corrupted. > (see > >> > attached one of them). I tried recovering using ODK Briefcase but it > >> > throws > >> > a XML error (org.opendatakit.briefcase.util.BadXMLFixer fixXML) > >> > > >> > Anyone knows the possible cause of this corruption. Could it be from > a > >> > low > >> > battery shutdown, file permissions, file system or android version? > >> > > >> > I see my device is currently using ODK Collect 1.4.4 and the OS is > >> > Android > >> > 4.2.2. > >> > > >> > Regards, > >> > piqo > >> > > >> > -- > >> > -- > >> > Post: opend...@googlegroups.com > >> > Unsubscribe: opendatakit...@googlegroups.com > >> > Options: http://groups.google.com/group/opendatakit?hl=en > >> > > >> > --- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "ODK Community" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send > >> > an > >> > email to opendatakit...@googlegroups.com. > >> > For more options, visit https://groups.google.com/d/optout. > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google > Groups > > "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to opendatakit...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. >