Issue #1075

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

··· -- Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

Tom,

We haven't seen this. My guess is that it's related to daylight savings
time and depends on the device locale config. Somebody will first need to
update the issue to be far more specific about the cases in which it does
or doesn't happen. I find it highly unlikely that it happens for all time
fields on all devices all the time.

Best,

Chris

··· On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

It happens on my Galaxy S5, freshly factory reset. And on a bunch of Carter
Center devices. But you're right it only happens when in DST. I changed
phone to Sydney timezone (currently in DST) and it happens, but not in EST.

I'll update the issue.

··· On 27 January 2015 at 11:24, Christopher Robert wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight savings
time and depends on the device locale config. Somebody will first need to
update the issue to be far more specific about the cases in which it does
or doesn't happen. I find it highly unlikely that it happens for all time
fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

Thanks.

As noted in the various date bugs, this is a very difficult problem to fix.

The easiest way to support legacy data and operations would be to introduce
a completely new datatype (dateTimeWithTimeZone) that preserves timezones
and actually works in date calculations, and to have that passed all the
way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix for
future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK Briefcase, we
could fix the existing date and dateTime datatypes, but there are so many
other flaws in the date arithmetic that is is probably safest to introduce
a brand new datatype and gradually allow the community to abandon the old
date and dateTime ones entirely.

··· On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert crobert@surveycto.com wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight savings
time and depends on the device locale config. Somebody will first need to
update the issue to be far more specific about the cases in which it does
or doesn't happen. I find it highly unlikely that it happens for all time
fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the bug
I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the phone
is in DST mode and correct for it, can't we?

··· On 29 January 2015 at 15:14, Mitch Sundt wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem to
fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix for
future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK Briefcase, we
could fix the existing date and dateTime datatypes, but there are so many
other flaws in the date arithmetic that is is probably safest to introduce
a brand new datatype and gradually allow the community to abandon the old
date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert crobert@surveycto.com wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight savings
time and depends on the device locale config. Somebody will first need to
update the issue to be far more specific about the cases in which it does
or doesn't happen. I find it highly unlikely that it happens for all time
fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

There might be a simple fix somewhere, but our assessment has also been
that this is a deeper problem, and we have tended to share the view that a
new data type is required. The time is shifting within Collect in this
case, but it can also shift in Aggregate and/or Briefcase. And, in general,
the shifting about of times between systems leads to difficulties in
higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

··· On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the bug
I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem to
fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix for
future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK Briefcase,
we could fix the existing date and dateTime datatypes, but there are so
many other flaws in the date arithmetic that is is probably safest to
introduce a brand new datatype and gradually allow the community to abandon
the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert crobert@surveycto.com wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight savings
time and depends on the device locale config. Somebody will first need to
update the issue to be far more specific about the cases in which it does
or doesn't happen. I find it highly unlikely that it happens for all time
fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Are you all saying that times are currently reported to the server without
timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500 or
-05 at the end of the string.

··· On 29 January 2015 at 22:17, Christopher Robert wrote:

There might be a simple fix somewhere, but our assessment has also been
that this is a deeper problem, and we have tended to share the view that a
new data type is required. The time is shifting within Collect in this
case, but it can also shift in Aggregate and/or Briefcase. And, in general,
the shifting about of times between systems leads to difficulties in
higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the bug
I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem to
fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix
for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK Briefcase,
we could fix the existing date and dateTime datatypes, but there are so
many other flaws in the date arithmetic that is is probably safest to
introduce a brand new datatype and gradually allow the community to abandon
the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert crobert@surveycto.com wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption. If
nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

I forget where each shift happens, but you may be right that the -0500 is
there from Collect, then the server shifts to server time before storing,
then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

··· On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth wrote:

Are you all saying that times are currently reported to the server without
timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500 or
-05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also been
that this is a deeper problem, and we have tended to share the view that a
new data type is required. The time is shifting within Collect in this
case, but it can also shift in Aggregate and/or Briefcase. And, in general,
the shifting about of times between systems leads to difficulties in
higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the bug
I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem to
fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix
for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK Briefcase,
we could fix the existing date and dateTime datatypes, but there are so
many other flaws in the date arithmetic that is is probably safest to
introduce a brand new datatype and gradually allow the community to abandon
the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert <crobert@surveycto.com wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption.
If nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on a
time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue. Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

··· On 29 January 2015 at 22:30, Christopher Robert wrote:

I forget where each shift happens, but you may be right that the -0500 is
there from Collect, then the server shifts to server time before storing,
then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500 or
-05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also been
that this is a deeper problem, and we have tended to share the view that a
new data type is required. The time is shifting within Collect in this
case, but it can also shift in Aggregate and/or Briefcase. And, in general,
the shifting about of times between systems leads to difficulties in
higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the
bug I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem to
fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix
for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption.
If nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

··· On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on a
time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the -0500 is
there from Collect, then the server shifts to server time before storing,
then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500 or
-05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also been
that this is a deeper problem, and we have tended to share the view that a
new data type is required. The time is shifting within Collect in this
case, but it can also shift in Aggregate and/or Briefcase. And, in general,
the shifting about of times between systems leads to difficulties in
higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the
bug I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem
to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a fix
for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch of
Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data corruption.
If nobody else has time I will try to fix myself but thought I'd check in
first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

I will be happy to take a look at this when I have time, and I will try to
prioritize.

Although I have to say, I'm not sure I agree with this stance. This is not
a feature request. This is what I would consider a major bug that can, and
has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

··· On 30 January 2015 at 11:36, Mitch Sundt wrote:

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on a
time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the -0500
is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500
or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the
bug I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when the
phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem
to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a
fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch
of Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

I agree.

Simply no time.

··· On Fri, Jan 30, 2015 at 8:54 AM, Tom Smyth wrote:

I will be happy to take a look at this when I have time, and I will try to
prioritize.

Although I have to say, I'm not sure I agree with this stance. This is not
a feature request. This is what I would consider a major bug that can, and
has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

On 30 January 2015 at 11:36, Mitch Sundt mitchellsundt@gmail.com wrote:

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on a
time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the -0500
is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500
or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert <crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the
bug I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when
the phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem
to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a
fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch
of Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Tom,

My sense is that an organization like the Carter Center is likely to have
considerable resources at its disposal. If it has a problem of this kind of
magnitude, it can and should dedicate non-zero resources to solving it.

This may take the form of purchasing a professionally-supported ODK variant
and (rightly) demanding that the paid provider solve the problem, or it may
take the form of contributing time or other resources to helping solve the
problem within the open-source community. Many organizations that use ODK
pay in some way for its extension or improvement, and there are a variety
of models for that. There is even overlap between the professional-variant
model and the community model, as when our customers press us to do things
and then we share them with the broader community.

Here, what you're simply seeing is that tangible resources aren't being
brought to bear on this particular issue. In some sense, market forces are
working if nobody is willing to pay to have it fixed and then it's not
being fixed. Apologies if that's in some way offensive. Some on this list
will recall that I am part economist, so I sometimes look at things from
that perspective. Resources are, sadly, finite.

Best,

Chris

P.S. There are also pretty easy work-arounds, right? For example, you can
easily avoid using the date/time widget altogether and put times in
calculate and/or text fields.

··· On Fri Jan 30 2015 at 11:54:39 AM Tom Smyth wrote:

I will be happy to take a look at this when I have time, and I will try to
prioritize.

Although I have to say, I'm not sure I agree with this stance. This is not
a feature request. This is what I would consider a major bug that can, and
has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

On 30 January 2015 at 11:36, Mitch Sundt mitchellsundt@gmail.com wrote:

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on a
time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the -0500
is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500
or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert <crobert@surveycto.com wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing the
bug I'm experiencing though? (Specifically: if phone's TZ is in DST, time
values are shifted by an hour when a saved form is re-opened. Works fine if
user just submits without closing and re-opening. Works fine if not in DST.)

Even if it is related to the date format stuff, we can check when
the phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult problem
to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a
fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch
of Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Robert, TCC does contribute to ODK development by paying Sassafras to
contribute. Looks like that's what will end up happening here.

··· On 30 January 2015 at 12:06, Christopher Robert wrote:

Tom,

My sense is that an organization like the Carter Center is likely to have
considerable resources at its disposal. If it has a problem of this kind of
magnitude, it can and should dedicate non-zero resources to solving it.

This may take the form of purchasing a professionally-supported ODK
variant and (rightly) demanding that the paid provider solve the problem,
or it may take the form of contributing time or other resources to helping
solve the problem within the open-source community. Many organizations that
use ODK pay in some way for its extension or improvement, and there are a
variety of models for that. There is even overlap between the
professional-variant model and the community model, as when our customers
press us to do things and then we share them with the broader community.

Here, what you're simply seeing is that tangible resources aren't being
brought to bear on this particular issue. In some sense, market forces are
working if nobody is willing to pay to have it fixed and then it's not
being fixed. Apologies if that's in some way offensive. Some on this list
will recall that I am part economist, so I sometimes look at things from
that perspective. Resources are, sadly, finite.

Best,

Chris

P.S. There are also pretty easy work-arounds, right? For example, you can
easily avoid using the date/time widget altogether and put times in
calculate and/or text fields.

On Fri Jan 30 2015 at 11:54:39 AM Tom Smyth tom@sassafras.coop wrote:

I will be happy to take a look at this when I have time, and I will try
to prioritize.

Although I have to say, I'm not sure I agree with this stance. This is
not a feature request. This is what I would consider a major bug that can,
and has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

On 30 January 2015 at 11:36, Mitch Sundt mitchellsundt@gmail.com wrote:

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on
a time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the -0500
is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g. -0500
or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert < crobert@surveycto.com> wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing
the bug I'm experiencing though? (Specifically: if phone's TZ is in DST,
time values are shifted by an hour when a saved form is re-opened. Works
fine if user just submits without closing and re-opening. Works fine if not
in DST.)

Even if it is related to the date format stuff, we can check when
the phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult
problem to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a
fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a bunch
of Carter Center devices. But you're right it only happens when in DST. I
changed phone to Sydney timezone (currently in DST) and it happens, but not
in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

Great! If one of our users presses us to solve it in the mean time, we'll
be sure to post updates to the issue.

Best,

Chris

··· On Fri, Jan 30, 2015, 12:30 PM Tom Smyth wrote:

Robert, TCC does contribute to ODK development by paying Sassafras to
contribute. Looks like that's what will end up happening here.

On 30 January 2015 at 12:06, Christopher Robert crobert@surveycto.com wrote:

Tom,

My sense is that an organization like the Carter Center is likely to have
considerable resources at its disposal. If it has a problem of this kind of
magnitude, it can and should dedicate non-zero resources to solving it.

This may take the form of purchasing a professionally-supported ODK
variant and (rightly) demanding that the paid provider solve the problem,
or it may take the form of contributing time or other resources to helping
solve the problem within the open-source community. Many organizations that
use ODK pay in some way for its extension or improvement, and there are a
variety of models for that. There is even overlap between the
professional-variant model and the community model, as when our customers
press us to do things and then we share them with the broader community.

Here, what you're simply seeing is that tangible resources aren't being
brought to bear on this particular issue. In some sense, market forces are
working if nobody is willing to pay to have it fixed and then it's not
being fixed. Apologies if that's in some way offensive. Some on this list
will recall that I am part economist, so I sometimes look at things from
that perspective. Resources are, sadly, finite.

Best,

Chris

P.S. There are also pretty easy work-arounds, right? For example, you can
easily avoid using the date/time widget altogether and put times in
calculate and/or text fields.

On Fri Jan 30 2015 at 11:54:39 AM Tom Smyth tom@sassafras.coop wrote:

I will be happy to take a look at this when I have time, and I will try
to prioritize.

Although I have to say, I'm not sure I agree with this stance. This is
not a feature request. This is what I would consider a major bug that can,
and has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

On 30 January 2015 at 11:36, Mitch Sundt mitchellsundt@gmail.com wrote:

The bottom line is that there is a bit of investigation and work to fix
this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46 on
a time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney time
(currently in DST, GMT+11), and enter 3:52, it's stored as 03:52:00.000+11.
So the stored values are correct. ODK is just messing them up somehow when
the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert <crobert@surveycto.com wrote:

I forget where each shift happens, but you may be right that the
-0500 is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g.
-0500 or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert < crobert@surveycto.com> wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing
the bug I'm experiencing though? (Specifically: if phone's TZ is in DST,
time values are shifted by an hour when a saved form is re-opened. Works
fine if user just submits without closing and re-opening. Works fine if not
in DST.)

Even if it is related to the date format stuff, we can check when
the phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult
problem to fix.

The easiest way to support legacy data and operations would be to
introduce a completely new datatype (dateTimeWithTimeZone) that preserves
timezones and actually works in date calculations, and to have that passed
all the way up to the server and through ODK Briefcase unaltered.

I.e., leave it broken and as-is for existing surveys, and offer a
fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a
bunch of Carter Center devices. But you're right it only happens when in
DST. I changed phone to Sydney timezone (currently in DST) and it happens,
but not in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to daylight
savings time and depends on the device locale config. Somebody will first
need to update the issue to be far more specific about the cases in which
it does or doesn't happen. I find it highly unlikely that it happens for
all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth tom@sassafras.coop wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sounds great; Sassafras / TCC contributed some good features a while ago.

Keep in mind that the core ODK team (2 developers + 3 grad students)
is research-funded
and is focused on advancing the state-of-the-art.

And, because we are research-funded, we need to publish and support
research, and that means we need to focus on the innovations in the 2.0
tools.

The 2.0 tools suite adds 4 new Android APKs (Survey, Tables, Scan and
Sensors), an entire Javascript-based forms rendering engine (Survey
javascript), 2 new primary forms development tools (XLSXConverter and Scan
form designer), and what are effectively 2 different brand-new servers.

This probably triples or quadruples the lines-of-code we are working on at
any given time.

There is no funding source that is earmarked for maintenance of, and
updates to, the existing 1.x software. We slip that in on the side.

Mitch

··· On Fri, Jan 30, 2015 at 9:43 AM, Christopher Robert wrote:

Great! If one of our users presses us to solve it in the mean time, we'll
be sure to post updates to the issue.

Best,

Chris

On Fri, Jan 30, 2015, 12:30 PM Tom Smyth tom@sassafras.coop wrote:

Robert, TCC does contribute to ODK development by paying Sassafras to
contribute. Looks like that's what will end up happening here.

On 30 January 2015 at 12:06, Christopher Robert crobert@surveycto.com wrote:

Tom,

My sense is that an organization like the Carter Center is likely to
have considerable resources at its disposal. If it has a problem of this
kind of magnitude, it can and should dedicate non-zero resources to solving
it.

This may take the form of purchasing a professionally-supported ODK
variant and (rightly) demanding that the paid provider solve the problem,
or it may take the form of contributing time or other resources to helping
solve the problem within the open-source community. Many organizations that
use ODK pay in some way for its extension or improvement, and there are a
variety of models for that. There is even overlap between the
professional-variant model and the community model, as when our customers
press us to do things and then we share them with the broader community.

Here, what you're simply seeing is that tangible resources aren't being
brought to bear on this particular issue. In some sense, market forces are
working if nobody is willing to pay to have it fixed and then it's not
being fixed. Apologies if that's in some way offensive. Some on this list
will recall that I am part economist, so I sometimes look at things from
that perspective. Resources are, sadly, finite.

Best,

Chris

P.S. There are also pretty easy work-arounds, right? For example, you
can easily avoid using the date/time widget altogether and put times in
calculate and/or text fields.

On Fri Jan 30 2015 at 11:54:39 AM Tom Smyth tom@sassafras.coop wrote:

I will be happy to take a look at this when I have time, and I will try
to prioritize.

Although I have to say, I'm not sure I agree with this stance. This is
not a feature request. This is what I would consider a major bug that can,
and has (for a Carter Center election observation mission), led to
inadvertently, silently corrupted data. The difference between a polling
station opening at 8:00am and 9:00am (an hour late) is non-trivial.

This is also not an edge case. This is any time widget in any timezone
currently in DST.

ODK is a data collection application. If it's corrupting the data it's
collecting, that's pretty severe in my opinion.

Of course I understand this is an open source product, and I deeply
appreciate everyone's efforts. But if the ODK team is so short staffed that
it can't even take a look at something like this, I guess that seems like
an issue that needs addressing by the team's leadership. Maybe this
conversation is already happening. I'm not sure.

I'd love to heart folks' thoughts on this.

Thanks.

On 30 January 2015 at 11:36, Mitch Sundt mitchellsundt@gmail.com wrote:

The bottom line is that there is a bit of investigation and work to
fix this, correcting an issue in one place is likely to present a problem
elsewhere, and there is insufficient developer or tester time to sort
through the issues and figure out the exact impacts.

On Fri, Jan 30, 2015 at 5:59 AM, Tom Smyth tom@sassafras.coop wrote:

Yeah, it's just as I thought. If I open ODK in EST and choose 10:46
on a time widget, it's stored as 10:46:00.000-05. If I open ODK in Sydney
time (currently in DST, GMT+11), and enter 3:52, it's stored as
03:52:00.000+11. So the stored values are correct. ODK is just messing them
up somehow when the form is reopened.

So I'm not sure if this is directly connected to the larger issue.
Thoughts?

BTW, I noticed that you need to restart ODK if you change the system
timezone. Not a problem at all, but something to be aware of.

On 29 January 2015 at 22:30, Christopher Robert < crobert@surveycto.com> wrote:

I forget where each shift happens, but you may be right that the
-0500 is there from Collect, then the server shifts to server time before
storing, then the exporting computer shifts to that computer's time before
exporting. I believe the immediate issue at-hand is related to the ISO
format not being up to the daylight-savings distinctions -- or, perhaps,
the way JR is rendering the ISO format. I'm far from an expert here.

Chris

On Thu Jan 29 2015 at 10:26:39 PM Tom Smyth tom@sassafras.coop wrote:

Are you all saying that times are currently reported to the server
without timezones? (Collect is my only concern as I don't use Aggregate or
Briefcase.)

I thought for sure it was sending times in ISO format with e.g.
-0500 or -05 at the end of the string.

On 29 January 2015 at 22:17, Christopher Robert < crobert@surveycto.com> wrote:

There might be a simple fix somewhere, but our assessment has also
been that this is a deeper problem, and we have tended to share the view
that a new data type is required. The time is shifting within Collect in
this case, but it can also shift in Aggregate and/or Briefcase. And, in
general, the shifting about of times between systems leads to difficulties
in higher-level ways; just recently, there were issues with shifts past
midnight not updating the associated dates. A new system in which times and
dates are passed through the entire system with their original timezones
intact would be a good one. But, it would be a fair bit of work.

Chris

On Thu Jan 29 2015 at 10:09:46 PM Tom Smyth tom@sassafras.coop wrote:

Holy crap, that's heavy.

Are we sure that the problem you're describing is what's causing
the bug I'm experiencing though? (Specifically: if phone's TZ is in DST,
time values are shifted by an hour when a saved form is re-opened. Works
fine if user just submits without closing and re-opening. Works fine if not
in DST.)

Even if it is related to the date format stuff, we can check when
the phone is in DST mode and correct for it, can't we?

On 29 January 2015 at 15:14, Mitch Sundt <mitchellsundt@gmail.com wrote:

Thanks.

As noted in the various date bugs, this is a very difficult
problem to fix.

The easiest way to support legacy data and operations would be
to introduce a completely new datatype (dateTimeWithTimeZone) that
preserves timezones and actually works in date calculations, and to have
that passed all the way up to the server and through ODK Briefcase
unaltered.

I.e., leave it broken and as-is for existing surveys, and offer
a fix for future surveys.

With considerable work to ODK Collect, ODK Aggregate and ODK
Briefcase, we could fix the existing date and dateTime datatypes, but
there are so many other flaws in the date arithmetic that is is probably
safest to introduce a brand new datatype and gradually allow the community
to abandon the old date and dateTime ones entirely.

On Tue, Jan 27, 2015 at 8:45 AM, Tom Smyth tom@sassafras.coop wrote:

It happens on my Galaxy S5, freshly factory reset. And on a
bunch of Carter Center devices. But you're right it only happens when in
DST. I changed phone to Sydney timezone (currently in DST) and it happens,
but not in EST.

I'll update the issue.

On 27 January 2015 at 11:24, Christopher Robert < crobert@surveycto.com> wrote:

Tom,

We haven't seen this. My guess is that it's related to
daylight savings time and depends on the device locale config. Somebody
will first need to update the issue to be far more specific about the cases
in which it does or doesn't happen. I find it highly unlikely that it
happens for all time fields on all devices all the time.

Best,

Chris

On Tue Jan 27 2015 at 11:05:37 AM Tom Smyth < tom@sassafras.coop> wrote:

Folks has anyone had a chance to look into
https://code.google.com/p/opendatakit/issues/detail?id=1075 ?

It seems fairly severe to me. Causing inadvertent data
corruption. If nobody else has time I will try to fix myself but thought
I'd check in first.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the
Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

Resident, Touchstone Cohousing
touchstonecohousing.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com