Data path to ODK/Instances

We have implemented a reliable way to move contents of ODK/Instances filled forms data to the ext-SD irregardless of which version of Android being used. Now our challenge is to create/change the data path variable to have ODK continue to maintain edited forms in that new ext-SD location.

Question: is the ODK Collect code structured such that the path ODK/Instances is a global variable? If not... How many places will we have to change in the code to implement a data path variable to Edit forms?

Has any one written a detailed plan for such surgery? Not being an insider to the code ... Having such a plan will be helpful so we don't miss something.

Thanks!
Bob

In ODK Collect, File path construction is scattered all over the code.

To do this, the first step is to create a static class and move all the
file path construction into that.

e.g., in the 2.0 tools, we created an "ODKFileUtils" class (
https://github.com/opendatakit/androidlibrary/blob/development/androidlibrary_lib/src/main/java/org/opendatakit/common/android/utilities/ODKFileUtils.java
) that has static methods like "getODKFolder()"

I would create a class like this for ODK Collect. It will be MUCH smaller
than the above class, as it will do very little -- most likely just having
a 5-6 methods.

You will also want to isolate all of the following calls inside static
methods of that class:
Environment.getExternalStorageState()
Environment.getExternalStorageDirectory()

Once all the file path construction is isolated into this one class, you
can then change the way the paths are constructed in a consistent,
centralized, way.

Note that the Application object may not yet exist when you are inside the
constructor of the Content Provider(s). If you need any information from
that object, you may need to defer obtaining it until the start of a
query(), delete(), insert(), etc. operation on the content provider.

If the location of the database can become dynamic, it is likely that the
content providers should change to always open the database at the start of
the query(), etc. methods, as otherwise the app will likely need to be
force-stopped in order for the content providers to pick up on the change.

··· On Tue, Nov 10, 2015 at 12:31 PM, wrote:

We have implemented a reliable way to move contents of ODK/Instances
filled forms data to the ext-SD irregardless of which version of Android
being used. Now our challenge is to create/change the data path variable
to have ODK continue to maintain edited forms in that new ext-SD location.

Question: is the ODK Collect code structured such that the path
ODK/Instances is a global variable? If not... How many places will we have
to change in the code to implement a data path variable to Edit forms?

Has any one written a detailed plan for such surgery? Not being an
insider to the code ... Having such a plan will be helpful so we don't miss
something.

Thanks!
Bob

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

We have hooked up Dropbox to sync the content of ODK/Instances directory to the user's/enjmerator's personal Dropbox account. Then the user/enumerator just shares that with the "agregate" Dropbox account where Briefcase style aggregation will be done.

Does that simplify the parts of code that we would need to deal with in that we are not worried with communication between forms data in the /Instances directory to either Agregate Server or Briefcase back end processing?

Which code modules are remaining to deal with path integration if data only goes into /Instances but never on to Agregate or Briefcase? For our purposes it only gets synced by the DropBox api.

We are pretty excited about the added flexibility our enumerators will have:

  1. Send large multimedia filled forms data in by...
    a) removable SD
    And/or
    b) Dropbox sync and share
  2. Insitu backup to Dropbox of field data in process of enumeration
  3. Atomic file level restart of data sync when wifi is spotty
  4. Efficient file handling of large multimedia.

Thanks for your advice in helping us to make this last stage of integration.

Bob

I would not want the code change to only work with the specific code paths
that your group is using.

It will be too difficult for new users to understand.

When they deviate from your usage and the app Force Closes, or appears to
have lost their submission, they will simply conclude that the app is buggy
and/or flaky, and will abandon it.

By pulling all of the file path construction into this one class, we can
systematically apply your change across every use case, and make everything
behave consistently.

It is not wasted effort.

And it will prevent many bugs from being introduced if you made edits
piecemeal throughout the code.

··· On Tue, Nov 10, 2015 at 3:19 PM, wrote:

We have hooked up Dropbox to sync the content of ODK/Instances directory
to the user's/enjmerator's personal Dropbox account. Then the
user/enumerator just shares that with the "agregate" Dropbox account where
Briefcase style aggregation will be done.

Does that simplify the parts of code that we would need to deal with in
that we are not worried with communication between forms data in the
/Instances directory to either Agregate Server or Briefcase back end
processing?

Which code modules are remaining to deal with path integration if data
only goes into /Instances but never on to Agregate or Briefcase? For our
purposes it only gets synced by the DropBox api.

We are pretty excited about the added flexibility our enumerators will
have:

  1. Send large multimedia filled forms data in by...
    a) removable SD
    And/or
    b) Dropbox sync and share
  2. Insitu backup to Dropbox of field data in process of enumeration
  3. Atomic file level restart of data sync when wifi is spotty
  4. Efficient file handling of large multimedia.

Thanks for your advice in helping us to make this last stage of
integration.

Bob

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

It probably is better to just remove the Send button if a survey project decides to set up their collectors to go the path of "Large Multimedia Data (LMD)" surveys.

Removing the Send button will ensure no forced closes because database code will be dormant.

So i pose the question...What one or two places in the code deal with saving a form and media data?

How does that sound?

Bob

That just creates spaghetti code.

Create the one class with static methods, consolidate everything in there.

This is not "A Big Deal"

Arguing about it is wasting more time than actually doing it.

If you are absolutely convinced that it should not be done this way,
give me a different justification. Code maintenance is always
work. Adding one-off hacks is easy ...and they make a pasta dish.

··· On Fri, Nov 13, 2015 at 6:23 AM, wrote:

It probably is better to just remove the Send button if a survey project
decides to set up their collectors to go the path of "Large Multimedia Data
(LMD)" surveys.

Removing the Send button will ensure no forced closes because database
code will be dormant.

So i pose the question...What one or two places in the code deal with
saving a form and media data?

How does that sound?

Bob

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

I'm just trying to look for a solution to my immediate need... Data path for form edit I/O. While serving and staying in tune with consensus input from the core ODK team.

As I understand there is no interest in ext-SD support for multiple reasons. And no interest in form data sync to the cloud for the same set of reasons. And no interest in supporting large media 2GB+ ... Expressed by the core team. Have I misread inputs?

I understood that code has already been written once by the core team to try to support ext-SD and consolidate I/O into a single variable structure as you suggest but that code while mostly done was abandoned because ???

Sooo, I'm not trying to argue by looking for advice on how to accomplish this last piece in implementing these enhancements for our project. I'm just under the impression that the core ODK team does not want them.

And we are some what limited in our skill and understanding to code and test such a sweeping change in ODK Collect that affects all I/O. Case in point that we never were smart enough to get the latest code to compile. So we are working with back level version of ODK. We could try to get smarter but is the ODK code documented well enough that more than only folks that have written it could identify all the places that I/O needs to happen. I'm under the impression that some adds last year are not documented???

So I am just trying to listen to what I'm hearing and not go off and attempt to code something that is not wanted by the core team.

That said... I was in Kenya working with Deaf volunteers as they videoed their hand signs to make a video dictionary for their people. Not being able to write all the data to the Ext-SD was causing a great problem for their work. So I am motivated to finish this task for the next deaf volunteers that will follow in their steps!

Thanks!
Bob

To be clear: if the implementation is generic, and based on the code tip,
we are happy to add it to the main code tree.

i.e., we'd be happy to accept code changes that support configuring to use
an Ex-SD path in a generic way and that are not tied to specific device
manufacturers or device models
(and it sounds like you've figured out how
to do that).

My understanding is that the Dropbox tie-in uses a DropBox APK to sync a
specified directory to DropBox -- that it does not require any
modifications to ODK Collect to support that and that it could work against
/sdcard/odk right now (without the Ex-SD changes). I believe the current
ODK Collect can already support that mechanism via the Admin Settings where
you can hide the "Send Finalized Form" button and "Get Blank Form" buttons
from the user (so they don't try to do any 1.x downloading or uploading of
forms and submissions).

Or was there something more that was needed to support Dropbox? We would
consider adding mechanisms to enable this.

Nafundi, for example, has just added a contribution to pull forms from
Google Drive and push submissions to Google Sheets. I.e., replacing ODK
Aggregate with this alternative mechanism and eliminating ODK Aggregate
entirely.

We are not opposed to expanding off-box transmission mechanisms. We just
want functionality to not be device or manufacturer-specific.

Mitch

··· On Fri, Nov 13, 2015 at 2:15 PM, wrote:

I'm just trying to look for a solution to my immediate need... Data path
for form edit I/O. While serving and staying in tune with consensus input
from the core ODK team.

As I understand there is no interest in ext-SD support for multiple
reasons. And no interest in form data sync to the cloud for the same set
of reasons. And no interest in supporting large media 2GB+ ... Expressed
by the core team. Have I misread inputs?

I understood that code has already been written once by the core team to
try to support ext-SD and consolidate I/O into a single variable structure
as you suggest but that code while mostly done was abandoned because ???

Sooo, I'm not trying to argue by looking for advice on how to accomplish
this last piece in implementing these enhancements for our project. I'm
just under the impression that the core ODK team does not want them.

And we are some what limited in our skill and understanding to code and
test such a sweeping change in ODK Collect that affects all I/O. Case in
point that we never were smart enough to get the latest code to compile.
So we are working with back level version of ODK. We could try to get
smarter but is the ODK code documented well enough that more than only
folks that have written it could identify all the places that I/O needs to
happen. I'm under the impression that some adds last year are not
documented???

So I am just trying to listen to what I'm hearing and not go off and
attempt to code something that is not wanted by the core team.

That said... I was in Kenya working with Deaf volunteers as they videoed
their hand signs to make a video dictionary for their people. Not being
able to write all the data to the Ext-SD was causing a great problem for
their work. So I am motivated to finish this task for the next deaf
volunteers that will follow in their steps!

Thanks!
Bob
Www.HisHandsReader.org

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com