Have people tried to use ODK with a very large cascading select one option
set? Say perhaps 120,000 options, several levels deep. So maybe 20MB worth
of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into
memory? I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has
anyone tried something like this?
Hi Tom, I tried something much smaller (about 3000 at the lowest level) and
I think my observations were the following:
-
The operating memory of the phone is the limiting factor. There were
other skip patterns in my survey that made the form far more sluggish than
the itemset. -
I could be wrong, but it looks like you have to program cascading
selects in ODK by hand and therefore I use KoBo Toolbox to import all the
options as a spreadsheet (there it is called an itemset) to do that in a
more automated way. However, when you do that the values that are stored in
the instance are generated automatically. So, it becomes a little bit of a
pain to ensure that you can still interpret your data after you are done
with it. However, if you're set on this idea, you could easily test the
import into ODK using the import itemset option. It would take you about an
hour to test.
Cheers,
j
···
On Tue, Jul 8, 2014 at 2:09 PM, Tom Smyth wrote:Have people tried to use ODK with a very large cascading select one option
set? Say perhaps 120,000 options, several levels deep. So maybe 20MB worth
of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into
memory? I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has
anyone tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re-asking this question.
I just tried to load about a 2MB form into ODK Collect. It contains only
one question with a four-level cascading select. The total number of nodes
in the tree is 23,721. You can see it here:
https://www.dropbox.com/s/94dynosysw54tpg/170.xml?dl=0.
ODK Collect downloads the form fine but when I open it, it says 'Please
wait a few moments, Reading form definition...' and stays on this screen
for about 5 minutes. Then it goes away and the app appears to be frozen.
Later I get a 'not responding' message from the OS. This is all on a
Samsung Galaxy S5, so very capable phone.
So it seems like ODK is not able to handle the load. Given that the XML
file is only 2MB, I'm guessing it's the XML parsing that is causing all the
fuss. I glanced at the Android monitor tool to see if it's running out of
memory, but that doesn't appear to be the issue.
Any thoughts on this? I feel like this is a pretty common use case: having
a large set of place names that you want folks to choose from. Would be
great if we could get it working.
Thanks.
···
On 8 July 2014 14:09, Tom Smyth wrote:Have people tried to use ODK with a very large cascading select one option
set? Say perhaps 120,000 options, several levels deep. So maybe 20MB worth
of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into
memory? I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has
anyone tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech
--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech
You can get more performance for large selects if you switch to
external itemsets.
http://opendatakit.org/help/form-design/external-itemsets/
How large depends on the device and how much RAM you have. Only one
way to find out...
Yaw
···
-- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.On Tue, Jul 8, 2014 at 11:24 AM, Jitka Hiscox jitka.hiscox@gmail.com wrote:
Hi Tom, I tried something much smaller (about 3000 at the lowest level) and
I think my observations were the following:
The operating memory of the phone is the limiting factor. There were
other skip patterns in my survey that made the form far more sluggish than
the itemset.I could be wrong, but it looks like you have to program cascading selects
in ODK by hand and therefore I use KoBo Toolbox to import all the options as
a spreadsheet (there it is called an itemset) to do that in a more automated
way. However, when you do that the values that are stored in the instance
are generated automatically. So, it becomes a little bit of a pain to ensure
that you can still interpret your data after you are done with it. However,
if you're set on this idea, you could easily test the import into ODK using
the import itemset option. It would take you about an hour to test.Cheers,
jOn Tue, Jul 8, 2014 at 2:09 PM, Tom Smyth tom@sassafras.coop wrote:
Have people tried to use ODK with a very large cascading select one option
set? Say perhaps 120,000 options, several levels deep. So maybe 20MB worth
of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into memory?
I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has anyone
tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi Tom,
For large itemsets, use external itemsets. They are designed for this
exact use case and they are much faster.
http://opendatakit.org/help/form-design/external-itemsets and
http://xlsform.org/#external have instructions.
Also, please don't post to both lists.
Yaw
···
-- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.On Mon, Oct 13, 2014 at 7:51 AM, Tom Smyth tom@sassafras.coop wrote:
Re-asking this question.
I just tried to load about a 2MB form into ODK Collect. It contains only one
question with a four-level cascading select. The total number of nodes in
the tree is 23,721. You can see it here:
https://www.dropbox.com/s/94dynosysw54tpg/170.xml?dl=0.ODK Collect downloads the form fine but when I open it, it says 'Please wait
a few moments, Reading form definition...' and stays on this screen for
about 5 minutes. Then it goes away and the app appears to be frozen. Later I
get a 'not responding' message from the OS. This is all on a Samsung Galaxy
S5, so very capable phone.So it seems like ODK is not able to handle the load. Given that the XML file
is only 2MB, I'm guessing it's the XML parsing that is causing all the fuss.
I glanced at the Android monitor tool to see if it's running out of memory,
but that doesn't appear to be the issue.Any thoughts on this? I feel like this is a pretty common use case: having a
large set of place names that you want folks to choose from. Would be great
if we could get it working.Thanks.
On 8 July 2014 14:09, Tom Smyth tom@sassafras.coop wrote:
Have people tried to use ODK with a very large cascading select one option
set? Say perhaps 120,000 options, several levels deep. So maybe 20MB worth
of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into memory?
I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has anyone
tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.For more options, visit https://groups.google.com/d/optout.
Tom,
You'll want to use .csv files, either via external itemsets or pre-loading.
The XML-based approach to cascading selects breaks down long before 23,000
(we advise against more than 1,000 total choices on the choices tab). It
will consume all available memory, and it will perform outrageously badly.
Probably both are happening in your case.
Best,
Chris
···
--- http://www.surveycto.com/On Mon, Oct 13, 2014 at 10:51 AM, Tom Smyth tom@sassafras.coop wrote:
Re-asking this question.
I just tried to load about a 2MB form into ODK Collect. It contains only
one question with a four-level cascading select. The total number of nodes
in the tree is 23,721. You can see it here:
https://www.dropbox.com/s/94dynosysw54tpg/170.xml?dl=0.ODK Collect downloads the form fine but when I open it, it says 'Please
wait a few moments, Reading form definition...' and stays on this screen
for about 5 minutes. Then it goes away and the app appears to be frozen.
Later I get a 'not responding' message from the OS. This is all on a
Samsung Galaxy S5, so very capable phone.So it seems like ODK is not able to handle the load. Given that the XML
file is only 2MB, I'm guessing it's the XML parsing that is causing all the
fuss. I glanced at the Android monitor tool to see if it's running out of
memory, but that doesn't appear to be the issue.Any thoughts on this? I feel like this is a pretty common use case: having
a large set of place names that you want folks to choose from. Would be
great if we could get it working.Thanks.
On 8 July 2014 14:09, Tom Smyth tom@sassafras.coop wrote:
Have people tried to use ODK with a very large cascading select one
option set? Say perhaps 120,000 options, several levels deep. So maybe 20MB
worth of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into
memory? I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has
anyone tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Folks, nevermind, I just discovered external itemsets. Going to build that
capability into ELMO.
···
On 13 October 2014 10:51, Tom Smyth wrote:Re-asking this question.
I just tried to load about a 2MB form into ODK Collect. It contains only
one question with a four-level cascading select. The total number of nodes
in the tree is 23,721. You can see it here:
https://www.dropbox.com/s/94dynosysw54tpg/170.xml?dl=0.ODK Collect downloads the form fine but when I open it, it says 'Please
wait a few moments, Reading form definition...' and stays on this screen
for about 5 minutes. Then it goes away and the app appears to be frozen.
Later I get a 'not responding' message from the OS. This is all on a
Samsung Galaxy S5, so very capable phone.So it seems like ODK is not able to handle the load. Given that the XML
file is only 2MB, I'm guessing it's the XML parsing that is causing all the
fuss. I glanced at the Android monitor tool to see if it's running out of
memory, but that doesn't appear to be the issue.Any thoughts on this? I feel like this is a pretty common use case: having
a large set of place names that you want folks to choose from. Would be
great if we could get it working.Thanks.
On 8 July 2014 14:09, Tom Smyth tom@sassafras.coop wrote:
Have people tried to use ODK with a very large cascading select one
option set? Say perhaps 120,000 options, several levels deep. So maybe 20MB
worth of data or so. Of course data transfer might be an issue depending on
connection but we could maybe get around that. I'm more concerned about how
the phone would handle it. Would it try to load the whole thing into
memory? I'm guessing yes. Though perhaps 20MB wouldn't be a big deal. Has
anyone tried something like this?--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech
--
Tom Smyth
Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech