Randomly calling a question set

Writing a logic-constrained survey that needs to randomly assign questions to sub-samples of respondents? For example, do you want to run a contingent valuation survey that needs to offer different bids to respondents at random to establish WTP/WTA?

To do this, you want to generate a number randomly from a specified range (i.e. the number of different question sets), store that number in a data field and use that number to call up questions.

Here's an easy how* with code in one place.

At some point in your survey, at least one page before you need the question(s) to diverge for different sub samples, insert a simple text entry item. You can have it say whatever you like, but it needs the following binding:

readonly="true()" calculate="once(int(random()*(X))+1)"

where X is the total number of different sub samples you wish to have. I.e., if you have eight sets of questions for eight sub samples, then X=8.

The computer will only make this calculation once, it will randomly return an integer 1 through X, and that integer cannot be changed by the surveyor (thanks to the readonly bind).

From that point on in the survey, you can assign your questions to sub samples with the following question binding:

relevant="(/data/randomvalue = Y)

where /data/randomvalue is the ID of the random value question and Y is the number of the sub sample for which the question is relevant.

That does the job. For the record, you should also be able to hide the random value generator simply by pulling references to it from the section of the XML while leaving other references.

A final note on the random function which perhaps is obvious but important from a research perspective. This will generate numbers randomly. That means that your subsamples will be assigned by pure chance and not evenly. If your overall sample is large enough, then each sub sample should approach 1/X of it.

But that's not the same as assigning respondents randomly into X equally sized sub samples.

To understand the ramifications of this, consider: I tested the above code 320 times, throwing up random numbers 1 through 8, with the following results.

1: 31 times
2: 39 times
3: 50 times
4: 46 times
5: 49 times
6: 26 times
7: 36 times
8: 43 times

Six really only came up 26 times which is disturbingly far from my ideal (even) assignment of 40.

I'm attaching a full sample XML form that generates 8 random values and calls eight different questions. I am sure there are other ways to go about this.

And I would very much appreciate any corrections here.

Warm regards,

Adam

*There may be other hows that are easier or better or more efficient. I'm an only-semi-tech savvy researcher using ODK to power surveys. Right now, I'm writing/coding a contingent valuation survey (with no budget for tech support).

But, after wracking my brain and scanning these message boards for a good long while, my code seems to work. So I'm posting my solution in one place for people to easily use.

Many, many thanks to all the people who contribute to all the various parts and platforms that make ODK work for researchers like me. And thanks to the regular message board contributors such as Yaw and Mitch; their earlier comments on other posts eventually helped sort me out.

questioncontingentonrandomval.xml (9.07 KB)

Thanks for writing this up, Adam!

Yaw

··· -- Need ODK consultants? Nafundi provides form design, server setup, in-field training, and software development for ODK. Go to https://nafundi.com to get started.

On Wed, May 4, 2016 at 6:19 PM, ajadhav@gmail.com wrote:

Writing a logic-constrained survey that needs to randomly assign questions to sub-samples of respondents? For example, do you want to run a contingent valuation survey that needs to offer different bids to respondents at random to establish WTP/WTA?

To do this, you want to generate a number randomly from a specified range (i.e. the number of different question sets), store that number in a data field and use that number to call up questions.

Here's an easy how* with code in one place.

At some point in your survey, at least one page before you need the question(s) to diverge for different sub samples, insert a simple text entry item. You can have it say whatever you like, but it needs the following binding:

readonly="true()" calculate="once(int(random()*(X))+1)"

where X is the total number of different sub samples you wish to have. I.e., if you have eight sets of questions for eight sub samples, then X=8.

The computer will only make this calculation once, it will randomly return an integer 1 through X, and that integer cannot be changed by the surveyor (thanks to the readonly bind).

From that point on in the survey, you can assign your questions to sub samples with the following question binding:

relevant="(/data/randomvalue = Y)

where /data/randomvalue is the ID of the random value question and Y is the number of the sub sample for which the question is relevant.

That does the job. For the record, you should also be able to hide the random value generator simply by pulling references to it from the section of the XML while leaving other references.

A final note on the random function which perhaps is obvious but important from a research perspective. This will generate numbers randomly. That means that your subsamples will be assigned by pure chance and not evenly. If your overall sample is large enough, then each sub sample should approach 1/X of it.

But that's not the same as assigning respondents randomly into X equally sized sub samples.

To understand the ramifications of this, consider: I tested the above code 320 times, throwing up random numbers 1 through 8, with the following results.

1: 31 times
2: 39 times
3: 50 times
4: 46 times
5: 49 times
6: 26 times
7: 36 times
8: 43 times

Six really only came up 26 times which is disturbingly far from my ideal (even) assignment of 40.

I'm attaching a full sample XML form that generates 8 random values and calls eight different questions. I am sure there are other ways to go about this.

And I would very much appreciate any corrections here.

Warm regards,

Adam

*There may be other hows that are easier or better or more efficient. I'm an only-semi-tech savvy researcher using ODK to power surveys. Right now, I'm writing/coding a contingent valuation survey (with no budget for tech support).

But, after wracking my brain and scanning these message boards for a good long while, my code seems to work. So I'm posting my solution in one place for people to easily use.

Many, many thanks to all the people who contribute to all the various parts and platforms that make ODK work for researchers like me. And thanks to the regular message board contributors such as Yaw and Mitch; their earlier comments on other posts eventually helped sort me out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

This is great Adam -- I had played with a similar method to randomly select
households as a sub-sample for a more comprehensive survey but never got
around to completing it. Thanks for sharing.

Would this also provide the means to randomly select household members from
a roster? Where the X in --> readonly="true()"
calculate="once(int(random()*(X))+1)" - is replaced with a variable
representing the total number of of members in a repeat group, such as

type name calculation
calculate count_child sum(${child_U5})
(where ${child_U5} is binary)

then use your binding to randomly select one child under 5 as:

readonly="true()" calculate="once(int(random()*(${count_child}))+1)

I'll need to try it out....

··· On Wednesday, May 4, 2016 at 12:19:42 PM UTC-4, aja...@gmail.com wrote: > > > Writing a logic-constrained survey that needs to randomly assign questions > to sub-samples of respondents? For example, do you want to run a contingent > valuation survey that needs to offer different bids to respondents at > random to establish WTP/WTA? > > To do this, you want to generate a number randomly from a specified range > (i.e. the number of different question sets), store that number in a data > field and use that number to call up questions. > > Here's an easy how* with code in one place. > > At some point in your survey, at least one page before you need the > question(s) to diverge for different sub samples, insert a simple text > entry item. You can have it say whatever you like, but it needs the > following binding: > > readonly="true()" calculate="once(int(random()*(X))+1)" > > where X is the total number of different sub samples you wish to have. > I.e., if you have eight sets of questions for eight sub samples, then X=8. > > The computer will only make this calculation once, it will randomly return > an integer 1 through X, and that integer cannot be changed by the surveyor > (thanks to the readonly bind). > > From that point on in the survey, you can assign your questions to sub > samples with the following question binding: > > relevant="(/data/randomvalue = Y) > > where /data/randomvalue is the ID of the random value question and Y is > the number of the sub sample for which the question is relevant. > > That does the job. For the record, you should also be able to hide the > random value generator simply by pulling references to it from the > section of the XML while leaving other references. > > A final note on the random function which perhaps is obvious but important > from a research perspective. This will generate numbers randomly. That > means that your subsamples will be assigned by pure chance and not evenly. > If your overall sample is large enough, then each sub sample should > approach 1/X of it. > > But that's not the same as assigning respondents randomly into X equally > sized sub samples. > > To understand the ramifications of this, consider: I tested the above code > 320 times, throwing up random numbers 1 through 8, with the following > results. > > 1: 31 times > 2: 39 times > 3: 50 times > 4: 46 times > 5: 49 times > 6: 26 times > 7: 36 times > 8: 43 times > > Six really only came up 26 times which is disturbingly far from my ideal > (even) assignment of 40. > > I'm attaching a full sample XML form that generates 8 random values and > calls eight different questions. I am sure there are other ways to go about > this. > > And I would very much appreciate any corrections here. > > Warm regards, > > Adam > > > *There may be other hows that are easier or better or more efficient. I'm > an only-semi-tech savvy researcher using ODK to power surveys. Right now, > I'm writing/coding a contingent valuation survey (with no budget for tech > support). > > But, after wracking my brain and scanning these message boards for a good > long while, my code seems to work. So I'm posting my solution in one place > for people to easily use. > > Many, many thanks to all the people who contribute to all the various > parts and platforms that make ODK work for researchers like me. And thanks > to the regular message board contributors such as Yaw and Mitch; their > earlier comments on other posts eventually helped sort me out.