I've been hitting my datastore read limits for the last while. Have just
turned off one of my unnecessary publishers to see if that helps any, and
the disable-faster-background-
actions has been ticked for a while already.
I've got really low submissions numbers - typically 3 or 4 records per day.
I do have 2 mappings set up via OpenFn, which check for new submissions on
2 of the forms every 10 minutes, but I was running fine previously without
these being an issue, so I'm pretty certain they are not the cause.
See attached graphs from the dashboard showing 5xx errors every evening
until the quotas are reset at 9am SAST again.
I'm beginning to wonder if there was a problem with the initial Aggregate
install that is causing errors pushing my datastore reads up. But with
devices in the field I want to put-off the haslle of relocating the forms
to a new Aggregate instance if at all possible.
Are there specific errors I should be checking my logs for that can affect
the number of datastore read operations?
Is there any way to confirm the integrity of the Aggregate instance?
Any other suggestions for tracking down the source of my problems?
I've been hitting my datastore read limits for the last while. Have just
turned off one of my unnecessary publishers to see if that helps any, and
the disable-faster-background-
actions has been ticked for a while already.
I've got really low submissions numbers - typically 3 or 4 records per day.
I do have 2 mappings set up via OpenFn, which check for new submissions on 2
of the forms every 10 minutes, but I was running fine previously without
these being an issue, so I'm pretty certain they are not the cause.
See attached graphs from the dashboard showing 5xx errors every evening
until the quotas are reset at 9am SAST again.
I'm beginning to wonder if there was a problem with the initial Aggregate
install that is causing errors pushing my datastore reads up. But with
devices in the field I want to put-off the haslle of relocating the forms to
a new Aggregate instance if at all possible.
Are there specific errors I should be checking my logs for that can affect
the number of datastore read operations?
Is there any way to confirm the integrity of the Aggregate instance?
Any other suggestions for tracking down the source of my problems?
It's a very small test form.
I've installed Android ODK on my phone
I submitted a few test submissions
The last few days I have exhausted my quote but I've not been using the form at all.
···
===================================================
AppEngine Application Problem
A problem has occurred that prevents your App Engine application from responding. Your application may have exceeded its per-minute or daily usage quotas.
etc...
When I look at the logs I get a single entry uuid that is being resubmitted repeatedly THOUSANDS of times every day.
14:06:27.150 GET 202 0 B 829 ms AppEngine-Google; (+http://code.google.com/appengine) /gae/uploadSubmissionsTask?fscUri=uuid%3Aa5218cb2-8a97-4db1-910d-efd726606ce4
And of course every day my quota gets exceeded: I've just woken up and looked and TWO quotas are exceeded
Frontend Instance Hours 23.00 of 28 Instance Hours 82% Standard
Datastore Read Operations 0.05 of 0.05 Million Ops 100% Standard
Obviously something has got corrupted and I could delete the form/instance/app and start again. But I'm sure it'll happen again and it's a bug which makes this AWESOME resource unusable (for me at least).
What I'm trying to do in Ethiopia is develop an app that allows an Ethiopian charity to log the amount of fluoride-removing fluor-appetite they are producing for local communities.
It was looking so cool until this happened.
Can you help? I can make you an admin or whatever.
Using OpenFn for this purpose simply does not make sense.
You are causing ODK Aggregate to fire up every 10 minutes. When it does
this, it reads in every form definition on the server so that it can cache
those definitions in memory. If you have a form with 100 fields, that is at
least 100 reads. If you do that every 10 minutes, that will be 14,400 reads
(624100) every day just to start up the server, and if you have two forms
of this size, that is 28,800 reads every day, again, just to start the
server.
If you then actually log in and look at one of the form's submissions tab
(not sure how you are using OpenFn), then each submission page will do at
least 100 reads (to read the first 100 submissions for that form). That's
another 14,400 reads.
And looking at a second form submissions tab to check for new data will
cause another 100 reads, minimum. That's another 14,400 reads.
At that point, you've burned 57,600 reads.
And you only have 60,000 free reads.
If you have a publisher to Google Spreadsheets, you can configure Google
Spreadsheets to send notifications when it receives a new row. See Tools
/ Notification rules... in the spreadsheet.
···
On Wed, Sep 16, 2015 at 5:14 AM, Yaw Anokwa wrote:
Hi Andrew,
What happens if you disable the OpenFn mappings?
Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.
I've been hitting my datastore read limits for the last while. Have just
turned off one of my unnecessary publishers to see if that helps any, and
the disable-faster-background-
actions has been ticked for a while already.
I've got really low submissions numbers - typically 3 or 4 records per
day.
I do have 2 mappings set up via OpenFn, which check for new submissions
on 2
of the forms every 10 minutes, but I was running fine previously without
these being an issue, so I'm pretty certain they are not the cause.
See attached graphs from the dashboard showing 5xx errors every evening
until the quotas are reset at 9am SAST again.
I'm beginning to wonder if there was a problem with the initial Aggregate
install that is causing errors pushing my datastore reads up. But with
devices in the field I want to put-off the haslle of relocating the
forms to
a new Aggregate instance if at all possible.
Are there specific errors I should be checking my logs for that can
affect
the number of datastore read operations?
Is there any way to confirm the integrity of the Aggregate instance?
Any other suggestions for tracking down the source of my problems?
It's a very small test form.
I've installed Android ODK on my phone
I submitted a few test submissions
The last few days I have exhausted my quote but I've not been using the
form at all.
When I visit the page I get an error message
AppEngine Application Problem
A problem has occurred that prevents your App Engine application from
responding. Your application may have exceeded its per-minute or daily
usage quotas.
etc...
When I look at the logs I get a single entry uuid that is being
resubmitted repeatedly THOUSANDS of times every day.
14:06:27.150 GET 202 0 B 829 ms AppEngine-Google; (+ http://code.google.com/appengine) /gae/uploadSubmissionsTask?
fscUri=uuid%3Aa5218cb2-8a97-4db1-910d-efd726606ce4
And of course every day my quota gets exceeded: I've just woken up and
looked and TWO quotas are exceeded
Frontend Instance Hours 23.00 of 28 Instance Hours 82% Standard
Datastore Read Operations 0.05 of 0.05 Million Ops 100% Standard
Obviously something has got corrupted and I could delete the
form/instance/app and start again. But I'm sure it'll happen again and
it's a bug which makes this AWESOME resource unusable (for me at least).
What I'm trying to do in Ethiopia is develop an app that allows an
Ethiopian charity to log the amount of fluoride-removing fluor-appetite
they are producing for local communities.
It was looking so cool until this happened.
Can you help? I can make you an admin or whatever.
It's a very small test form.
I've installed Android ODK on my phone
I submitted a few test submissions
The last few days I have exhausted my quote but I've not been using the
form at all.
When I visit the page I get an error message
AppEngine Application Problem
A problem has occurred that prevents your App Engine application from
responding. Your application may have exceeded its per-minute or daily
usage quotas.
etc...
When I look at the logs I get a single entry uuid that is being
resubmitted repeatedly THOUSANDS of times every day.
14:06:27.150 GET 202 0 B 829 ms AppEngine-Google; (+ http://code.google.com/appengine) /gae/uploadSubmissionsTask?fsc
Uri=uuid%3Aa5218cb2-8a97-4db1-910d-efd726606ce4
And of course every day my quota gets exceeded: I've just woken up and
looked and TWO quotas are exceeded
Frontend Instance Hours 23.00 of 28 Instance Hours 82% Standard
Datastore Read Operations 0.05 of 0.05 Million Ops 100% Standard
Obviously something has got corrupted and I could delete the
form/instance/app and start again. But I'm sure it'll happen again and
it's a bug which makes this AWESOME resource unusable (for me at least).
What I'm trying to do in Ethiopia is develop an app that allows an
Ethiopian charity to log the amount of fluoride-removing fluor-appetite
they are producing for local communities.
It was looking so cool until this happened.
Can you help? I can make you an admin or whatever.
