This appears to be a change in behavior by Google.
Testing this on opendatakit.appspot.com, I see this error in the log:
Unable to insert data into spreadsheet test_big_upload exception: 429
TOO_MANY_REQUESTS (GoogleSpreadsheet.java:652) { "code" : 429,
"errors" : [
{ "domain" : "global", "message" : "Insufficient tokens for quota
'WriteGroup' and limit 'USER-100s' of service 'sheets.googleapis.com'
for
consumer 'project_number:322300403941'.", "reason" :
"rateLimitExceeded" }
], "message" : "Insufficient tokens for quota 'WriteGroup' and limit
'USER-100s' of service 'sheets.googleapis.com' for consumer
'project_number:322300403941'.", "status" : "RESOURCE_EXHAUSTED" }
There is now a per-second and per-day quota for publishing into Google
Sheets. To see your quota status, navigate to the Google Cloud
Platform
console, the API Manager, the Library page, select Google Sheets and
then
select the quotas tab.
Or, click on this:
https://console.cloud.google.com/apis/api/sheets.googleapis.
com/quotas
Then sign into your appropriate gmail account (or change to the
appropriate one like you can do in gmail), and select the project in
which
you created your server's Oauth2 credentials.
There is a 100 writes per second per user default quota limit.
This limit is causing the publishing to enter a fail-and-retry
sequence,
which cycles into an Active_pause state and/or an active_retry and
eventually a paused state (which adds an even longer delay between
retries).
The quickest fix is to apply for an increased writes per second per
user
quota limit (click the pencil to the right of that value). I am
curious to
hear how the Google team will respond.
I will make a code change to reduce the size of the batch of rows
that ODK
Aggregate submits to Google Sheets, but this default quota limit is
very
very low, and there is little likelihood that we can stay below it,
especially if you are publishing several different forms, or forms
that have
many fields.
Pushing Google to allow greater publisher rates will be part of the
long-term solution to this issue.
On Tue, Apr 4, 2017 at 1:43 AM, otrtinteriors@gmail.com wrote:
Hi, I am running into a similar issue but with google spreadsheet
option.
All of the publishers created before seem to be working fine, but
the new
one created on 03/04/2017 initially has status of ACTIVE_PAUSE, then
it's
changing to ACTIVE_RETRY for a while to become Paused at the end.
This
process is being repeated whenever I click "Restart Publisher
Paused".
Strangely enough spreadsheet created by aggregate server is blank
(only
headers) despite of choosing "BOTH Upload Existing & Stream New
Submission
Data" option.
I'm using the newest version of ODK Collect (v1.5.1) and ODK
Aggregate
v1.4.13.
Thanks,
Rafal
On Thursday, 28 April 2016 00:18:32 UTC+1, sama...@verasolutions.org wrote:
Hi all, having some trouble publishing Aggregate v1.4.8
submissions to
a JSON server. I've outlined setup steps and error logs below - is
anyone
running into a similar issue?
I've tested a few different forms, all run through ODK Validate
without
errors. Submissions have been made using the latest version of
Collect.
Under Form Mgmt > Forms List > Publish, publisher settings for each
form are as follows:
- Publish to: Z-ALPHA JSON Server
- Data to publish: BOTH Upload Existing & Stream New Submission
Data
(have tested all three options unsuccessfully)
- Url to publish to: OpenFn inbox URL
- Authorization Token: [none]
- Include Media as: Links(URLs) to Media
Under Form Mgmt > Published Data, the STATUS of each submission is
either paused, active_retry, or active_paused; nothing is being
published.
App engine log files
- The following message is repeatedly thrown when attempting to
publish data to the OpenFn inbox URL:
org.opendatakit.aggregate.task.UploadSubmissionsWorkerImpl:
uploadAllSubmissions:
org.opendatakit.aggregate.exception.ODKExternalServiceException:
java.lang.IllegalStateException: Unrecognized Http request method
- More detail on the error found under 'View request logs':
0.1.0.2 - - [27/Apr/2016:19:36:03 +0530] "GET
/gae/uploadSubmissionsTask?fscUri=uuid%3A65d3eab1-f706-416f-
9fb6-a8987eec9dea
HTTP/1.1" 500 476
https://building-markets-1292.appspot.com/aggregateui/servic
esadminservice
"AppEngine-Google; (+http://code.google.com/appengine)"
"1.background.building-markets-1292.appspot.com" ms=394
cpu_ms=1619
cpm_usd=5.3196999999999997e-8
instance=00c61b117c8d9c5acb01ac6e5e9eb03290fbfaeea1a536abeb
app_engine_release=1.9.36 trace_id=bdc066555e45b7ced14d2
18ca3a01339
I've attached some screenshots here as well. Any troubleshooting
advice
is much appreciated.
Best,
Sam
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
