REST API Submission Error: Payload String too Long

Tariqul_Islam · February 23, 2025, 5:26am

Hi All,

Thank you in advance to give me some time to read my problem and providing me elegant solutions.

My problem is: When I am trying to send my Submission in ODK Central REST Submission API then it is giving me 500 Internal Server Error. In server I use docker compose log service and I got error: payload string too long. When I am sending relatively small form then it is fine and working nicely.

My server has 4GB RAM and 40GB Disk Space. I have 20GB free space and only 1GB Ram is used.

Please help me to resolve my Issue so that I can send big forms. Please its an urgent matter to me. Please help.

renato_cava · April 30, 2025, 7:57pm

Hi @Tariqul_Islam

I have the same problem right now. I'm looking for solutions.

LN · May 1, 2025, 4:51pm

Apologies you didn't get a response @Tariqul_Islam. Were you able to address the issue?

That's an error coming from the database layer. How big are these submissions @renato_cava and @Tariqul_Islam?

renato_cava · May 1, 2025, 5:37pm

Hello @LN .

I haven't calculated the exact size, but the form has 500 fields to fill out. That puts it over the limit. I was checking the function that limits the size.

Actually, everything starts with the webhook we deployed alongside Docker Central. This creates the function named audit.new_audit_log that listens and notifies.

github.com/hotosm/central-webhook

db/trigger.go

main


      
          	// This trigger runs on the `audits` table by default, and creates a new event
          	// in the odk-events queue when a new event is created in the table
          
          	if tableName == "" {
          		// default table (this is configurable for easier tests mainly)
          		tableName = "audits"
          	}
          
          	// SQL for creating the function
          	createFunctionSQL := `
          		CREATE OR REPLACE FUNCTION new_audit_log() RETURNS trigger AS
          		$$
          		DECLARE
          			js jsonb;
          			action_type text;
          			result_data jsonb;
          		BEGIN
          			-- Serialize the NEW row into JSONB
          			SELECT to_jsonb(NEW.*) INTO js;
          
          			-- Add the DML action (INSERT/UPDATE)

@spwoodcock

Inside the new_audit_log function, a function called pg_notify is executed (i.e. PERFORM pg_notify('odk-events', js::text);).

This notify function is the one you're referring to, which then calls the Async_Notify function to use the NOTIFY_PAYLOAD_MAX_LENGTH variable.

#define NOTIFY_PAYLOAD_MAX_LENGTH (BLCKSZ - NAMEDATALEN - 128)

My final question is whether it's possible to condition the new_audit_log function created for the webhook and condition the execution of the pg_notify function.

For example:

IF length(js::text) <= 8000 THEN
PERFORM pg_notify('odk-events', js::text);
END IF;

Thanks for your time

spwoodcock · May 1, 2025, 6:24pm

Hi @renato_cava, please direct any requests about the webhook service to the github repo

Its not maintained by the ODK team, so support probably doesnt belong on the forums.

If you open an issue with the request, we could consider adding a new trigger event