Error(-11): read Timeout for long JSON in POST requests

[santichente]

Basic Infos

• [ X] This issue complies with the issue POLICY doc.
• [X ] I have read the documentation at readthedocs and the issue is not addressed there.
• [ X] I have tested that the issue is present in current master branch (aka latest git).
• [ X] I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• [X ] I have filled out all fields below.

Platform

• Hardware: [ESP-12|ESP-01|ESP-07|ESP8285 device|other]
• Core Version: [latest git hash or date]
• Development Env: [Arduino IDE|Platformio|Make|other]
• Operating System: [Windows|Ubuntu|MacOS]

Settings in IDE

• Module: [Generic ESP8266 Module|Wemos D1 mini r2|Nodemcu|other]
• Flash Mode: [qio|dio|other]
• Flash Size: [4MB/1MB]
• lwip Variant: [v1.4|v2 Lower Memory|Higher Bandwidth]
• Reset Method: [ck|nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz|160MHz]
• Upload Using: [OTA|SERIAL]
• Upload Speed: [115200|other] (serial upload only)

Problem Description

Hi, I send an HTTP POST request with a JSON to my server and it works, but when I tried to send a longer JSON POST with a size 2940 of capacity or as with 1666 characters, or more, the response code is (-11) read timeout. I tried to change the CPU frequency, other features of the module, and the set time out, but didn't work fine. I send the same HTTP POST request in POSTMAN and this works correctly.
Thanks

MCVE Sketch

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266HTTPClient.h>
#include <ArduinoJson.h>

const char* wifiName = "******";
const char* wifiPass = "******";

//Web Server address to read/write from
const   char *host = "http://development.plantit.io/device/getState/?device_id=1";

void setup() {

  Serial.begin(115200);
  delay(10);
  Serial.println("Comienza");

  Serial.print("Connecting to ");
  Serial.println(wifiName);

  WiFi.begin(wifiName, wifiPass);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());   //You can get IP address assigned to ESP
}

void loop() {

  HTTPClient http;
  http.setTimeout(60000);  //Se configura el tiempo de espera para realizar la conexión
  String   SdJson = "[{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":0,\"SLAG2\":0,\"SLAG3\":0,\"SLAG4\":0},\"times\":{\"time1\":\"12:00\"}},{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":0,\"SLAG2\":0,\"SLAG3\":0,\"SLAG4\":0},\"times\":{\"time1\":\"12:00\"}},{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":0,\"SLAG2\":0,\"SLAG3\":0,\"SLAG4\":0},\"times\":{\"time1\":\"12:00\"}},{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":0,\"SLAG2\":0,\"SLAG3\":0,\"SLAG4\":0},\"times\":{\"time1\":\"12:00\"}}]";
  // "[{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":0,\"SLAG2\":0,\"SLAG3\":0,\"SLAG4\":0},\"times\":{\"time1\":\"12:00\"}},{\"sensor\":{\"S1\":{\"T\":99,\"H\":60.73},\"S2\":{\"T\":22.22,\"H\":86.22},\"S3\":{\"T\":19.45,\"H\":71.98},\"S4\":{\"T\":25.67,\"H\":49.67}},\"SLAL\":{\"SLAL1\":0,\"SLAL2\":0,\"SLAL3\":0,\"SLAL4\":0},\"device\":{\"MAC\":\"50:02:91:4D:F6:ED\",\"type\":\"prueba\",\"function\":\"prueba\",\"n_devices\":4,\"id\":\"XXX\",\"GateWayIP\":\"192.168.4.1\"},\"SLAG\":{\"SLAG1\":1,\"SLAG2\":1,\"SLAG3\":0,\"SLAG4\":1},\"times\":{\"time1\":\"12:00\"}}]";
  int lenoutput = SdJson.length();
  const size_t sdcapacity = lenoutput * 3; //Capacidad del JSON a almacenar en la conexión del Cliente
  DynamicJsonDocument sdroot(sdcapacity);//Declaración JSON
  auto error = deserializeJson(sdroot, SdJson);  //Deserialización del Json, para añadirle el SLA Global y reenviar al Servidor global
  if (error) { //Condición de error de deserialización
    Serial.println(F("deserializeJson() failed with code"));
    Serial.println(error.c_str()); //Mensaje de error en específico
    return; //Se rompe el ciclo
  }
  JsonObject obje = sdroot.as<JsonObject>(); //Declaración jsonobject con el mensaje proveniente del client
  String sdoutput;  //Declaración String JSON a enviar al servidor con toda la información del slave
  serializeJson(sdroot, sdoutput);  //Serialización del JSONObject con el string JSON a enviar
  // Send request
  http.begin(host);
  Serial.println("Post!");

//  Serial.println(SdJson);  //Se Imprime el JSON para su comprobación
  int httpCode = http.POST(sdoutput); // POST request con el JSON del cliente al servidor global

  if (httpCode > 0) { //Condición de conexión exitosa
    Serial.printf("[HTTP] GET... code: %d\n", httpCode); //Código obtenido de la respuesta exitosa
  }
  else { //Condición de fallo en conexión al servidor global
    Serial.printf("[HTTP] POST... failed, error: %s\n", http.errorToString(httpCode).c_str());  //Impresión de error en conexión Global
  }

  // Disconnect
  http.end();
}

Debug Messages

WiFi connected
IP address:
192.168.0.3
[HTTP-Client][begin] url: http://development.plantit.io/device/getState/?device_id=1
[HTTP-Client][begin] host: development.plantit.io port: 80 url: /device/getState/?device_id=1
Post!
[HTTP-Client][sendRequest] type: 'POST' redirCount: 0
[HTTP-Client] connected to development.plantit.io:80
[HTTP-Client] sending request header

POST /device/getState/?device_id=1 HTTP/1.1
Host: development.plantit.io
User-Agent: ESP8266HTTPClient
Connection: keep-alive
Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0
Content-Length: 1465

[HTTP-Client][returnError] error(-11): read Timeout
[HTTP-Client][returnError] tcp stop
[HTTP] POST... failed, error: read Timeout
[HTTP-Client][end] tcp is closed

[devyte]

In the MCVE I see this:

Serial.println(SdJson); //Se Imprime el JSON para su comprobación
But I don't see anything printed in the Debug messages, so the posted MCVE doesn't match the debug messages shown. Please fix that.

Also:

• Why are doing the following, it makes no sense:
  JsonObject obje = sdroot.as<JsonObject>();
• what is the output of adding this:
  Serial.println(sdoutput);
  just before http.POST() ?

[santichente]

Hi, I erase the SdJson because it was a long message to visualize, but this does appear in the debug messages, I am going to comment a line of code to avoid confusion.
About the following, I tried to deserialize and serialize the JSON String to avoid this problem, it works with short JSON anyway.
...
JsonObject obje = sdroot.as();
...

Regarding Serial.println(sdoutput); the following is the sdoutput:
[{"sensor":{"S1":{"T":99,"H":60.73},"S2":{"T":22.22,"H":86.22},"S3":{"T":19.45,"H":71.98},"S4":{"T":25.67,"H":49.67}},"SLAL":{"SLAL1":0,"SLAL2":0,"SLAL3":0,"SLAL4":0},"device":{"MAC":"50:02:91:4D:F6:ED","type":"prueba","function":"prueba","n_devices":4,"id":"XXX","GateWayIP":"192.168.4.1"},"SLAG":{"SLAG1":0,"SLAG2":0,"SLAG3":0,"SLAG4":0},"times":{"time1":"12:00"}},{"sensor":{"S1":{"T":99,"H":60.73},"S2":{"T":22.22,"H":86.22},"S3":{"T":19.45,"H":71.98},"S4":{"T":25.67,"H":49.67}},"SLAL":{"SLAL1":0,"SLAL2":0,"SLAL3":0,"SLAL4":0},"device":{"MAC":"50:02:91:4D:F6:ED","type":"prueba","function":"prueba","n_devices":4,"id":"XXX","GateWayIP":"192.168.4.1"},"SLAG":{"SLAG1":0,"SLAG2":0,"SLAG3":0,"SLAG4":0},"times":{"time1":"12:00"}},{"sensor":{"S1":{"T":99,"H":60.73},"S2":{"T":22.22,"H":86.22},"S3":{"T":19.45,"H":71.98},"S4":{"T":25.67,"H":49.67}},"SLAL":{"SLAL1":0,"SLAL2":0,"SLAL3":0,"SLAL4":0},"device":{"MAC":"50:02:91:4D:F6:ED","type":"prueba","function":"prueba","n_devices":4,"id":"XXX","GateWayIP":"192.168.4.1"},"SLAG":{"SLAG1":0,"SLAG2":0,"SLAG3":0,"SLAG4":0},"times":{"time1":"12:00"}},{"sensor":{"S1":{"T":99,"H":60.73},"S2":{"T":22.22,"H":86.22},"S3":{"T":19.45,"H":71.98},"S4":{"T":25.67,"H":49.67}},"SLAL":{"SLAL1":0,"SLAL2":0,"SLAL3":0,"SLAL4":0},"device":{"MAC":"50:02:91:4D:F6:ED","type":"prueba","function":"prueba","n_devices":4,"id":"XXX","GateWayIP":"192.168.4.1"},"SLAG":{"SLAG1":0,"SLAG2":0,"SLAG3":0,"SLAG4":0},"times":{"time1":"12:00"}}]

This is the same resultant JSON of Serial.println(SdJson), and both were sent correctly in POSTMAN, so in https://arduinojson.org/v6/assistant/ you would to prove that its a correct JSON.

[Coconuts-Palace]

Hello, same problem here, long JSON can't be POST, I get timeout each time, but in postman it work fine with the same request

[d-a-v]

I can't reproduce with MCE.
Even with a 3KB post I get 200/OK with a local server

Can you enable HTTP_CLIENT debug mode and provide its output ?

[Coconuts-Palace]

Here is my HTTP_CLIENT debug, but i cannot manage to put the connection in close mode, I'm supposed to do it with http.addHeader no ?

12:37:01.784 -> User-Agent: ESP8266HTTPClient
12:37:01.784 -> Connection: keep-alive
12:37:01.784 -> Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0
12:37:01.784 -> Content-Type: text/plain
12:37:01.784 -> Transfer-Encoding: chunked
12:37:01.784 -> Content-Length: 10
12:37:01.784 ->
12:37:01.784 -> -----
12:37:06.100 -> pm open,type:2 0
12:37:43.111 -> [HTTP-Client][returnError] error(-11): read Timeout
12:37:43.111 -> [HTTP-Client][returnError] tcp stop
12:37:43.111 -> [HTTP] POST... failed, error: read Timeout
12:37:43.111 -> [HTTP-Client][end] tcp is closed

[santichente]

Its here
...
Post!
...
[{"sensor":{"s1":{"t":615.35,"h":"-6274.38"},"s2":{"t":615.35,"h":"-6274.38"},"s3":{"t":615.35,"h":"-6274.38"},"s4":{"t":615.35,"h":"-6274.38"},"s5":{"t":615.35,"h":"-6274.38"},"s6":{"t":-40,"h":"-4.65"}},"SLAL":{"SLAL1":10,"SLAL2":10,"SLAL3":0,"SLAL4":0},"device":{"MAC":"2C:F4:32:65:0D:22","type":"suelo","function":"sensor","n_devices":4,"hash":"TAT-008","MACMaster":"2C:F4:32:65:0E:D2","GateWayIP":"192.168.4.1","Conexion":"local"},"SLAG":{"SLAG1":9,"SLAG2":9,"SLAG3":0,"SLAG4":9},"times":{"time1":"2020:8:27:11:41:38"}},{"sensor":{"s1":{"t":19,"h":60.46122},"s2":{"lx":1175.04},"s3":{"lx":1685},"s4":{"ppm":400},"s5":{"ppm":388}},"SLAL":{"SLAL1":1,"SLAL2":1,"SLAL3":0,"SLAL4":0},"device":{"MAC":"48:3F:DA:48:BD:E7","type":"ambiente","function":"sensor","n_devices":4,"hash":"plantit","MACMaster":"plantit","GateWayIP":"192.168.4.1","Conexion":"local"},"SLAG":{"SLAG1":10,"SLAG2":10,"SLAG3":0,"SLAG4":10},"times":{"time1":"2020:8:27: 11:43:2"}},{"sensor":{"s1":{"t":615.35,"h":"-6274.38"},"s2":{"t":615.35,"h":"-6274.38"},"s3":{"t":615.35,"h":"-6274.38"},"s4":{"t":615.35,"h":"-6274.38"},"s5":{"t":615.35,"h":"-6274.38"},"s6":{"t":-40,"h":"-4.65"}},"SLAL":{"SLAL1":0,"SLAL2":0,"SLAL3":0,"SLAL4":0},"device":{"MAC":"2C:F4:32:65:0D:22","type":"suelo","function":"sensor","n_devices":4,"hash":"TAT-008","MACMaster":"2C:F4:32:65:0E:D2","GateWayIP":"192.168.4.1","Conexion":"local"},"SLAG":{"SLAG1":11,"SLAG2":11,"SLAG3":0,"SLAG4":11},"times":{"time1":"2020:8:27: 11:43:27"}}]
[HTTP-Client][sendRequest] type: 'POST' redirCount: 0
[HTTP-Client] connected to development.plantit.io:80
[HTTP-Client] sending request header
....
POST /device/getState/?device_id=1 HTTP/1.1
Host: development.plantit.io
User-Agent: ESP8266HTTPClient
Connection: keep-alive
Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0
Content-Length: 1475

---

[HTTP-Client][returnError] error(-11): read Timeout
[HTTP-Client][returnError] tcp stop
[HTTP] POST... failed, error: read Timeout
[HTTP-Client][end] tcp is closed
....

[d-a-v]

@santichente with latest master or beta version, can you add this to your sketch:

#include <Netdump.h>
NetCapture::Netdump nd;

in setup():

  Serial.begin(115200);
  nd.printDump(Serial, NetCapture::Packet::PacketDetail::FULL, [](NetCapture::Packet p) { return p.hasPort(80); });

Then you can reproduce the error and copy your logs (including netdump logs).