ESP8266 RealtimeDatabase OTA don't flash the new firmware

[ulver2812]

Hi, I'm using the RealtimeDatabase OTA example provided with this library, I changed the sketch environment variables, the RTDB firmware path, set the correct RTDB rules and uploaded the .bin firmware converted in base64 string on Firebase RTDB but when I run the sketch I get this in the Serial monitor:

Firebase Client v2.1.3
Initializing app...
Event task: 🔐 authTask, msg: authenticating, code: 7
Debug task: 🔐 authTask, msg: Connecting to server...
Event task: 🔐 authTask, msg: auth request sent, code: 8
Event task: 🔐 authTask, msg: auth response received, code: 9
Debug task: 🔐 authTask, msg: Terminating the server connection...
Event task: 🔐 authTask, msg: ready, code: 10
Updating your firmware (OTA)... 

and after this nothing happens, I tried alternatively the three functions in the code:

void ota_async();
void ota_async2();
void ota_await();

but nothing happens, apparently the esp8266 download the firmware in the RTDB but don't flash it in memory for some reason.

Where am I going wrong?

Thanks

[mobizt]

Plese make sure you define the macro for ota.

FirebaseClient/examples/RealtimeDatabase/OTA/OTA.ino

Line 17 in 2e2d7f7

#define ENABLE_OTA

For ESP8266, the WiFiClientSecure io buffers should set large enough to handle large SSL encrypted data for downloading.

You should set the maximum 16k for rx buffer like this

ssl_client.setBufferSizes(16384, 1024);

And the base64 encoded string of your firmware is already in your database, or using set/push File as in this example.

The result from running OTA example.

�Connecting to Wi-Fi...........
Connected with IP: 192.168.246.236

Firebase Client v2.1.3
Initializing app...
Event task: 🔐 authTask, msg: authenticating, code: 7
Debug task: 🔐 authTask, msg: Connecting to server...
Event task: 🔐 authTask, msg: auth request sent, code: 8
Event task: 🔐 authTask, msg: auth response received, code: 9
Debug task: 🔐 authTask, msg: Terminating the server connection...
Event task: 🔐 authTask, msg: ready, code: 10
Updating your firmware (OTA)...
Debug task: otaTask, msg: Connecting to server...
Download task: otaTask, downloaded 0% (2049 of 361582)
Download task: otaTask, downloaded 2% (8193 of 361582)
Download task: otaTask, downloaded 4% (16385 of 361582)
Download task: otaTask, downloaded 6% (22529 of 361582)
Download task: otaTask, downloaded 8% (30721 of 361582)
Download task: otaTask, downloaded 10% (36865 of 361582)
Download task: otaTask, downloaded 12% (45057 of 361582)
Download task: otaTask, downloaded 14% (51201 of 361582)
Download task: otaTask, downloaded 16% (59393 of 361582)
Download task: otaTask, downloaded 18% (65537 of 361582)
Download task: otaTask, downloaded 20% (73729 of 361582)
Download task: otaTask, downloaded 22% (79873 of 361582)
Download task: otaTask, downloaded 24% (88065 of 361582)
Download task: otaTask, downloaded 26% (94209 of 361582)
Download task: otaTask, downloaded 28% (102401 of 361582)
Download task: otaTask, downloaded 30% (108545 of 361582)
Download task: otaTask, downloaded 32% (116737 of 361582)
Download task: otaTask, downloaded 34% (124929 of 361582)
Download task: otaTask, downloaded 36% (131073 of 361582)
Download task: otaTask, downloaded 38% (139265 of 361582)
Download task: otaTask, downloaded 40% (145409 of 361582)
Download task: otaTask, downloaded 42% (153601 of 361582)
Download task: otaTask, downloaded 44% (159745 of 361582)
Download task: otaTask, downloaded 46% (167937 of 361582)
Download task: otaTask, downloaded 48% (174081 of 361582)
Download task: otaTask, downloaded 50% (182273 of 361582)
Download task: otaTask, downloaded 52% (188417 of 361582)
Download task: otaTask, downloaded 54% (196609 of 361582)
Download task: otaTask, downloaded 56% (202753 of 361582)
Download task: otaTask, downloaded 58% (210945 of 361582)
Download task: otaTask, downloaded 60% (217089 of 361582)
Download task: otaTask, downloaded 62% (225281 of 361582)
Download task: otaTask, downloaded 64% (231425 of 361582)
Download task: otaTask, downloaded 66% (239617 of 361582)
Download task: otaTask, downloaded 68% (247809 of 361582)
Download task: otaTask, downloaded 70% (253953 of 361582)
Download task: otaTask, downloaded 72% (262145 of 361582)
Download task: otaTask, downloaded 74% (268289 of 361582)
Download task: otaTask, downloaded 76% (276481 of 361582)
Download task: otaTask, downloaded 78% (282625 of 361582)
Download task: otaTask, downloaded 80% (290817 of 361582)
Download task: otaTask, downloaded 82% (296961 of 361582)
Download task: otaTask, downloaded 84% (305153 of 361582)
Download task: otaTask, downloaded 86% (311297 of 361582)
Download task: otaTask, downloaded 88% (319489 of 361582)
Download task: otaTask, downloaded 90% (325633 of 361582)
Download task: otaTask, downloaded 92% (333825 of 361582)
Download task: otaTask, downloaded 94% (339969 of 361582)
Download task: otaTask, downloaded 96% (348161 of 361582)
Download task: otaTask, downloaded 98% (356353 of 361582)
Download task: otaTask, downloaded 100% (361582 of 361582)
Download task: otaTask, complete!✅️
Update firmware completed.

Restarting...



 ets Jan  8 2013,rst cause:2, boot mode:(3,7)

[ulver2812]

Hi, I followed your instructions this is my code:

#define ENABLE_USER_AUTH
#define ENABLE_DATABASE
#define ENABLE_OTA

#include <FirebaseClient.h>
#include "ExampleFunctions.h" // Provides the functions used in the examples.

#if defined(ARDUINO_ARCH_SAMD)
#include <Internal_Storage_OTA.h>
#define OTA_STORAGE InternalStorage
#endif

#define WIFI_SSID "my_wifi"
#define WIFI_PASSWORD "my_wifi_psw"

#define API_KEY "my_api_key"
#define USER_EMAIL "user@test.com"
#define USER_PASSWORD "123456789"
#define DATABASE_URL "my_db_url"

void processData(AsyncResult &aResult);
void ota_async();
void ota_async2();
void ota_await();
void restart();

SSL_CLIENT ssl_client;

using AsyncClient = AsyncClientClass;
AsyncClient aClient(ssl_client);

UserAuth user_auth(API_KEY, USER_EMAIL, USER_PASSWORD, 3000 /* expire period in seconds (<3600) */);
FirebaseApp app;
RealtimeDatabase Database;
AsyncResult databaseResult;

bool taskComplete = false;

void setup()
{
    ssl_client.setBufferSizes(16384, 1024);

    Serial.begin(9600);
    WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

    Serial.print("Connecting to Wi-Fi");
    while (WiFi.status() != WL_CONNECTED)
    {
        Serial.print(".");
        delay(300);
    }
    Serial.println();
    Serial.print("Connected with IP: ");
    Serial.println(WiFi.localIP());
    Serial.println();

    Firebase.printf("Firebase Client v%s\n", FIREBASE_CLIENT_VERSION);

    set_ssl_client_insecure_and_buffer(ssl_client);

    Serial.println("Initializing app...");
    initializeApp(aClient, app, getAuth(user_auth), auth_debug_print, "🔐 authTask");

    // Or intialize the app and wait.
    // initializeApp(aClient, app, getAuth(user_auth), 120 * 1000, auth_debug_print);

    app.getApp<RealtimeDatabase>(Database);

    Database.url(DATABASE_URL);
}

void loop()
{
    // To maintain the authentication and async tasks
    app.loop();

    if (app.ready() && !taskComplete)
    {
        taskComplete = true;

#if defined(OTA_STORAGE)
        Database.setOTAStorage(OTA_STORAGE);
#endif

        // Assume that your firmware bin file was converted to base64 encoded string
        // and stores at "/examples/OTA/firmware"

        Serial.println("Updating your firmware (OTA)... ");
        ota_async();
        // ota_async2();
        // ota_await();
    }

    // For async call with AsyncResult.
    processData(databaseResult);
}

void processData(AsyncResult &aResult)
{
    // Exits when no result available when calling from the loop.
    if (!aResult.isResult())
        return;

    if (aResult.isEvent())
    {
        Firebase.printf("Event task: %s, msg: %s, code: %d\n", aResult.uid().c_str(), aResult.eventLog().message().c_str(), aResult.eventLog().code());
    }

    if (aResult.isDebug())
    {
        Firebase.printf("Debug task: %s, msg: %s\n", aResult.uid().c_str(), aResult.debug().c_str());
    }

    if (aResult.isError())
    {
        Firebase.printf("Error task: %s, msg: %s, code: %d\n", aResult.uid().c_str(), aResult.error().message().c_str(), aResult.error().code());
    }

    if (aResult.downloadProgress())
    {
        Firebase.printf("Download task: %s, downloaded %d%s (%d of %d)\n", aResult.uid().c_str(), aResult.downloadInfo().progress, "%", aResult.downloadInfo().downloaded, aResult.downloadInfo().total);
        if (aResult.downloadInfo().total == aResult.downloadInfo().downloaded)
        {
            Firebase.printf("Download task: %s, complete!✅️\n", aResult.uid().c_str());
            if (aResult.isOTA())
                restart();
        }
    }

    if (aResult.uploadProgress())
    {
        Firebase.printf("Upload task: %s, uploaded %d%s (%d of %d)\n", aResult.uid().c_str(), aResult.uploadInfo().progress, "%", aResult.uploadInfo().uploaded, aResult.uploadInfo().total);
        if (aResult.uploadInfo().total == aResult.uploadInfo().uploaded)
            Firebase.printf("Upload task: %s, complete!✅️\n", aResult.uid().c_str());
    }
}

void ota_async()
{
    // Async call with callback function.
    Database.ota(aClient, "/firmware/bin", processData, "otaTask");
}

void ota_async2()
{
    // Async call with AsyncResult for returning result.
    Database.ota(aClient, "/firmware/bin", databaseResult);
}

void ota_await()
{
    // Sync call which waits until the operation complete.
    bool status = Database.ota(aClient, "/firmware/bin");
    if (status)
        Serial.println("⚡OTA dowload task (await), complete!✅️");
    else
        Firebase.printf("Error, msg: %s, code: %d\n", aClient.lastError().message().c_str(), aClient.lastError().code());
}

void restart()
{
    Serial.println("Update firmware completed.");
    Serial.println();
#if defined(OTA_STORAGE)
    Serial.println("Applying update...");
    OTA_STORAGE.apply();
#elif defined(ESP32) || defined(ESP8266)
    Serial.println("Restarting...\n\n");
    ESP.restart();
#elif defined(ARDUINO_RASPBERRY_PI_PICO_W)
    Serial.println("Restarting...\n\n");
    rp2040.restart();
#endif
}

In my Firebase RTDB there is already the .bin converted in base64 as you can see in the screen here:

but the result is stuck on this, no new firmware flashing:

......
Connected with IP: 192.168.67.217

Firebase Client v2.1.3
Initializing app...
Event task: 🔐 authTask, msg: authenticating, code: 7
Debug task: 🔐 authTask, msg: Connecting to server...
Event task: 🔐 authTask, msg: auth request sent, code: 8
Event task: 🔐 authTask, msg: auth response received, code: 9
Debug task: 🔐 authTask, msg: Terminating the server connection...
Event task: 🔐 authTask, msg: ready, code: 10
Updating your firmware (OTA)... 

any advice?
Thanks 🙏

[mobizt]

You should follow the BareMinimum example for how to use library correctly.

The function set_ssl_client_insecure_and_buffer from ExampleFunctions.h is for demonstration or examples only.

You have to look at set_ssl_client_insecure_and_buffer, it overwrites setBufferSizes you already set.

[mobizt]

In brief

You should replace set_ssl_client_insecure_and_buffer() with ssl_client.setInsecure() and ssl_buffer.setBufferSizes(16384, 1024).

[ulver2812]

Yes, thank you I just tried in this way:

    ssl_client.setInsecure();
    ssl_client.setBufferSizes(16384, 1024);

    //set_ssl_client_insecure_and_buffer(ssl_client);

And then the ota_async(); and ota_async2(); do the firmware update but ota_await(); doesn't work. It start the firmware download and then print on serial:

Firebase Client v2.1.3
Initializing app...
Event task: 🔐 authTask, msg: authenticating, code: 7
Debug task: 🔐 authTask, msg: Connecting to server...
Event task: 🔐 authTask, msg: auth request sent, code: 8
Event task: 🔐 authTask, msg: auth response received, code: 9
Debug task: 🔐 authTask, msg: Terminating the server connection...
Event task: 🔐 authTask, msg: ready, code: 10
Updating your firmware (OTA)... 
⚡OTA dowload task (await), complete!✅️

and after nothing happens, I miss something more?

Thanks

[mobizt]

Your code should work if it has enough memory.

You should set the Heap (MMU options) that is suitable for SSL application, please see the Readme.md.

[ulver2812]

Yes, I already set the MMU to 16Kb cache + 48KB IRAM and second heap shared...
The ota_await(); is more memory consuming than the async task?

[mobizt]

There is no different between async and await because the internal process is the same.

The problem can be your flash memory e.g. free space or improper flash configuration depends on your flash size.

You should set the debug option in Arduino IDE to see the system or core library debug info.

You should try the OTA with Firebaase Storage and Google Cloud Storage examples. If it stucks in this process, your flash memory or flash config is the prroblem.

The OTA process depends on the Updater.h or Update.h of the device core libraries.

You should see more info when you enable debug for it.

[mobizt]

All OTA examples work fine for ESP32, ESP8266, Raspberry Pi Pico W and Arduino MKR (SAMD) devices.

[mobizt]

You should erase your flash and upload only firmware and no files in the data folder.

[mobizt]

I missed this sentence.

And then the ota_async(); and ota_async2(); do the firmware update but ota_await(); doesn't work. It start the firmware download and then print on serial:

There is no callback or async result assigned with all await functions then there is no way to sending such information to user.

Library has the debugging system and will not print anything to serial port without user consent.

If you want to debug or seeing the progress, use async instead.

The await OTA also works but no progress printing.

[ulver2812]

Ok thanks, I'm using the async version, this works well and I can also print the firmware update progress on the oled display.