Understanding how example code works

[krupis]

Hello. I am very interested to learn how to save EEPROM life expectancy. I have slightly changed the example code "basic" but whenever I write to EEPROM and try to read back from it, I always read 0. .

My full code:

/*
 *  This sketch shows sector hoping across reboots
 */

#include <EEPROM32_Rotate.h>

EEPROM32_Rotate EEPROMr;

void setup() {

    // Init DEBUG --------------------------------------------------------------

    Serial.begin(115200);
    delay(2000);
    Serial.println();
    Serial.println();

    // Init EEPROM32_Rotate ----------------------------------------------------

    // You can add partitions manually by name
    EEPROMr.add_by_name("eeprom");
    EEPROMr.add_by_name("eeprom2");

    // Or add them by subtype (it will search and add all partitions with that subtype)
    //EEPROMr.add_by_subtype(0x99);

    // Offset where the magic bytes will be stored (last 16 bytes block)
    EEPROMr.offset(0xFF0);

    // Look for the most recent valid data and populate the memory buffer
    EEPROMr.begin(4096);

    // -------------------------------------------------------------------------

    uint8_t data;
    uint8_t quantity;
    quantity = 10;
    Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0));
    Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1));
    Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2));
    Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5));
    Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6));
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(5));

    Serial.println();
    Serial.printf("Writing 0x%02X to data\n", quantity);
    //EEPROMr.write(0, data + 1);
    EEPROMr.writeUShort(5, 10);

    Serial.println();
    Serial.printf("Commit %s\n", EEPROMr.commit() ? "OK" : "KO");
    Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0));
    Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1));
    Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2));
    Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5));
    Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6));
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(5));

    Serial.println("2ND TIME WRITING TO EEPROM");
    quantity = quantity +5;

    Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0));
    Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1));
    Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2));
    Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5));
    Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6));
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(6));
    
    
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(0));

    Serial.println();
    Serial.printf("Writing 0x%02X to data\n", quantity);
    //EEPROMr.write(0, data + 1);
    EEPROMr.writeUShort(6, quantity);

    Serial.println();
    Serial.printf("Commit %s\n", EEPROMr.commit() ? "OK" : "KO");
    Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0));
    Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1));
    Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2));
    Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5));
    Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6));
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(6));
    Serial.printf("Data      : 0x%02X\n", data = EEPROMr.read(0));

}

void loop() {}

I am doing 2 write cycles, first I write 10 to address 5, then I write 15 to address 6, during the read operation, I am not able to read back any of the values that I have set.

[xoseperez]

Hi I know it is not very helpful to say that it works for me :) This is my log of your code: Magic value for partition #0 is 16 Calculated CRC: 0x9129 Stored CRC : 0x9129 Magic value for partition #1 is 255 Calculated CRC: 0x4D4F Stored CRC : 0xFFFF Partition #1 has not passed the CRC check Current partition is #0 (eeprom) Current magic value is #16 Position 0: 0x9A Position 1: 0x10 Position 2: 0x53 Position 5: 0x0A Position 6: 0x0F Data : 0x0A Writing 0x0A to data Writing to partition #1 (eeprom2) Writing magic value #17 Commit OK Position 0: 0x9A Position 1: 0x10 Position 2: 0x53 Position 5: 0x0A Position 6: 0x00 Data : 0x0A 2ND TIME WRITING TO EEPROM Position 0: 0x9A Position 1: 0x10 Position 2: 0x53 Position 5: 0x0A Position 6: 0x00 Data : 0x00 Data : 0x9A Writing 0x0F to data Writing to partition #0 (eeprom) Writing magic value #18 Commit OK Position 0: 0x9A Position 1: 0x10 Position 2: 0x53 Position 5: 0x0A Position 6: 0x0F Data : 0x0F Data : 0x9A Please notice that you are using writeUShort which writes a 2-bytes unsigned short to memory in little endian. So if you do writeUShort(5,10) you are actually writing 0x0A to position 5 and 0x00 to position 6. And then when you do writeUShort(6,15) you are overwriting position 6 with a 0x0F and the writing 0x00 to position 7. Anyway this should still show the proper byte values on positions 5 and 6. So I guess the problem is somewhere else. Does it recognize the two EEPROM partitions correctly?

…

On Wed, 27 Jan 2021 at 07:43, krupis ***@***.***> wrote: Hello. I am very interested to learn how to save EEPROM life expectancy. I have slightly changed the example code "basic" but whenever I write to EEPROM and try to read back from it, I always read 0. . My full code: /* * This sketch shows sector hoping across reboots */ #include <EEPROM32_Rotate.h> EEPROM32_Rotate EEPROMr; void setup() { // Init DEBUG -------------------------------------------------------------- Serial.begin(115200); delay(2000); Serial.println(); Serial.println(); // Init EEPROM32_Rotate ---------------------------------------------------- // You can add partitions manually by name EEPROMr.add_by_name("eeprom"); EEPROMr.add_by_name("eeprom2"); // Or add them by subtype (it will search and add all partitions with that subtype) //EEPROMr.add_by_subtype(0x99); // Offset where the magic bytes will be stored (last 16 bytes block) EEPROMr.offset(0xFF0); // Look for the most recent valid data and populate the memory buffer EEPROMr.begin(4096); // ------------------------------------------------------------------------- uint8_t data; uint8_t quantity; quantity = 10; Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0)); Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1)); Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2)); Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5)); Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(5)); Serial.println(); Serial.printf("Writing 0x%02X to data\n", quantity); //EEPROMr.write(0, data + 1); EEPROMr.writeUShort(5, 10); Serial.println(); Serial.printf("Commit %s\n", EEPROMr.commit() ? "OK" : "KO"); Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0)); Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1)); Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2)); Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5)); Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(5)); Serial.println("2ND TIME WRITING TO EEPROM"); quantity = quantity +5; Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0)); Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1)); Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2)); Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5)); Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(0)); Serial.println(); Serial.printf("Writing 0x%02X to data\n", quantity); //EEPROMr.write(0, data + 1); EEPROMr.writeUShort(6, quantity); Serial.println(); Serial.printf("Commit %s\n", EEPROMr.commit() ? "OK" : "KO"); Serial.printf("Position 0: 0x%02X\n", EEPROMr.read(0)); Serial.printf("Position 1: 0x%02X\n", EEPROMr.read(1)); Serial.printf("Position 2: 0x%02X\n", EEPROMr.read(2)); Serial.printf("Position 5: 0x%02X\n", EEPROMr.read(5)); Serial.printf("Position 6: 0x%02X\n", EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(6)); Serial.printf("Data : 0x%02X\n", data = EEPROMr.read(0)); } void loop() {} I am doing 2 write cycles, first I write 10 to address 5, then I write 15 to address 6, during the read operation, I am not able to read back any of the values that I have set. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMPNZDL5O2NODYHIZ2NR3S36YYTANCNFSM4WUW3UIQ> .

-- *Xose Pérez* xose.perez@gmail.com @xoseperez@mastodont.cat <https://mastodont.cat/@xoseperez> @xoseperez <https://twitter.com/xoseperez> http://espurna.io http://tinkerman.cat http://thethingsnetwork.cat

[krupis]

Thanks for the response!

Okay, that is quite strange! I have now ran another example script that is provided by this library. Unfortunately, I dont think it recongnizes the EEPROM partitions.

However, I sucessfully ran simple EEPROM examples on the same device without any issues.

I assume my settings for the device are also correct:

[krupis]

Running my code above, the values that I get :

[xoseperez]

It looks like you don't have the proper partition table defined. I have never done it with the Arduino IDE but a quick google search threw some results you might want to check (like this one: https://robotzero.one/arduino-ide-partitions/). In the examples in my repo you have some definitions you can use (check https://github.com/xoseperez/eeprom32_rotate/blob/master/examples/basic/partition-table.csv ). BTW, using PlatformIO makes things much easier...

…

On Thu, 28 Jan 2021 at 09:53, krupis ***@***.***> wrote: Running my code above, the values that I get : [image: image] <https://user-images.githubusercontent.com/22237034/106113305-03478580-6157-11eb-873c-caa0ac117f21.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMPN3WUN6OVFGQB6VJDQ3S4EQY5ANCNFSM4WUW3UIQ> .

-- *Xose Pérez* xose.perez@gmail.com @xoseperez@mastodont.cat <https://mastodont.cat/@xoseperez> @xoseperez <https://twitter.com/xoseperez> http://espurna.io http://tinkerman.cat http://thethingsnetwork.cat

[krupis]

Are none of the preset partition tables correct?

Also, I am not sure whether that makes any difference but as I mentioned above, I am able to use original EEPROM without any problems. If you are using the same API as the original EEPROM, it should work the same right?

The code I tested for original EEPROM and it worked:

/*
   EEPROM Write

   Stores random values into the EEPROM.
   These values will stay in the EEPROM when the board is
   turned off and may be retrieved later by another sketch.
*/

#include "EEPROM.h"

// the current address in the EEPROM (i.e. which byte
// we're going to write to next)
int addr = 0;
#define EEPROM_SIZE 64
void setup()
{
  Serial.begin(115200);
  Serial.println("start...");
  if (!EEPROM.begin(EEPROM_SIZE))
  {
    Serial.println("failed to initialise EEPROM"); delay(1000000);
  }

  Serial.print("Value in EEPROM 0 before writing =");
  Serial.println(EEPROM.read(0));
  Serial.println("Writing value 10 to EEPROM 0 address");
  EEPROM.write(0,10);
  EEPROM.commit();
  
  Serial.print("Value in EEPROM 0 after writing =");
  Serial.println(EEPROM.read(0));

  Serial.println("Writing value 19 to EEPROM 0 address");
  EEPROM.write(0,19);
  EEPROM.commit();
  
  Serial.print("Value in EEPROM 0 after second writing =");
  Serial.println(EEPROM.read(0));

  
}

void loop()
{
  // need to divide by 4 because analog inputs range from

}

[krupis]

Okay I might have confused myself. Do I need to manually create partitions for eeprom1 and eeprom2 as suggested here?:

If so, I have created a new .csv file as suggested by your previous links:

and saved it under partitions folder.

In the boards.txt I have also added 3 lines:

esp32.menu.PartitionScheme.test=rotate_eeprom
esp32.menu.PartitionScheme.test.build.partitions=test
esp32.menu.PartitionScheme.test.upload.maximum_size=1310720

When I try to compile the code using this new partition I am getting an error:

Traceback (most recent call last):
  File "gen_esp32part.py", line 524, in <module>
  File "gen_esp32part.py", line 471, in main
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)
Failed to execute script gen_esp32part
exit status -1
Error compiling for board ESP32 Dev Module.

[krupis]

I figured out what was the problem. The CSV that I have saved in the excell was not correct format. I must make sure that it does not have a border between different rows/collums as shown here:

I now able to compile the code and read/write from flash

[xoseperez]

Nice! Glad you got it working!

…

On Fri, 29 Jan 2021 at 11:33, krupis ***@***.***> wrote: I figured out what was the problem. The CSV that I have saved in the excell was not correct format. I must make sure that it does not have a border between different rows/collums as shown here: [image: image] <https://user-images.githubusercontent.com/22237034/106264200-09f4fc00-622e-11eb-85ad-5212c191fd17.png> I now able to compile the code and read/write from flash [image: image] <https://user-images.githubusercontent.com/22237034/106264242-18dbae80-622e-11eb-8b33-69aa03380385.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMPN5CGM6LM36IQBCBARTS4KFGHANCNFSM4WUW3UIQ> .

-- *Xose Pérez* xose.perez@gmail.com @xoseperez@mastodont.cat <https://mastodont.cat/@xoseperez> @xoseperez <https://twitter.com/xoseperez> http://espurna.io http://tinkerman.cat http://thethingsnetwork.cat

[krupis]

Yes thanks for clarifying! I did not properly understood the part that you manually have to define your own custom partition. Also, could you clarify something for me...

In my project when the reset is initiated, I use:

        EEPROM.writeUShort(eeprom_counter, item_inside.quantity);               // 2^16 - 1
        eeprom_counter += sizeof(unsigned short);
        Serial.print("address after quantity=");
        Serial.println(eeprom_counter);
        
        EEPROM.writeString(eeprom_counter, item_inside.serial);
        eeprom_counter = eeprom_counter + 10;
        Serial.print("address after string=");
        Serial.println(eeprom_counter);
        
        //UPDATE THE eeprom counter
        EEPROM.write(0,eeprom_counter);
        EEPROM.commit();

And then when the program starts up one function is being called up:

void initial_eeprom_values(){
  eeprom_counter = EEPROM.read(0);
  Serial.print("Initial eeprom values = ");
  Serial.println(eeprom_counter);
  if(eeprom_counter >= 12)
    prev_eeprom_counter = eeprom_counter -12;
    
  EEPROM.get(prev_eeprom_counter,item_inside.quantity);
  Serial.print("quantity inside initial=");
  Serial.println(item_inside.quantity);
  if(item_inside.quantity == 65535){
    item_inside.quantity=0;
  }
  
  prev_eeprom_counter += sizeof(unsigned short);
  EEPROM.get(prev_eeprom_counter,item_inside.serial);
  prev_eeprom_counter = prev_eeprom_counter +10;
  Serial.print("serial isnide initial=");
  Serial.println(item_inside.serial);
}

This was my initial implementation of wear leveling but I soon realized that this is not going to do any good for me since its going to wear down Address 0 very fast since I save counter value there.

If I implement your EEPROM rotation code to my project where I have implemented 2 partitions "eeprom" and "eeprom2", does that mean that 1 reset cycle it is going to write values to "eeprom" and the second restart it will automatically write to "eeprom2". That should extend my EEPROM life cycle by 2 times right?

And since I do not use SPIFFS at all for my project, I could create as many partitions as possible and replace SPIFFS

[xoseperez]

Problem with the ESP32 EEPROM is that it is not an EEPROM. Instead it is emulated on top of a NAND flash that writes data in blocks of 4k. Even to write a single byte it erases the whole block and rewrites it again. So yeah, using the EEPROM32_Rotate library adds a layer on top that takes care of doing a round robin on the different EEPROM "partitions" which translates into extending lifetime. You can add more partitions if you want (3, 4,...).

…

On Fri, 29 Jan 2021 at 12:01, krupis ***@***.***> wrote: Yes thanks for clarifying! I did not properly understood the part that you manually have to define your own custom partition. Also, could you clarify something for me... In my project when the reset is initiated, I use: EEPROM.writeUShort(eeprom_counter, item_inside.quantity); // 2^16 - 1 eeprom_counter += sizeof(unsigned short); Serial.print("address after quantity="); Serial.println(eeprom_counter); EEPROM.writeString(eeprom_counter, item_inside.serial); eeprom_counter = eeprom_counter + 10; Serial.print("address after string="); Serial.println(eeprom_counter); //UPDATE THE eeprom counter EEPROM.write(0,eeprom_counter); EEPROM.commit(); And then when the program starts up one function is being called up: void initial_eeprom_values(){ eeprom_counter = EEPROM.read(0); Serial.print("Initial eeprom values = "); Serial.println(eeprom_counter); if(eeprom_counter >= 12) prev_eeprom_counter = eeprom_counter -12; EEPROM.get(prev_eeprom_counter,item_inside.quantity); Serial.print("quantity inside initial="); Serial.println(item_inside.quantity); if(item_inside.quantity == 65535){ item_inside.quantity=0; } prev_eeprom_counter += sizeof(unsigned short); EEPROM.get(prev_eeprom_counter,item_inside.serial); prev_eeprom_counter = prev_eeprom_counter +10; Serial.print("serial isnide initial="); Serial.println(item_inside.serial); } This was my initial implementation of wear leveling but I soon realized that this is not going to do any good for me since its going to wear down Address 0 very fast since I save counter value there. If I implement your EEPROM rotation code to my project where I have implemented 2 partitions "eeprom" and "eeprom2", does that mean that 1 reset cycle it is going to write values to "eeprom" and the second restart it will automatically write to "eeprom2". That should extend my EEPROM life cycle by 2 times right? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMPNZXX67JFXCTWGSX3CLS4KIRTANCNFSM4WUW3UIQ> .

-- *Xose Pérez* xose.perez@gmail.com @xoseperez@mastodont.cat <https://mastodont.cat/@xoseperez> @xoseperez <https://twitter.com/xoseperez> http://espurna.io http://tinkerman.cat http://thethingsnetwork.cat

[krupis]

Hey. Can you clarify me little bit regarding my project and use of rotate EEPROM? Perhaps I am still not fully understanding how to use it?

When the restart is initiated, I am executing this code:

        int addr=0;  
        Serial.printf("[EEPROM] Current partition   : %s\n", EEPROMr.current());
        EEPROMr.writeUShort(addr, item_inside.quantity);               // 2^16 - 1
        addr += sizeof(unsigned short);
        Serial.print("Quantity written=");
        Serial.println(item_inside.quantity);
        
        EEPROMr.writeString(addr, item_inside.serial);
        Serial.print("Serial written=");
        Serial.println(item_inside.serial);

        EEPROMr.commit();

        addr=0;

        EEPROMr.get(addr,item_inside.quantity);
        Serial.print("quantity inside after write initial=");
        Serial.println(item_inside.quantity);

        addr += sizeof(unsigned short);
        EEPROMr.get(addr,item_inside.serial);
        Serial.print("serial inside after write=");
        Serial.println(item_inside.serial);
        
        ESP.restart();

As you can see from the code above, I start from address 0, and write 2 variables to EEPROM. Quantity and serial. After EEPROMr.commit(), I read back the values that I have written just to confrim whether its correct and it looks fine
ESP.restart() will restart the ESP32 devices.

When the device start up again, I call this function:

void initial_eeprom_values(){
  int addr = 0;
  Serial.printf("[EEPROM] Current partition   : %s\n", EEPROMr.current());
 
    
  EEPROMr.get(addr,item_inside.quantity);
  Serial.print("quantity inside initial=");
  Serial.println(item_inside.quantity);
  
  if(item_inside.quantity == 65535){
    item_inside.quantity=0;
  }
  
  addr += sizeof(unsigned short);
  EEPROMr.get(addr,item_inside.serial);
  Serial.print("serial isnide initial=");
  Serial.println(item_inside.serial);
}

The function above supposed to read back the last values that have been saved in EEPROM before the erase but it does not work. Also, it does not look like partition 2 of EEPROM is ever reached, it is always first partition:

[krupis]

Perhaps I havent properly understood the working principle of this library. I have also noticed some strange behaviour of that program I have shown you before where we write to EEPROM 2 times to position 5 and position 6. When I write these values I can read them back correctly, however, when I restart the device, I expect the position5 and position6 have some values saved from the previous time, but it reads back 0xFF from both? Surely that is not correct behaviour right?

[xoseperez]

Are you certain the sketch defines both partitions? What do you get if you add the following code at the beginning? Serial.printf("[EEPROM] Reserved partitions : %u\n", EEPROMr.size()); Serial.printf("[EEPROM] Partition size : %u\n", EEPROMr.length()); Serial.printf("[EEPROM] Partitions in use : "); for (uint32_t i = 0; i < EEPROMr.size(); i++) { if (i>0) Serial.print(", "); Serial.print(EEPROMr.name(i)); } Serial.println(); Serial.printf("[EEPROM] Current partition : %s\n", EEPROMr.current());

…

On Tue, 2 Feb 2021 at 06:45, krupis ***@***.***> wrote: Perhaps I havent properly understood the working principle of this library. The example as I initially showed where we write 2 times to the EEPROM works fine, however, in my real program I only write once before the restart and that does not seem to work at all. When I write just once before the restart. it never gets to partition 2 and cannot read back the values that I have previously set. Are this library intended to be used where we write to EEPROM more than once before restart? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMPNY76C6SQRBF4AMNMNDS46GPPANCNFSM4WUW3UIQ> .

-- *Xose Pérez* xose.perez@gmail.com @xoseperez@mastodont.cat <https://mastodont.cat/@xoseperez> @xoseperez <https://twitter.com/xoseperez> http://espurna.io http://tinkerman.cat http://thethingsnetwork.cat

[krupis]

I have swapped the ESP32 device, and added your sketch:

As you can see, I have both partitions recognized. Its weird why I am not able to read back the data that I have written after the restart