CairoByteArray toBuffer() and toHexString() corruption

[xxu-circle]

Describe the bug
Follow up from #1455
it works partially, but the parsing the response still get corrupted bytearray. I have below as minimal repro
I believe the problem are inside toHexString, toBuffer which was doing some bigint conversion - and lost some zeroes along the way

To Reproduce

import { CairoByteArray } from "starknet";

describe("bytearray", () => {
  describe("bytearray", () => {
    const content = "0x000000010000001900000002";
    const contentBuffer = Buffer.from(content.slice(2), "hex");
    const cairoByteArray = new CairoByteArray(contentBuffer);
    const cairoBuffer: Buffer = cairoByteArray.toBuffer();
    const cairoHexString = cairoByteArray.toHexString();

    it("toBuffer() should match", () => {
      expect(cairoBuffer.toString("hex")).toEqual(contentBuffer.toString("hex"));
    });

    it("toHexString() should match", () => {
      expect(cairoHexString).toEqual(content);
    });
  });
});

 FAIL  e2e/bytearray.test.ts
  ● bytearray › bytearray › toBuffer() should match

    expect(received).toEqual(expected) // deep equality

    Expected: "000000010000001900000002"
    Received: "010000001900000002"

  ● bytearray › bytearray › toHexString() should match

    expect(received).toEqual(expected) // deep equality

    Expected: "0x000000010000001900000002"
    Received: "0x10000001900000002000000"

Expected behavior
toBuffer() and toHexString() should resolve to the same byte array.

• Starknet.js version 8.5.0

[xxu-circle]

@tabaktoni

[github-actions]

🎉 This issue has been resolved in version 8.5.1 🎉

The release is available on:

• npm package (@next dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[xxu-circle]

Thanks, there is still an edge case - but we're getting very close

import { CairoByteArray } from "starknet";

describe("bytearray", () => {
  describe("bytearray", () => {
    const content =
      "0x00000001111111111111111111111111111111111111111111111111111111111111111104d8bb41636b42d3c69039f3537333581cc19356a0c93904fa3e569498c23ad000000000000000000000000000000000000000000000000000000000000f42403333333333333333333333333333333333333333333333333333333333333333000000000000000000000000000000000000000000000000000000000000c3500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000054657374206275726e206d6573736167652066726f6d204176616c616e636865";
    const contentBuffer = Buffer.from(content.slice(2), "hex");
    const cairoByteArray = new CairoByteArray(contentBuffer);
    const felt252Hexes = cairoByteArray.toApiRequest();

    it("toBuffer() should match", () => {
      const reconstructed = CairoByteArray.factoryFromApiResponse(felt252Hexes.values());
      expect(reconstructed.toHexString()).toEqual(content);
    });
  });
});

  ● bytearray › bytearray › toBuffer() should match

    expect(received).toEqual(expected) // deep equality

    Expected: "0x00000001111111111111111111111111111111111111111111111111111111111111111104d8bb41636b42d3c69039f3537333581cc19356a0c93904fa3e569498c23ad000000000000000000000000000000000000000000000000000000000000f42403333333333333333333333333333333333333333333333333333333333333333000000000000000000000000000000000000000000000000000000000000c3500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000054657374206275726e206d6573736167652066726f6d204176616c616e636865"                                                                                              
    Received: "0x01111111111111111111111111111111111111111111111111111111111111111104d8bb41636b42d3c69039f3537333581cc19356a0c93904fa3e569498c23ad0000000000000000000000000000000000000000000000000000f424033333333333333333333333333333333333333333333333333333333333333330000000000000000000000000000000000000000000000c350000000000000000000000000000000000000000000000054657374206275726e206d6573736167652066726f6d204176616c616e636865

Likely relevant to

starknet.js/src/utils/cairoDataTypes/byteArray.ts

Line 247 in 2c2e1a2

const allChunks: Uint8Array[] = this.data.flatMap((chunk) => chunk.data);

where (chunk) => chunk.data lost all leading zeroes.

[penovicp]

This issue from the preceding comment has been addressed in version 8.5.2.

[xxu-circle]

Thanks. @penovicp can I ask for one more change.
When we are receiving hex strings from the RPC response, parsing the events from the transaction receipt using below code snippet

const customParsingStrategy: ParsingStrategy = {
  request: fastParsingStrategy.request,
  response: {
    ...fastParsingStrategy.response,
    [CairoByteArray.abiSelector]: (responseIterator: Iterator<string>) => {
      return CairoByteArray.factoryFromApiResponse(responseIterator).toBuffer();
    },
  },
};

factoryFromApiResponse does not work with actual RPC response because the hex string in the response does not preserve leading zeroes.
Looking at the code, either we will have to pad the leading zeroes in factoryFromApiResponse, or padd the chunks to 31 bytes in reconstructBytes

[penovicp]

The previous PR included a test that uses factoryFromApiResponse with values where the data chunks have leading zeros and they get preserved in the reconstructed CairoByteArray. So I checked the request/response handling, and the RPC nodes themselves will respond with hex strings without the leading zeros.

The approach that I'm leaning towards is to modify the underlying Uint8Array data property of the CairoBytes31 class so its size no longer corresponds to the initial input but is instead always exactly 31 elements. Toni should be back soon, I'll confer with him about what approach we will take.

Until an approach is chosen, it should be possible to modify the custom parsing strategy to do the padding with something similar to the following:

const customParsingStrategy: ParsingStrategy = {
  request: fastParsingStrategy.request,
  response: {
    ...fastParsingStrategy.response,
    [CairoByteArray.abiSelector]: (responseIterator: Iterator<string>) => {
      const response = Array.from<string>(responseIterator as any);
      const padded = response.map(x => `0x${(BigInt(x).toString(16).padStart(62, '0'))}`);
      return CairoByteArray.factoryFromApiResponse(padded.values()).toBuffer();
    },
  },
};

[xxu-circle]

Sounds good. I'll use the provided code snippet for now before you decide on the approach!

[xxu-circle]

Provided code works with some modification - I do not pad last two felts because padding pending_word will cause paddingDifference to become negative.