histogram: malloc: Cannot allocate memory

[eribertomota]

The command histogram always returns an error.

$ ./src/histogram tests/test.jpg
histogram: malloc: Cannot allocate memory

[hiirotsuki]

the bytes variable is negative when its passed to malloc. I think its trying to allocate enough memory to hold the file? I modified it to simply allocate the size of the test image but the result doesn't seem correct?

Bits: -2555935
One: 0, -0.000000
Zero: 0, -0.000000
1: 1, -0.000000
2: 0, -0.000000
3: 0, -0.000000
4: 0, -0.000000
5: 0, -0.000000
6: 0, -0.000000
7: 0, -0.000000
8: 0, -0.000000
9: 0, -0.000000
10: 0, -0.000000
11: 0, -0.000000
12: 0, -0.000000
13: 0, -0.000000
14: 0, -0.000000
15: 0, -0.000000
16: 0, -0.000000
17: 0, -0.000000
18: 0, -0.000000
19: 0, -0.000000
20: 0, -0.000000
21: 0, -0.000000
22: 0, -0.000000
23: 0, -0.000000
24: 0, -0.000000

[pyrolitic]

It's not immediately obvious what kind of file histogram expects. It opens the file and reads the first four bytes as a big endian number that is interpreted as the number of bits in the remainder of the file (rounded up to a whole byte). This turns out to match what the outguess program writes to the output file when extractonly is set:

        if (extractonly) {
                int bits;
                /* Wen extracting get the bitmap from the source handler */
                srch->get_bitmap(&bitmap, image, STEG_RETRIEVE);

                fprintf(stderr, "Writing %d bits\n", bitmap.bits);
                bits = htonl(bitmap.bits);
                fwrite(&bits, 1, sizeof(int), fout);
                fwrite(bitmap.bitmap, bitmap.bytes, sizeof(char), fout);
                exit (1);

Now the only way to set extractonly is to name program binary outguess-extract, as compares argv[0] against this string. I have no idea this is the case. I made a symlink with this name and invoked ~/outguess_0.4/bin/outguess-extract tests/test.jpg ~/out and it prints out

Reading tests/test.jpg....
Writing 9676 bits

The first four bytes of this file are 0000 25cc, (which is 0x25cc as it's big endian) which is indeed 9676.
Running histogram ~/out prints

Bits:   9676
One:    5555, 0.574101
Zero:   4121, 0.425899
  1:    6882, 0.711244
  2:     574, 0.059322
  3:     243, 0.025114
  4:     111, 0.011472
  5:      52, 0.005374
  6:      22, 0.002274
  7:       2, 0.000207
  8:       2, 0.000207
  9:       1, 0.000103
 10:       2, 0.000207
 11:       2, 0.000207
 12:       0, 0.000000
 13:       0, 0.000000
 14:       0, 0.000000
 15:       0, 0.000000
 16:       0, 0.000000
 17:       0, 0.000000
 18:       0, 0.000000
 19:       0, 0.000000
 20:       0, 0.000000
 21:       0, 0.000000
 22:       0, 0.000000
 23:       0, 0.000000
 24:       0, 0.000000

So in conclusion this isn't really an issue with histogram but a lack of documentation.