Visualization of the latent representation

[nitishmital]

This is not an issue with the code. I would just like to know how you generate the visualizations of the latents in Fig. 2 of your paper "Variational Image Compression with a Scale Hyperprior".

The latents consist of 192 channels. To obtain Fig. 2b, do you plot one channel out of them, or do you pass a subset of the channels through the decoder to obtain Fig. 2b? The dimensions of the latents are not the same as the image, so I cannot figure out how the images in Fig. 2 above are generated with the same dimensions as the input image.

Thanks.

[jonaballe]

Hi nitishmital,

the right three subfigures are all visualizations of the latent space (y), without passing them through the decoder. We simply picked one of the 192 channels for visualization. Other channels look quite similar. The spatial resolution of the latent space is reduced by a factor of 16 in each dimension, which is why the visualizations look blocky compared to the image on the left.

Hope this helps!

[nitishmital]

Thanks, that helps.

[hzyBupt]

Hi,I am a student, new to this field. I am reproducing these four pictures.

According to the description of Fig. 2, can I directly regarded the right three subfigures as the visualization of y, indexes and y_hat in the BMSHJ2018Model model（select the same channel）?

[jonaballe]

Not quite, the left one would correspond to y, the middle one to self.scale_fn(indexes), the right one to y / self.scale_fn(indexes). Possibly you need to round y first, I forgot how we did it for the paper, but it should give you almost the same result.

[hzyBupt]

Thanks!