Skip to content

Add the ability to scale down images during decode (IDCT scaling) #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 7, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 21 additions & 10 deletions src/decoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ pub struct ImageInfo {
/// JPEG decoder
pub struct Decoder<R> {
reader: R,
scale: usize,

frame: Option<FrameInfo>,
dc_huffman_tables: Vec<Option<HuffmanTable>>,
Expand All @@ -70,8 +71,15 @@ pub struct Decoder<R> {
impl<R: Read> Decoder<R> {
/// Creates a new `Decoder` using the reader `reader`.
pub fn new(reader: R) -> Decoder<R> {
Decoder::new_scaled(reader, 8)
}

/// Creates a new `Decoder` using the reader `reader` that scales
/// down the image by a factor of `scale/8`.
pub fn new_scaled(reader: R, scale: usize) -> Decoder<R> {
Decoder {
reader: reader,
scale: scale,
frame: None,
dc_huffman_tables: vec![None, None, None, None],
ac_huffman_tables: vec![None, None, None, None],
Expand Down Expand Up @@ -100,8 +108,8 @@ impl<R: Read> Decoder<R> {
};

Some(ImageInfo {
width: frame.image_size.width,
height: frame.image_size.height,
width: frame.output_size.width,
height: frame.output_size.height,
pixel_format: pixel_format,
})
},
Expand Down Expand Up @@ -153,7 +161,7 @@ impl<R: Read> Decoder<R> {
return Err(Error::Unsupported(UnsupportedFeature::Hierarchical));
}

let frame = parse_sof(&mut self.reader, marker)?;
let frame = parse_sof(&mut self.reader, marker, self.scale)?;
let component_count = frame.components.len();

if frame.is_differential {
Expand Down Expand Up @@ -329,7 +337,7 @@ impl<R: Read> Decoder<R> {
}

let frame = self.frame.as_ref().unwrap();
compute_image(&frame.components, &planes, frame.image_size, self.is_jfif, self.color_transform)
compute_image(&frame.components, &planes, frame.output_size, self.is_jfif, self.color_transform)
}

fn read_marker(&mut self) -> Result<Marker> {
Expand Down Expand Up @@ -436,7 +444,7 @@ impl<R: Read> Decoder<R> {
let x = (block_num % blocks_per_row) as u16;
let y = (block_num / blocks_per_row) as u16;

if x * 8 >= component.size.width || y * 8 >= component.size.height {
if x * component.dct_scale as u16 >= component.size.width || y * component.dct_scale as u16 >= component.size.height {
continue;
}

Expand Down Expand Up @@ -762,12 +770,15 @@ fn compute_image(components: &[Component],
return Ok(data[0].clone())
}

let mut buffer = vec![0u8; component.size.width as usize * component.size.height as usize];
let line_stride = component.block_size.width as usize * 8;
let width = component.size.width as usize;
let height = component.size.height as usize;

let mut buffer = vec![0u8; width * height];
let line_stride = width * component.dct_scale;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was inadvertently changed from component.block_size.width to component.size.width, causing #125


for y in 0 .. component.size.height as usize {
for x in 0 .. component.size.width as usize {
buffer[y * component.size.width as usize + x] = data[0][y * line_stride + x];
for y in 0 .. width {
for x in 0 .. height {
buffer[y * width + x] = data[0][y * line_stride + x];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#125 seems to have been introduced by this line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I swapped width and height in the for loop ranges when extracting those expressions as variables. 🤦‍♂

Also, this presumably means there are no single-channel JPEG images in the test suite?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, I missed that ! I fixed that too in #126

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test suite could clearly be more extensive !

}
}

Expand Down
9 changes: 8 additions & 1 deletion src/idct.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,15 @@
// One example is tests/crashtest/images/imagetestsuite/b0b8914cc5f7a6eff409f16d8cc236c5.jpg
// That's why wrapping operators are needed.

pub fn dequantize_and_idct_block_1x1(coefficients: &[i16], quantization_table: &[u16; 64], _output_linestride: usize, output: &mut [u8]) {
debug_assert_eq!(coefficients.len(), 64);

let s0 = (coefficients[0] as i32 * quantization_table[0] as i32).wrapping_add(128 * 8) / 8;
output[0] = stbi_clamp(s0);
}

// This is based on stb_image's 'stbi__idct_block'.
pub fn dequantize_and_idct_block(coefficients: &[i16], quantization_table: &[u16; 64], output_linestride: usize, output: &mut [u8]) {
pub fn dequantize_and_idct_block_8x8(coefficients: &[i16], quantization_table: &[u16; 64], output_linestride: usize, output: &mut [u8]) {
debug_assert_eq!(coefficients.len(), 64);

let mut temp = [0i32; 64];
Expand Down
11 changes: 8 additions & 3 deletions src/parser.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ pub struct FrameInfo {
pub precision: u8,

pub image_size: Dimensions,
pub output_size: Dimensions,
pub mcu_size: Dimensions,
pub components: Vec<Component>,
}
Expand All @@ -58,6 +59,8 @@ pub struct Component {

pub quantization_table_index: usize,

pub dct_scale: usize,

pub size: Dimensions,
pub block_size: Dimensions,
}
Expand Down Expand Up @@ -104,7 +107,7 @@ fn skip_bytes<R: Read>(reader: &mut R, length: usize) -> Result<()> {
}

// Section B.2.2
pub fn parse_sof<R: Read>(reader: &mut R, marker: Marker) -> Result<FrameInfo> {
pub fn parse_sof<R: Read>(reader: &mut R, marker: Marker, scale: usize) -> Result<FrameInfo> {
let length = read_length(reader, marker)?;

if length <= 6 {
Expand Down Expand Up @@ -201,6 +204,7 @@ pub fn parse_sof<R: Read>(reader: &mut R, marker: Marker) -> Result<FrameInfo> {
horizontal_sampling_factor: horizontal_sampling_factor,
vertical_sampling_factor: vertical_sampling_factor,
quantization_table_index: quantization_table_index as usize,
dct_scale: scale,
size: Dimensions {width: 0, height: 0},
block_size: Dimensions {width: 0, height: 0},
});
Expand All @@ -214,8 +218,8 @@ pub fn parse_sof<R: Read>(reader: &mut R, marker: Marker) -> Result<FrameInfo> {
};

for component in &mut components {
component.size.width = (width as f32 * (component.horizontal_sampling_factor as f32 / h_max as f32)).ceil() as u16;
component.size.height = (height as f32 * (component.vertical_sampling_factor as f32 / v_max as f32)).ceil() as u16;
component.size.width = (width as f32 * component.horizontal_sampling_factor as f32 * component.dct_scale as f32 / (h_max as f32 * 8.0)).ceil() as u16;
component.size.height = (height as f32 * component.vertical_sampling_factor as f32 * component.dct_scale as f32 / (v_max as f32 * 8.0)).ceil() as u16;

component.block_size.width = mcu_size.width * component.horizontal_sampling_factor as u16;
component.block_size.height = mcu_size.height * component.vertical_sampling_factor as u16;
Expand All @@ -228,6 +232,7 @@ pub fn parse_sof<R: Read>(reader: &mut R, marker: Marker) -> Result<FrameInfo> {
entropy_coding: entropy_coding,
precision: precision,
image_size: Dimensions {width: width, height: height},
output_size: Dimensions {width: (width as f32 * scale as f32 / 8.0).ceil() as u16, height: (height as f32 * scale as f32 / 8.0).ceil() as u16},
mcu_size: mcu_size,
components: components,
})
Expand Down
2 changes: 1 addition & 1 deletion src/upsampler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ impl Upsampler {
upsampler: upsampler,
width: component.size.width as usize,
height: component.size.height as usize,
row_stride: component.block_size.width as usize * 8,
row_stride: component.block_size.width as usize * component.dct_scale,
});
}

Expand Down
25 changes: 15 additions & 10 deletions src/worker/immediate.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use decoder::MAX_COMPONENTS;
use error::Result;
use idct::dequantize_and_idct_block;
use idct::{ dequantize_and_idct_block_8x8, dequantize_and_idct_block_1x1 };
use std::mem;
use std::sync::Arc;
use parser::Component;
Expand All @@ -26,7 +26,7 @@ impl ImmediateWorker {
assert!(self.results[data.index].is_empty());

self.offsets[data.index] = 0;
self.results[data.index].resize(data.component.block_size.width as usize * data.component.block_size.height as usize * 64, 0u8);
self.results[data.index].resize(data.component.block_size.width as usize * data.component.block_size.height as usize * data.component.dct_scale * data.component.dct_scale, 0u8);
self.components[data.index] = Some(data.component);
self.quantization_tables[data.index] = Some(data.quantization_table);
}
Expand All @@ -36,20 +36,25 @@ impl ImmediateWorker {
let component = self.components[index].as_ref().unwrap();
let quantization_table = self.quantization_tables[index].as_ref().unwrap();
let block_count = component.block_size.width as usize * component.vertical_sampling_factor as usize;
let line_stride = component.block_size.width as usize * 8;
let line_stride = component.block_size.width as usize * component.dct_scale;

assert_eq!(data.len(), block_count * 64);

for i in 0..block_count {
let x = (i % component.block_size.width as usize) * 8;
let y = (i / component.block_size.width as usize) * 8;
dequantize_and_idct_block(&data[i * 64..(i + 1) * 64],
quantization_table,
line_stride,
&mut self.results[index][self.offsets[index] + y * line_stride + x..]);
let x = (i % component.block_size.width as usize) * component.dct_scale;
let y = (i / component.block_size.width as usize) * component.dct_scale;

let coefficients = &data[i * 64..(i + 1) * 64];
let output = &mut self.results[index][self.offsets[index] + y * line_stride + x..];

match component.dct_scale {
8 => dequantize_and_idct_block_8x8(coefficients, quantization_table, line_stride, output),
1 => dequantize_and_idct_block_1x1(coefficients, quantization_table, line_stride, output),
_ => unimplemented!(),
}
}

self.offsets[index] += data.len();
self.offsets[index] += block_count * component.dct_scale * component.dct_scale;
}
pub fn get_result_immediate(&mut self, index: usize) -> Vec<u8> {
mem::replace(&mut self.results[index], Vec::new())
Expand Down