disccofan is a C image processing tool that provides utilities for analyzing patterns and structures in 2D and 3D datasets.
Read the documentation here:
https://disccofan.readthedocs.io/
Techniques that disccofan is using were published in these papers:
- Concurrent Computation of Attribute Filters on Shared Memory Parallel Machines
- Distributed Component Forests in 2-D: Hierarchical Image Representations Suitable for Tera-Scale Images
- Distributed Connected Component Filtering and Analysis in 2D and 3D Tera-Scale Data Sets
- Parallel Attribute Computation for Distributed Component Forests
The following pictures present the main scaling results, for running disccofan with and without attribute computation, comparing its performance with the only other available code. The tests are based on 2-D and 3-D remote-sensing and astronomical datasets with sizes from 5 to 160 Gigapixels.
8 bits per pixel - decreasing tile experiment - MPI + threading:
Figure 1: Hybrid performance of DISCCOFAN (with only parent-child computation or with area attribute) and GÖTZ-2D on the 2D 8-bpp quantization of the ESO luminance channel (≈ 9 Gpx). The size of the individual data chunks is decreasing as the number of MPI processes increases. DISCCOFAN has a small memory overhead, but is faster and scales almost linearly up to 64 processes.
floating point - 32 bits per pixel - decreasing tile experiment - MPI + threading:
Figure 2: Hybrid performance of DISCCOFAN (with only parent-child computation or with area attribute) and GÖTZ-2D on the single-precision floating point ESO luminance channel (≈ 9 Gpx). The size of the individual data chunks is decreasing as the number of MPI processes increases.
floating point - 32 bits per pixel - decreasing tile experiment - MPI only:
Figure 3: Speed-up of DISCCOFAN and GÖTZ-2 D on the 2D, floating point, ESO luminance channel (≈ 9 Gpx) when using only MPI processes without threads.
floating point - 32 bits per pixel - increasing tile experiment - 162 Gvoxels - MPI only:
Figure 4: Execution time (left), speed-up (middle) and memory gain (right) on the LOFAR 3D observation, with single precision floating point values. The 3D tile size remains constant (15000 × 15000 × 15 pixels), such that the volume size increases linearly with the number of processes used. With 48 processes, the total volume processed is 162 Gvoxels.
- Simon Gazagnes <sgsgazagnes@gmail.com>
This project is licensed under the MIT License - see the LICENSE file for details.