Merge branch 'gh-pages' into 181-style-polishes

Author: tobyhodges

episodes/01-introduction.md

@@ -18,12 +18,48 @@ image analysis problems."
 objects in an image."
 ---
 
-We can use relatively simple image processing and computer vision techniques in
-Python, using [the skimage library](https://scikit-image.org/).
-With careful experimental design, a digital camera or a flatbed scanner,
-in conjunction with some Python code,
-can be a powerful instrument in answering many different kinds of problems.
-Consider the following problem that might be of interest to a scientist.
+
+As computer systems have become faster and more powerful,
+and cameras and other imaging systems have become commonplace
+in many other areas of life,
+the need has grown for researchers to be able to
+process and analyse image data.
+Considering the large volumes of data that can be involved -
+high-resolution images that take up a lot of disk space/virtual memory,
+and/or collections of many images that must be processed together -
+and the time-consuming and error-prone nature of manual processing,
+it can be advantageous or even necessary for this processing and analysis
+to be automated as a computer program.
+
+This lesson introduces an open source toolkit for processing image data:
+the Python programming language
+and [the _scikit-image_ (`skimage`) library](https://scikit-image.org/).
+With careful experimental design, 
+Python code can be a powerful instrument in answering many different kinds of questions.
+
+
+## Uses of Image Processing in Research
+
+Automated processing can be used to analyse many different properties of an image,
+including the distribution and change in colours in the image,
+the number, size, position, orientation, and shape of objects in the image,
+and even - when combined with machine learning techniques for object recognition -
+the type of objects in the image.
+
+Some examples of image processing methods applied in research include:
+
+- [imaging a Black Hole](https://iopscience.iop.org/article/10.3847/2041-8213/ab0e85)
+- [estimating the population of Emperor Penguins](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3325796/)
+- [the global-scale analysis of marine plankton diversity](https://www.cell.com/cell/fulltext/S0092-8674(19)31124-9)
+- [segmentation of liver and vessels from CT images](https://doi.org/10.1016/j.cmpb.2017.12.008)
+
+With this lesson,
+we aim to provide a thorough grounding in the fundamental concepts and skills
+of working with image data in Python.
+Most of the examples used in this lesson focus on
+one particular class of image processing technique, _morphometrics_,
+but what you will learn can be used to solve a much wider range of problems.
+
 
 ## Morphometrics
 
@@ -72,10 +108,12 @@ resulting in an image like this:
 > research.
 {: .callout}
 
-As we move through this workshop,
-we will learn image analysis methods useful for many different scientific problems.
-These will be linked together and applied to a real problem in
-the final end-of-workshop [capstone challenge]({{page.root}}{% link _episodes/09-challenges.md %}).
+
+As we move through this workshop, 
+we will learn image analysis methods useful for many different scientific problems. 
+These will be linked together
+and applied to a real problem in the final end-of-workshop
+[capstone challenge]({{ page.root }}{% link _episodes/09-challenges.md %}).
 
 Let's get started,
 by learning some basics about how images are represented and stored digitally.

episodes/02-image-basics.md

@@ -958,5 +958,6 @@ image formats:
 |          |               |            | high quality          |                    |
 | JPEG     | Lossy         | Yes        | Universally viewable, | Detail may be lost |
 |          |               |            | smaller file size     |                    |
+| PNG      | Lossless      | [Yes](https://www.w3.org/TR/PNG/#11keywords) | Universally viewable, [open standard](https://www.w3.org/TR/PNG/), smaller file size | Metadata less flexible than TIFF, RGB only |
 | TIFF     | None, lossy,  | Yes        | High quality or       | Not universally viewable   |
 |          | or lossless   |            | smaller file size     |                    |

episodes/06-blurring.md

@@ -39,6 +39,7 @@ A blur is a very common operation we need to perform before other tasks such as
 There are several different blurring functions in the `skimage.filters` module,
 so we will focus on just one here, the *Gaussian blur*.
 
+
 > ## Filters
 >
 > In the day-to-day, macroscopic world,
@@ -285,7 +286,14 @@ plt.show()
 
 > ## Experimenting with sigma values (10 min)
 >
-> Try running the code above with a range of smaller and larger sigma values.
+> The size and shape of the kernel used to blur an image can have a
+> significant effect on the result of the blurring and any downstream analysis
+> carried out on the blurred image.
+> The next two exercises ask you to experiment with the sigma values of the kernel,
+> which is a good way to develop your understanding of how the choice of kernel
+> can influence the result of blurring.
+>
+> First, try running the code above with a range of smaller and larger sigma values.
 > Generally speaking, what effect does the sigma value have on the
 > blurred image?
 >
@@ -302,9 +310,9 @@ plt.show()
 
 > ## Experimenting with kernel shape (10 min - optional, not included in timing)
 >
-> Now, try running the code above with different sigmas in the y and x direction.
+> Now, what is the effect of applying an asymmetric kernel to blurring an image?
+> Try running the code above with different sigmas in the y and x direction.
 > For example, a sigma of 1.0 in the y direction, and 6.0 in the x direction.
-> What is the effect on the blurred image?
 >
 > > ## Solution
 > >
@@ -320,9 +328,20 @@ plt.show()
 > > plt.show()
 > > ~~~
 > > {: .language-python}
+> >
 > > ![Rectangular kernel blurred image](../fig/rectangle-gaussian-blurred.png)
-> > This produces a kernel that is rectangular instead of square. Notice that the image
-> > is much more blurred in the x direction than the y direction.
+> >
+> > These unequal sigma values produce a kernel that is rectangular instead of square.
+> > The result is an image that is much more blurred in the x direction than the
+> > y direction.
+> > For most use cases, a uniform blurring effect is desirable and
+> > this kind of asymmetric blurring should be avoided.
+> > However, it can be helpful in specific circumstances e.g. when noise is present in
+> > your image in a particular pattern or orientation, such as vertical lines,
+> > or when you want to 
+> > [remove uniform noise without blurring edges present in the image in a particular orientation](https://www.researchgate.net/publication/228567435_An_edge_detection_algorithm_based_on_rectangular_Gaussian_kernels_for_machine_vision_applications).
+> >
+> >
 > {: .solution}
 {: .challenge}
 

episodes/07-thresholding.md

@@ -151,7 +151,10 @@ Here, we want to turn "on" all pixels which have values smaller than the thresho
 so we use the less operator `<` to compare the `blurred_image` to the threshold `t`.
 The operator returns a mask, that we capture in the variable `binary_mask`.
 It has only one channel, and each of its values is either 0 or 1.
-The binary mask created by the thresholding operation can be shown with `plt.imshow`.
+The binary mask created by the thresholding operation can be shown with `plt.imshow`,
+where the `False` entries are shown as black pixels
+(0-valued) and the `True` entries are shown as white pixels
+(1-valued).
 
 ~~~
 # create a mask based on the threshold
@@ -200,8 +203,8 @@ What we are left with is only the coloured shapes from the original.
 
 ~~~
 # use the binary_mask to select the "interesting" part of the image
-selection = np.zeros_like(image)
-selection[binary_mask] = image[binary_mask]
+selection = image.copy()
+selection[~binary_mask] = 0
 
 fig, ax = plt.subplots()
 plt.imshow(selection)
@@ -275,8 +278,8 @@ plt.show()
 > > And here are the commands to apply the mask and view the thresholded image
 > > ~~~
 > > image = skimage.io.imread("data/shapes-02.jpg")
-> > selection = np.zeros_like(image)
-> > selection[binary_mask] = image[binary_mask]
+> > selection = image.copy()
+> > selection[~binary_mask] = 0
 > >
 > > fig, ax = plt.subplots()
 > > plt.imshow(selection)
@@ -393,8 +396,8 @@ Finally, we use the mask to select the foreground:
 
 ~~~
 # apply the binary mask to select the foreground
-selection = np.zeros_like(image)
-selection[binary_mask] = image[binary_mask]
+selection = image.copy()
+selection[~binary_mask] = 0
 
 fig, ax = plt.subplots()
 plt.imshow(selection)
@@ -625,10 +628,10 @@ data/trial-293.jpg,0.13607895611702128
 > >     # blur before thresholding
 > >     blurred_image = skimage.filters.gaussian(image, sigma=sigma)
 > >
-> >     # perform inverse binary thresholding to mask the white label and circle
-> >     binary_mask = blurred_image > 0.95
+> >     # perform binary thresholding to mask the white label and circle
+> >     binary_mask = blurred_image < 0.95
 > >     # use the mask to remove the circle and label from the blurred image
-> >     blurred_image[binary_mask] = 0
+> >     blurred_image[~binary_mask] = 0
 > >
 > >     # perform automatic thresholding to produce a binary image
 > >     t = skimage.filters.threshold_otsu(blurred_image)

episodes/08-connected-components.md

@@ -1,7 +1,7 @@
 ---
 title: "Connected Component Analysis"
 teaching: 70
-exercises: 45
+exercises: 55
 questions:
 - "How to extract separate objects from an image and describe these objects quantitatively."
 objectives:
@@ -709,7 +709,7 @@ This will produce the output
 > {: .solution}
 {: .challenge}
 
-> ## Colour objects by area (10 min)
+> ## Colour objects by area (optional, not included in timing)
 >
 > Finally, we would like to display the image with the objects coloured
 > according to the magnitude of their area.
@@ -742,5 +742,22 @@ This will produce the output
 > > {: .language-python}
 > >
 > > ![Objects colored by area](../fig/shapes-01-objects-coloured-by-area.png)
+> >
+> > > You may have noticed that in the solution, we have used the
+> > > `labeled_image` to index the array `object_areas`. This is an
+> > > example of [advanced indexing in
+> > > Numpy](https://numpy.org/doc/stable/user/basics.indexing.html#advanced-indexing)
+> > > The result is an array of the same shape as the `labeled_image`
+> > > whose pixel values are selected from `object_areas` according to
+> > > the object label. Hence the objects will be colored by area when
+> > > the result is displayed. Note that advanced indexing with an
+> > > integer array works slightly different than the indexing with a
+> > > Boolean array that we have used for masking. While Boolean array
+> > > indexing returns only the entries corresponding to the `True`
+> > > values of the index, integer array indexing returns an array
+> > > with the same shape as the index. You can read more about advanced
+> > > indexing in the [Numpy
+> > > documentation](https://numpy.org/doc/stable/user/basics.indexing.html#advanced-indexing).
+> > {: .callout}
 > {: .solution}
 {: .challenge}

fig/jupyter_overview.png

@@ -23,3 +23,4 @@ This lesson shows how to use Python and skimage to do basic image processing.
 > to be familiar with.
 {: .prereq}
 
+Before following the lesson, please [make sure you have the software and data required]({{ page.root }}{% link setup.md %}).

index.md

@@ -4,7 +4,22 @@ title: Setup
 permalink: /setup/
 ---
 
-## Setup instructions for the Image Processing workshop
+Before joining the workshop or following the lesson, please complete the data and software setup described in this page.
+
+
+## Data
+
+The example images used in this lesson are available on [FigShare](https://figshare.com/).
+To download the data, please visit [the dataset page for this workshop][figshare-data]
+and click the "Download all" button.
+Unzip the downloaded file, and save the contents as a folder  called `data` somewhere you will easily find it again,
+e.g. your Desktop or a folder you have created for using in this workshop.
+(The name `data` is optional but recommended, as this is the name we will use to refer to the folder throughout the lesson.)
+
+[figshare-data]: https://figshare.com/articles/dataset/Data_Carpentry_Image_Processing_Data_beta_/19260677
+
+
+## Software
 
 1. Download and install the latest [Anaconda
    distribution](https://www.anaconda.com/distribution/) for your
@@ -41,15 +56,32 @@ permalink: /setup/
    > package.
    {: .callout}
 
-   To test your environment, open a Jupyter notebook and copy the following lines into a cell:
+3. Open a Jupyter notebook:
+
+   > ## Instructions for Linux & Mac
+   >
+   > Open a terminal and type `jupyter lab`.
+   {: .solution }
+
+   > ## Instructions for Windows
+   >
+   > Launch the Anaconda Prompt program and type `jupyter lab`.
+   > (Running this command on the standard Command Prompt will return an error:
+   > `'conda' is not recognized as an internal or external command, operable program or batch file.`)
+   {: .solution }
+
+   After Jupyter Lab has launched, click the "Python 3" button under "Notebook" in the launcher window,
+   or use the "File" menu, to open a new Python 3 notebook.
+
+4. To test your environment, run the following lines in a cell of the notebook:
    ~~~
    import skimage.io
    import matplotlib.pyplot as plt
    %matplotlib widget
-   
+
    # load an image
-   image = skimage.io.imread(fname='fig/00-colonies01.jpg')
-   
+   image = skimage.io.imread(fname='data/colonies-01.tif')
+
    # display the image
    fig, ax = plt.subplots()
    plt.imshow(image, cmap='gray')
@@ -58,8 +90,15 @@ permalink: /setup/
    {: .language-python}
    Upon execution of the cell, an image should be displayed in an interactive widget. When hovering over the image with the mouse pointer, the pixel coordinates and color values are displayed below the image.
 
-3. The example image files are available through Figshare. Learners
-   can download the images from [FIXME
-   figshare](https://figshare.com/). We recommend to create a
-   directory for the Jupyter notebooks/code created during the
-   lesson. The images should be located in a subfolder named `images/`.
+   > ## Running Cells in a Notebook
+   >
+   >
+   > ![Overview of the Jupyter Notebook graphical user interface](../fig/jupyter_overview.png)
+   > To run Python code in a Jupyter notebook cell, click on a cell in the notebook
+   > (or add a new one by clicking the `+` button in the toolbar),
+   > make sure that the cell type is set to "Code" (check the dropdown in the toolbar),
+   > and add the Python code in that cell.
+   > After you have added the code,
+   > you can run the cell by selecting "Run" -> "Run selected cell" in the top menu,
+   > or pressing <kbd>Shift</kbd>+<kbd>Enter</kbd>.
+   {: .solution }