Support fixing adjacent inline references (pre-sort) (hellt#10)

* Add support for separating adjacent footnote references with a space

* Simplify 'adjacent' argument definition and improve conditional logic

* Fix unexpected pattern replace when the same inline ref appears twice

* Modify test data for adjacent inline refs

* Modify adjacent spacing function name and move function call to main

* Add command line argument documentation to README.md

* Rename for accuracy (space_adjacent_footnotes to space_adjacent_references)

* Remove commented test function as it isn't needed

* Update CONTRIBUTING.md -  replace emoji with text version

Replace unicode emoji with the text version so the markdown is plain text. `:tada:`

* Update CONTRIBUTING.md - missed the bulb emoji

Missed the bulb emoji in the last commit. Plaintext for the win.

* Remove extra argument from sort_footnotes in main

* Remove named argument (as there's a single arg)

* Add adjacent inline refs to test data

Author: mjbear

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-🎉 Thank you for your interest in contributing to this project! 🎉
+:tada: Thank you for your interest in contributing to this project! :tada:
 
 A couple of quick items:
 
@@ -55,4 +55,4 @@ Choose one of the below testing commands methods that suits your needs.
 1. Discover unit tests in specific directory (in this case, `tests`) with verbosity
     * `python -m unittest discover -s tests -v`
 
-💡 For additional `unittest` command line options, refer to the [official Python unittest documentation](https://docs.python.org/3/library/unittest.html#command-line-interface).
+:bulb: For additional `unittest` command line options, refer to the [official Python unittest documentation](https://docs.python.org/3/library/unittest.html#command-line-interface).

README.md

@@ -27,8 +27,16 @@ Or download the script and put it in your `$PATH`:
 
 ```bash
 curl -sL https://raw.githubusercontent.com/hellt/markdown-footnote-sorter/main/fnsort.py
+
+fnsort.py path/to/doc.md
 ```
 
+## Command Line Arguments
+### --adjacent
+Adjacent inline references that are not separated by other characters become problematic (by default).
+This option adds spacing between those inline references so they are properly identified during sorting.
+
+`fnsort.py path/to/doc.md --adjacent`
 
 ## Contributing
 For information about contributing to this project, see the [contributing guidelines](CONTRIBUTING.md).

fnsort.py

@@ -28,6 +28,13 @@ def parse_arguments():
         default="-",
         help="Input file to process (use - for stdin)",
     )
+
+    parser.add_argument(
+        "--adjacent",
+        action="store_true",
+        help="Fix adjacent footnotes by adding a space between them",
+    )
+    
     return parser.parse_args()
 
 
@@ -48,6 +55,32 @@ def replace_reference(m, order):
     return f"{m.group(0)[:1]}[^{order.index(m.group(2)) + 1}]"
 
 
+def space_adjacent_references(text):
+    # add space between two inline footnotes as long as a space isn't present
+    #   ex: [^1][^2] becomes [^1] [^2]
+
+    # inline refs that do NOT begin with white space
+    inline_note = re.compile(r"([^\s]\[\^\w+\])")
+
+    notes = inline_note.findall(text)
+
+    for note in notes:
+        # matches cannot be at the beginning of a line
+        note_re = r"(?<!^)" + re.escape(note)
+
+        # slice regex to remove the negative look behind and
+        #   pattern replace backslash escape chars
+        repl = note_re[6:].replace("\\", "")
+
+        # print(f"\nFindall: {re.findall(note_re, text, flags=re.MULTILINE)}\n")
+
+        # slice the string to separate preceding character from inline reference
+        #   ex: s[^4]
+        text = re.sub(note_re, f"{repl[0]} {repl[1:]}", text, flags=re.MULTILINE)
+    
+    return text
+
+
 def sort_footnotes(text):
     # removes the last newline from EOF so there is no EOL on the last line
     text = text.rstrip()
@@ -84,6 +117,10 @@ def main():
     args = parse_arguments()
     with open(args.file, "r+") as file:
         text = file.read()
+
+        if args.adjacent:
+            text = space_adjacent_references(text)
+
         processed_text = sort_footnotes(text)
         file.seek(0)
         file.write(processed_text)

tests/adjacent/adjacent.md

@@ -0,0 +1,31 @@
+# Wrens
+
+## Overview
+Wrens [^4][^7] are a family of brown birds of which there are 88 species.
+
+## Size
+Wrens are medium-small to very small birds. The Eurasian wren is among the smallest birds in its range, while the smaller species from the Americas are among the smallest passerines in that part of the world.[^1]
+
+## Coloration
+The dominating colors of their plumage are generally drab, composed of gray, brown, black, and white, and most species show some barring, especially on the tail or wings[^1].
+
+## Diet
+Wrens are primarily insectivorous, eating insects, spiders and other small invertebrates, but many species also eat vegetable matter and some eat small frogs and lizards[^4][^7].
+
+## Species
+### Eurasian Wren
+![Eurasian wren](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6c/Ein_neugieriger_Zaunk%C3%B6nig.jpg/240px-Ein_neugieriger_Zaunk%C3%B6nig.jpg)[^3]
+
+The Eurasian wren is the only species of wren found outside the Americas, as restricted to Europe, Asia, and northern Africa.[^2]
+
+The Eurasian wren has been recorded wading into shallow water to catch small fish and tadpoles[^6] .
+
+The most common call of the Eurasian is a sharp repeated "tic-tic-tic"[^5].
+
+[^4]: https://en.wikipedia.org/wiki/Wren
+[^7]: https://www.allaboutbirds.org/guide/browse/shape/Wrens
+[^1]: https://en.wikipedia.org/wiki/Wren#Description
+[^5]: https://en.wikipedia.org/wiki/Eurasian_wren#Vocalizations
+[^2]: https://en.wikipedia.org/wiki/Wren#Distribution_and_habitat
+[^6]: https://en.wikipedia.org/wiki/Wren#Behavior_and_ecology
+[^3]: https://en.wikipedia.org/wiki/File:Ein_neugieriger_Zaunk%C3%B6nig.jpg

tests/adjacent/adjacent_expected.md

@@ -0,0 +1,31 @@
+# Wrens
+
+## Overview
+Wrens [^1] [^2] are a family of brown birds of which there are 88 species.
+
+## Size
+Wrens are medium-small to very small birds. The Eurasian wren is among the smallest birds in its range, while the smaller species from the Americas are among the smallest passerines in that part of the world. [^3]
+
+## Coloration
+The dominating colors of their plumage are generally drab, composed of gray, brown, black, and white, and most species show some barring, especially on the tail or wings [^3].
+
+## Diet
+Wrens are primarily insectivorous, eating insects, spiders and other small invertebrates, but many species also eat vegetable matter and some eat small frogs and lizards [^1] [^2].
+
+## Species
+### Eurasian Wren
+![Eurasian wren](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6c/Ein_neugieriger_Zaunk%C3%B6nig.jpg/240px-Ein_neugieriger_Zaunk%C3%B6nig.jpg) [^4]
+
+The Eurasian wren is the only species of wren found outside the Americas, as restricted to Europe, Asia, and northern Africa. [^5]
+
+The Eurasian wren has been recorded wading into shallow water to catch small fish and tadpoles [^6] .
+
+The most common call of the Eurasian is a sharp repeated "tic-tic-tic" [^7].
+
+[^1]: https://en.wikipedia.org/wiki/Wren
+[^2]: https://www.allaboutbirds.org/guide/browse/shape/Wrens
+[^3]: https://en.wikipedia.org/wiki/Wren#Description
+[^4]: https://en.wikipedia.org/wiki/File:Ein_neugieriger_Zaunk%C3%B6nig.jpg
+[^5]: https://en.wikipedia.org/wiki/Wren#Distribution_and_habitat
+[^6]: https://en.wikipedia.org/wiki/Wren#Behavior_and_ecology
+[^7]: https://en.wikipedia.org/wiki/Eurasian_wren#Vocalizations

tests/adjacent/adjacent_spacing.md

@@ -0,0 +1,31 @@
+# Wrens
+
+## Overview
+Wrens [^4] [^7] are a family of brown birds of which there are 88 species.
+
+## Size
+Wrens are medium-small to very small birds. The Eurasian wren is among the smallest birds in its range, while the smaller species from the Americas are among the smallest passerines in that part of the world. [^1]
+
+## Coloration
+The dominating colors of their plumage are generally drab, composed of gray, brown, black, and white, and most species show some barring, especially on the tail or wings [^1].
+
+## Diet
+Wrens are primarily insectivorous, eating insects, spiders and other small invertebrates, but many species also eat vegetable matter and some eat small frogs and lizards [^4] [^7].
+
+## Species
+### Eurasian Wren
+![Eurasian wren](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6c/Ein_neugieriger_Zaunk%C3%B6nig.jpg/240px-Ein_neugieriger_Zaunk%C3%B6nig.jpg) [^3]
+
+The Eurasian wren is the only species of wren found outside the Americas, as restricted to Europe, Asia, and northern Africa. [^2]
+
+The Eurasian wren has been recorded wading into shallow water to catch small fish and tadpoles [^6] .
+
+The most common call of the Eurasian is a sharp repeated "tic-tic-tic" [^5].
+
+[^4]: https://en.wikipedia.org/wiki/Wren
+[^7]: https://www.allaboutbirds.org/guide/browse/shape/Wrens
+[^1]: https://en.wikipedia.org/wiki/Wren#Description
+[^5]: https://en.wikipedia.org/wiki/Eurasian_wren#Vocalizations
+[^2]: https://en.wikipedia.org/wiki/Wren#Distribution_and_habitat
+[^6]: https://en.wikipedia.org/wiki/Wren#Behavior_and_ecology
+[^3]: https://en.wikipedia.org/wiki/File:Ein_neugieriger_Zaunk%C3%B6nig.jpg

tests/test_footnote_sorting.py

@@ -1,7 +1,14 @@
+import argparse
 import unittest
 
 import fnsort
 
+
+def set_command_line_args(args):
+    """ Set (what would otherwise be) command-line arguments """
+    return argparse.Namespace(**args)
+
+
 class TestDefaults(unittest.TestCase):
     @classmethod
     def setUpClass(self):
@@ -16,6 +23,12 @@ def setUpClass(self):
         with open(f"{path}/example_expected.md") as fh:
             self.expected_text = fh.read()
 
+        # technically there is also a "file" kwarg by default
+        args = {"adjacent": False}
+        self.args = set_command_line_args(args)
+        if self.args.adjacent:
+            self.text = fnsort.space_adjacent_references(self.text)
+
         # allow for full diff output
         # self.maxDiff = None
 
@@ -51,6 +64,12 @@ def setUpClass(self):
         with open(f"{path}/duplicates_expected.md") as fh:
             self.expected_text = fh.read()
 
+        # technically there is also a "file" kwarg by default
+        args = {"adjacent": False}
+        self.args = set_command_line_args(args)
+        if self.args.adjacent:
+            self.text = fnsort.space_adjacent_references(self.text)
+
         # allow for full diff output
         # self.maxDiff = None
 
@@ -90,6 +109,12 @@ def setUpClass(self):
         with open(f"{path}/must_be_last_expected.md") as fh:
             self.expected_text = fh.read()
 
+        # technically there is also a "file" kwarg by default
+        args = {"adjacent": False}
+        self.args = set_command_line_args(args)
+        if self.args.adjacent:
+            self.text = fnsort.space_adjacent_references(self.text)
+
         # allow for full diff output
         # self.maxDiff = None
 
@@ -104,12 +129,50 @@ def test_footnote_sort_trailing_text(self):
           footnote creating a choppy looking list
 
         in short this is not expected to return the desired output
-
-        just so happened to luck out that the last footnote reference of the
-            "duplicates" example was indeed the last footnote :shrug:
         """
         self.assertNotEqual(fnsort.sort_footnotes(self.text), self.expected_text)
 
 
+class TestAdjacentFootnotes(unittest.TestCase):
+    @classmethod
+    def setUpClass(self):
+        # example markdown files intentionally have had trailing EOL trimmed
+        #   from the end of the file (EOF)
+
+        path = "tests/adjacent"
+
+        with open(f"{path}/adjacent.md") as fh:
+            self.text = fh.read()
+
+        with open(f"{path}/adjacent_expected.md") as fh:
+            self.expected_text = fh.read()
+
+        # technically there is also a "file" kwarg by default
+        args = {"adjacent": True}
+        self.args = set_command_line_args(args)
+
+        # allow for full diff output
+        # self.maxDiff = None
+
+
+    def test_adjacent_inline_reference_spacing(self):
+        """ Test spacing out adjacent inline references """
+        with open(f"tests/adjacent/adjacent_spacing.md") as fh:
+            spacing_text = fh.read()
+
+        self.assertEqual(
+            fnsort.space_adjacent_references(self.text),
+            spacing_text
+        )
+
+
+    def test_adjacent_footnote_sort(self):
+        """ Entire footnote sort process with adjacent footnote references """
+        if self.args.adjacent:
+            self.text = fnsort.space_adjacent_references(self.text)
+
+        self.assertEqual(fnsort.sort_footnotes(self.text), self.expected_text)
+
+
 if __name__ == "__main__":
     unittest.main()