Added README Automation (#30)

* Abstracted the Wiki into the generator

* Documented the new generator class

* Separator files out into responsibilities

* Reworked output page method

* Refactoring some of the repo code

* Doing a bit of refactoring

* Clean and polished the repo file

* Added documentation to wiki code

* Working through documentation

* Slowly adding documentation

* Added readme path to the language collection

* Added constructor documentation

* Adding yet more documentation

* Updated test file link code

* Beginning to toy with readme generation

* Back to documenting code

* Documented and reworked wiki code

* Refactoring code for convenience

* Working through README code

* Made repo module read-only

* Moved source directory code to repo

* Created a README generator prototype

* Successfully tested previous functionality

* Generated my first READMEs 👍

* Generated all of the readmes!

* Fixed up some bugs

* Implemented URL generation for projects

* Added URL generator

* Added doc links

* Added issue query to missing articles

* Tested Issue Linking

* Updated README format

* Experimenting with updating repo directly

* Fixed an issue with program lists

* Not sure how this got here!

* Refactored the generator code

* Added new command to setup.py

* Updated version number

* Working through the documentation

* Cleaned up README

* Deleted Travis CI

Author: jrg94

.gitignore

@@ -107,4 +107,5 @@ venv.bak/
 .idea/
 
 # Wiki
-wiki/
+/generate_docs/wiki/
+/generate_docs/readme/

.travis.yml

@@ -1,34 +1,38 @@
-# The Sample Programs Wiki Generator
+# The Sample Programs Docs Generator
 
-Currently, the Sample Programs Wiki Generator repo houses the `generate-wiki.py` script which
-we use in the Sample Programs repository to generate our wiki documentation. The script
-is automated by a Travis CI build in the Sample Programs repo.
+Previously known as the Sample Programs Wiki Generator, the Sample Programs Docs Generator repo houses
+the `generator.py` script which we use in the Sample Programs repository to generate our documentation. The script is
+automated by a GitHub Actions build in the Sample Programs repo.
 
 If you would like to propose a change, feel free to leverage the issues tab or make a pull request.
 
 ## How It Works
 
-The `generate_wiki.py` script works by analyzing the information in the Sample Programs
-repository and storing that information in objects. These objects are then used to
-generate various wiki pages in Markdown. 
+The `generator.py` script works by analyzing the information in the Sample Programs repository and storing that
+information in objects. These objects are then used to generate various documentation pages in Markdown.
 
 ## What is Automated
 
-Currently, the script generates 27 pages: 1 alphabet catalog and 26 alphabet pages. 
+Currently, the script does two things:
 
-The alphabet catalog contains a table with links to each alphabet page as well as meta data like the number
-of scripts and languages per letter. In addition, the alphabet catalog contains the total number of scripts
-and languages for the entire repo. 
+- It maintains our entire [Sample Programs wiki](https://github.com/TheRenegadeCoder/sample-programs/wiki)
+- It maintains all of our READMEs in the Sample Programs repo
 
-Each alphabet page contains a table which lists each language for that particular letter as well as meta data
-like the number of scripts per language and links to articles and issues.
+In terms of wiki automation, it generates 27 pages: 1 alphabet catalog and 26 alphabet pages.
+
+The alphabet catalog contains a table with links to each alphabet page as well as metadata like the number of scripts
+and languages per letter. In addition, the alphabet catalog contains the total number of scripts and languages for the
+entire repo.
+
+Each alphabet page contains a table which lists each language for that particular letter as well as metadata like the
+number of scripts per language as well as linking to articles and issues.
 
 ## How to Run
 
-At this time, the wiki generator is hardcoded for the Sample Programs repo. In order to run it, 
-you can install it as a package using `pip`:
+At this time, the wiki generator is hardcoded for the Sample Programs repo. In order to run it, you can install it as a
+package using `pip`:
 
-`pip install generate_wiki`
+`pip install generate_docs`
 
 After that, you'll need a copy of the Sample Programs repo:
 
@@ -38,12 +42,16 @@ Finally, you can build the wiki using the following command:
 
 `wikig /path/to/sample-programs/repo/archive`
 
+Likewise, you can build the READMEs using the following command:
+
+`wikir /path/to/sample-programs/repo/archive`
+
 Alternatively, you can clone this repo to run the `generate_wiki.py` script directly:
 
-`python generate-wiki.py /path/to/sample-programs/repo/archive`
+`python generator.py /path/to/sample-programs/repo/archive`
 
-Both solutions are designed to handle repo exploration from the `/archive/` directory. If successful, you should
-begin to see print statements for the various links under test for The Renegade Coder. When finished, you'll
-have a `/wiki/` directory next to your script which contains the wiki.
+Both solutions are designed to handle repo exploration from the `/archive/` directory. If successful, you should begin
+to see print statements for the various links under test for The Renegade Coder. When finished, you'll have a `/wiki/`
+directory next to your script which contains the wiki. Likewise, all the READMEs should be updated.
 
 At this point, you can push the wiki directly to the Sample Programs wiki.

README.md

@@ -0,0 +1 @@
+name = "generate_docs"

generate_docs/__init__.py

@@ -0,0 +1,66 @@
+import sys
+from typing import Optional
+
+from generate_docs.readme import ReadMeCatalog
+from generate_docs.repo import Repo
+from generate_docs.wiki import Wiki
+
+
+class Generator:
+    """
+    The top-level class used to generate wiki and README objects.
+    """
+    def __init__(self, source_dir):
+        self.source_dir = source_dir
+        self.repo: Repo = Repo(self.source_dir)
+
+    def generate_wiki(self) -> None:
+        """
+        Builds and outputs the wiki.
+        :return: None
+        """
+        wiki = Wiki(self.repo)
+        for page in wiki.pages:
+            page.output_page("wiki")
+
+    def generate_readmes(self) -> None:
+        """
+        Builds and outputs the READMEs.
+        :return:
+        """
+        readme_catalog = ReadMeCatalog(self.repo)
+        for language, page in readme_catalog.pages.items():
+            page.output_page(f"{self.source_dir}/{language[0]}/{language}")
+
+
+def _create_generator() -> Optional[Generator]:
+    """
+    Creates the generator object from
+    :return:
+    """
+    if len(sys.argv) > 1:
+        generator = Generator(sys.argv[1])
+        return generator
+    else:
+        print("Please supply an input path")
+        exit(1)
+
+
+def main_wiki():
+    """
+    Builds the wiki.
+    :return: None
+    """
+    _create_generator().generate_wiki()
+
+
+def main_readmes():
+    """
+    Builds the READMEs.
+    :return: None
+    """
+    _create_generator().generate_readmes()
+
+
+if __name__ == '__main__':
+    main_wiki()

generate_docs/generator.py

@@ -0,0 +1,107 @@
+import os
+import pathlib
+from urllib.error import HTTPError
+from urllib import request
+
+from generate_docs.repo import LanguageCollection, SampleProgram
+
+
+def create_md_link(text: str, url: str) -> str:
+    """
+    Generates a markdown link in the form [text](url).
+    :param text: the link text
+    :param url: the url to link
+    :return: a markdown link
+    """
+    separator = ""
+    return separator.join(["[", text, "]", "(", url, ")"])
+
+
+def build_language_link(language: LanguageCollection) -> str:
+    """
+    A handy abstraction for the create_md_link() method which creates a link to a sample programs language page.
+    (e.g., https://sample-programs.therenegadecoder.com/languages/c/)
+    :param language: the language to link
+    :return: a markdown link to the language page if it exists; an empty string otherwise
+    """
+    if not verify_link(language.sample_program_url):
+        markdown_url = ""
+    else:
+        markdown_url = create_md_link("Here", language.sample_program_url)
+    return markdown_url
+
+
+def build_doc_link(sample_program: SampleProgram, text: str) -> str:
+    """
+    A handy abstraction for the create_md_link() method which creates a link to a sample programs docs page.
+    (e.g., https://sample-programs.therenegadecoder.com/projects/even-odd/c/)
+    :param sample_program: the sample program
+    :param text: the label for the link
+    :return: a markdown link to the docs page if it exists; an empty string otherwise
+    """
+    if not verify_link(sample_program.sample_program_doc_url):
+        markdown_url = f":warning: {create_md_link(text, sample_program.sample_program_issue_url)}"
+    else:
+        markdown_url = f":white_check_mark: {create_md_link(text, sample_program.sample_program_doc_url)}"
+    return markdown_url
+
+
+def verify_link(url: str) -> bool:
+    """
+    Verifies that a URL is a valid URL.
+    :param url: a website URL
+    :return: True if the URL is valid; False otherwise
+    """
+    req = request.Request(url)
+    req.get_method = lambda: 'HEAD'
+    print(f"Trying: {url}")
+    try:
+        request.urlopen(req)
+        print(f"\tVALID")
+        return True
+    except HTTPError:
+        print(f"\tINVALID")
+        return False
+
+
+class MarkdownPage:
+    def __init__(self, name: str):
+        self.name: str = name
+        self.ext = ".md"
+        self.wiki_url_base: str = "/jrg94/sample-programs/wiki/"
+        self.content = list()
+
+    def __str__(self):
+        return f"{self.name}\n{self._build_page()}"
+
+    def _build_page(self):
+        return "\n".join(self.content)
+
+    def add_content(self, *lines: str):
+        self.content.extend(lines)
+
+    def add_table_header(self, *args):
+        column_separator = " | "
+        header = column_separator.join(args)
+        divider = column_separator.join(["-----"] * len(args))
+        self.content.append(header)
+        self.content.append(divider)
+
+    def add_table_row(self, *args):
+        column_separator = " | "
+        row = column_separator.join(args)
+        self.content.append(row)
+
+    def add_section_break(self):
+        self.content.append("")
+
+    def output_page(self, dump_dir):
+        pathlib.Path(dump_dir).mkdir(parents=True, exist_ok=True)
+        output_file = open(os.path.join(dump_dir, self._get_file_name()), "w+")
+        output_file.write(self._build_page())
+        output_file.close()
+
+    def _get_file_name(self):
+        separator = "-"
+        file_name = f"{separator.join(self.name.split())}{self.ext}"
+        return file_name

generate_docs/markdown.py

@@ -0,0 +1,73 @@
+from generate_docs.markdown import MarkdownPage, build_doc_link, verify_link
+from generate_docs.repo import Repo, LanguageCollection
+
+
+def _get_intro_text(language: LanguageCollection) -> str:
+    introduction = f"Welcome to Sample Programs in {language.name.capitalize()}!"
+    docs = f"""To find documentation related to the {language.name.capitalize()} 
+    code in this repo, look [here]({language.sample_program_url}).
+    """
+    interlude_valid = "Otherwise, below you'll find a list of code snippets in this collection."
+    interlude_invalid = "Below, you'll find a list of code snippets in this collection."
+    emojis = """
+    Code snippets preceded by :warning: link to a GitHub 
+    issue query featuring a possible article request issue. If an article request issue 
+    doesn't exist, we encourage you to create one. Meanwhile, code snippets preceded 
+    by :white_check_mark: link to an existing article which provides further documentation.
+    """
+    if not verify_link(language.sample_program_url):
+        return " ".join([introduction, interlude_invalid, emojis])
+    else:
+        return " ".join([introduction, docs, interlude_valid, emojis])
+
+
+def _generate_program_list(language: LanguageCollection) -> list:
+    """
+    A helper function which generates a list of programs for the README.
+    :param language: a language collection
+    :return: a list of sample programs list items
+    """
+    list_items = list()
+    for program in language.sample_programs:
+        readable_name = program.normalized_name.replace("-", " ").title()
+        doc_link = build_doc_link(program, f"{readable_name} in {program.language.capitalize()}")
+        list_items.append(f"- {doc_link}")
+    return list_items
+
+
+class ReadMeCatalog:
+    """
+    An representation of the collection of READMEs in the Sample Programs repo.
+    """
+
+    def __init__(self, repo: Repo):
+        """
+        Constructs an instance of a ReadMeCatalog.
+        :param repo: a repository instance
+        """
+        self.repo: Repo = repo
+        self.pages: dict[str, MarkdownPage] = dict()
+        self._build_readmes()
+
+    def _build_readme(self, language: LanguageCollection) -> None:
+        """
+        Creates a README page from a language collection.
+        :param language: a programming language collection (e.g., Python)
+        :return: None
+        """
+        page = MarkdownPage("README")
+        page.add_content(f"# Sample Programs in {language.name.capitalize()}")
+        page.add_section_break()
+        page.add_content(_get_intro_text(language))
+        page.add_section_break()
+        page.add_content(*_generate_program_list(language))
+        page.add_section_break()
+        self.pages[language.name] = page
+
+    def _build_readmes(self) -> None:
+        """
+        Generates all READMEs for the repo.
+        :return: None
+        """
+        for language in self.repo.languages:
+            self._build_readme(language)