`scripts/_common.py`: add a shared Python file to move duplicated code (#12755)

* `scripts/_common.py`: add a shared Python file to move duplicated code

---------

Co-authored-by: Vítor Henrique <87824454+vitorhcl@users.noreply.github.com>
Co-authored-by: K.B.Dharun Krishna <kbdharunkrishna@gmail.com>

.github/workflows/ci.yml

@@ -34,7 +34,7 @@ jobs:
       run: npm ci
 
     - name: Install pip dependencies
-      run: pip install -r requirements.txt -r scripts/pdf/requirements.txt
+      run: pip install -r requirements.txt -r scripts/pdf/requirements.txt -r scripts/test-requirements.txt
 
     - name: Test
       run: npm test

.gitignore

@@ -28,3 +28,6 @@ scripts/pdf/tldr-pages.pdf
 # Python venv for testing the PDF script
 # Create it with: python3 -m venv scripts/pdf/venv/
 venv
+
+# Generated pycache
+__pycache__

scripts/README.md

@@ -2,6 +2,9 @@
 
 The current directory contains useful scripts used/to use with `tldr` pages.
 
+> [!NOTE]
+> [Git](https://git-scm.com/) and [Python](https://www.python.org/) must be installed in your system to run/test the scripts locally.
+
 ## Summary
 
 This section contains a summary of the scripts available in this directory. For more information about each script, please refer to the header of each script.
@@ -9,27 +12,27 @@ This section contains a summary of the scripts available in this directory. For
 - [pdf](pdf/README.md) directory contains the `render.py` and `build-pdf.sh` script and related resources to generate a PDF document of tldr-pages for a specific language or platform (or both).
 - [build.sh](build.sh) script builds the ZIP archives of the `pages` directory.
 - [build-index.sh](build-index.sh) script builds the index of available pages.
-- [check-pr.sh](check-pr.sh) script checks the pages syntax and performs various checks on the PR.
+- [check-pr.sh](check-pr.sh) script checks the page's syntax and performs various checks on the PR.
 - [deploy.sh](deploy.sh) script deploys the ZIP and PDF archives to the static website repository.
-- [send-to-bot.py](send-to-bot.py) is a Python script that send the build or tests output to tldr-bot.
+- [send-to-bot.py](send-to-bot.py) is a Python script that sends the build or test output to tldr-bot.
 - [set-alias-page.py](set-alias-page.py) is a Python script to generate or update alias pages.
 - [set-more-info-link.py](set-more-info-link.py) is a Python script to generate or update more information links across pages.
 - [set-page-title.py](set-page-title.py) is a Python script to update the title across pages.
-- [test.sh](test.sh) script runs some basic tests on every PR/commit to make sure that the pages are valid and that the code is formatted correctly.
+- [test.sh](test.sh) script runs some basic tests on every PR/commit to ensure the pages are valid and the code is formatted correctly.
 - [wrong-filename.sh](wrong-filename.sh) script checks the consistency between the filenames and the page title.
 - [update-command.py](update-command.py) is a Python script to update the common contents of a command example across all languages.
 
 ## Compatibility
 
-The below table shows the compatibility of user-executable scripts with different platforms.
+The table below shows the compatibility of user-executable scripts with different platforms:
 
 | Script | Linux | macOS (`osx`) | Windows |
 | ------ | ----- | ----- | ------- |
 | [render.py](pdf/render.py) | ✅ | ✅ | ✅ |
-| [build-pdf.sh](pdf/build-pdf.sh) | ✅ | ✅ | ❌ |
-| [build.sh](build.sh) | ✅ | ✅ | ❌ |
+| [build-pdf.sh](pdf/build-pdf.sh) | ✅ | ✅ | ❌ (WSL ✅)|
+| [build.sh](build.sh) | ✅ | ✅ | ❌ (WSL ✅)|
 | [set-alias-pages.py](set-alias-pages.py) | ✅ | ✅ | ✅ |
 | [set-more-info-link.py](set-more-info-link.py) | ✅ | ✅ | ✅ |
 | [set-page-title.py](set-page-title.py) | ✅ | ✅ | ✅ |
-| [wrong-filename.sh](wrong-filename.sh) | ✅ | ❌ | ❌ |
+| [wrong-filename.sh](wrong-filename.sh) | ✅ | ❌ | ❌ (WSL ✅)|
 | [update-command.py](update-command.py) | ✅ | ✅ | ✅ |

scripts/_common.py

@@ -0,0 +1,393 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: MIT
+
+"""
+A Python file that makes some commonly used functions available for other scripts to use.
+"""
+
+from enum import Enum
+from pathlib import Path
+from unittest.mock import patch
+import shutil
+import os
+import argparse
+import subprocess
+
+IGNORE_FILES = (".DS_Store",)
+
+
+class Colors(str, Enum):
+    def __str__(self):
+        return str(
+            self.value
+        )  # make str(Colors.COLOR) return the ANSI code instead of an Enum object
+
+    RED = "\x1b[31m"
+    GREEN = "\x1b[32m"
+    BLUE = "\x1b[34m"
+    CYAN = "\x1b[36m"
+    RESET = "\x1b[0m"
+
+
+def test_ignore_files():
+    assert IGNORE_FILES == (".DS_Store",)
+    assert ".DS_Store" in IGNORE_FILES
+    assert "tldr.md" not in IGNORE_FILES
+
+
+def get_tldr_root(lookup_path: Path = None) -> Path:
+    """
+    Get the path of the local tldr repository, looking for it in each part of the given path. If it is not found, the path in the environment variable TLDR_ROOT is returned.
+
+    Parameters:
+    lookup_path (Path): the path to search for the tldr root. By default, the path of the script.
+
+    Returns:
+    Path: the local tldr repository.
+    """
+
+    if lookup_path is None:
+        absolute_lookup_path = Path(__file__).resolve()
+    else:
+        absolute_lookup_path = Path(lookup_path).resolve()
+    if (
+        tldr_root := next(
+            (path for path in absolute_lookup_path.parents if path.name == "tldr"), None
+        )
+    ) is not None:
+        return tldr_root
+    elif "TLDR_ROOT" in os.environ:
+        return Path(os.environ["TLDR_ROOT"])
+    raise SystemExit(
+        f"{Colors.RED}Please set the environment variable TLDR_ROOT to the location of a clone of https://github.com/tldr-pages/tldr{Colors.RESET}"
+    )
+
+
+def test_get_tldr_root():
+    tldr_root = get_tldr_root("/path/to/tldr/scripts/test_script.py")
+    assert tldr_root == Path("/path/to/tldr")
+
+    # Set TLDR_ROOT in the environment
+    os.environ["TLDR_ROOT"] = "/path/to/tldr_clone"
+
+    tldr_root = get_tldr_root("/tmp")
+    assert tldr_root == Path("/path/to/tldr_clone")
+
+    del os.environ["TLDR_ROOT"]
+
+    # Remove TLDR_ROOT from the environment
+    original_env = os.environ.pop("TLDR_ROOT", None)
+
+    # Check if SystemExit is raised
+    raised = False
+    try:
+        get_tldr_root("/tmp")
+    except SystemExit:
+        raised = True
+    assert raised
+
+    # Restore the original values
+    if original_env is not None:
+        os.environ["TLDR_ROOT"] = original_env
+
+
+def get_pages_dir(root: Path) -> list[Path]:
+    """
+    Get all pages directories.
+
+    Parameters:
+    root (Path): the path to search for the pages directories.
+
+    Returns:
+    list (list of Path's): Path's of page entry and platform, e.g. "page.fr/common".
+    """
+
+    return [d for d in root.iterdir() if d.name.startswith("pages")]
+
+
+def test_get_pages_dir():
+    # Create temporary directories with names starting with "pages"
+
+    root = Path("test_root")
+
+    shutil.rmtree(root, True)
+
+    root.mkdir(exist_ok=True)
+
+    # Create temporary directories with names that do not start with "pages"
+    (root / "other_dir_1").mkdir(exist_ok=True)
+    (root / "other_dir_2").mkdir(exist_ok=True)
+
+    # Call the function and verify that it returns an empty list
+    result = get_pages_dir(root)
+    assert result == []
+
+    (root / "pages").mkdir(exist_ok=True)
+    (root / "pages.fr").mkdir(exist_ok=True)
+    (root / "other_dir").mkdir(exist_ok=True)
+
+    # Call the function and verify the result
+    result = get_pages_dir(root)
+    expected = [root / "pages", root / "pages.fr"]
+    assert result.sort() == expected.sort()  # the order differs on Unix / macOS
+
+    shutil.rmtree(root, True)
+
+
+def get_target_paths(page: Path, pages_dirs: Path) -> list[Path]:
+    """
+    Get all paths in all languages that match the page.
+
+    Parameters:
+    page (Path): the page to search for.
+
+    Returns:
+    list (list of Path's): A list of Path's.
+    """
+
+    target_paths = []
+
+    if not page.lower().endswith(".md"):
+        page = f"{page}.md"
+    arg_platform, arg_page = page.split("/")
+
+    for pages_dir in pages_dirs:
+        page_path = pages_dir / arg_platform / arg_page
+
+        if not page_path.exists():
+            continue
+        target_paths.append(page_path)
+
+    target_paths.sort()
+
+    return target_paths
+
+
+def test_get_target_paths():
+    root = Path("test_root")
+
+    shutil.rmtree(root, True)
+
+    root.mkdir(exist_ok=True)
+
+    shutil.os.makedirs(root / "pages" / "common")
+    shutil.os.makedirs(root / "pages.fr" / "common")
+
+    file_path = root / "pages" / "common" / "tldr.md"
+    with open(file_path, "w"):
+        pass
+
+    file_path = root / "pages.fr" / "common" / "tldr.md"
+    with open(file_path, "w"):
+        pass
+
+    target_paths = get_target_paths("common/tldr", get_pages_dir(root))
+    for path in target_paths:
+        rel_path = "/".join(path.parts[-3:])
+        print(rel_path)
+
+    shutil.rmtree(root, True)
+
+
+def get_locale(path: Path) -> str:
+    """
+    Get the locale from the path.
+
+    Parameters:
+    path (Path): the path to extract the locale.
+
+    Returns:
+    str: a POSIX Locale Name in the form of "ll" or "ll_CC" (e.g. "fr" or "pt_BR").
+    """
+
+    # compute locale
+    pages_dirname = path.parents[1].name
+    if "." in pages_dirname:
+        _, locale = pages_dirname.split(".")
+    else:
+        locale = "en"
+
+    return locale
+
+
+def test_get_locale():
+    assert get_locale(Path("path/to/pages.fr/common/tldr.md")) == "fr"
+
+    assert get_locale(Path("path/to/pages/common/tldr.md")) == "en"
+
+    assert get_locale(Path("path/to/other/common/tldr.md")) == "en"
+
+
+def get_status(action: str, dry_run: bool, type: str) -> str:
+    """
+    Get a colored status line.
+
+    Parameters:
+    action (str): The action to perform.
+    dry_run (bool): Whether to perform a dry-run.
+    type (str): The kind of object to modify (alias, link).
+
+    Returns:
+    str: A colored line
+    """
+
+    match action:
+        case "added":
+            start_color = Colors.CYAN
+        case "updated":
+            start_color = Colors.BLUE
+        case _:
+            start_color = Colors.RED
+
+    if dry_run:
+        status = f"{type} would be {action}"
+    else:
+        status = f"{type} {action}"
+
+    return create_colored_line(start_color, status)
+
+
+def test_get_status():
+    # Test dry run status
+    assert (
+        get_status("added", True, "alias")
+        == f"{Colors.CYAN}alias would be added{Colors.RESET}"
+    )
+    assert (
+        get_status("updated", True, "link")
+        == f"{Colors.BLUE}link would be updated{Colors.RESET}"
+    )
+
+    # Test non-dry run status
+    assert (
+        get_status("added", False, "alias") == f"{Colors.CYAN}alias added{Colors.RESET}"
+    )
+    assert (
+        get_status("updated", False, "link")
+        == f"{Colors.BLUE}link updated{Colors.RESET}"
+    )
+
+    # Test default color for unknown action
+    assert (
+        get_status("unknown", True, "alias")
+        == f"{Colors.RED}alias would be unknown{Colors.RESET}"
+    )
+
+
+def create_colored_line(start_color: str, text: str) -> str:
+    """
+    Create a colored line.
+
+    Parameters:
+    start_color (str): The color for the line.
+    text (str): The text to display.
+
+    Returns:
+    str: A colored line
+    """
+
+    return f"{start_color}{text}{Colors.RESET}"
+
+
+def test_create_colored_line():
+    assert (
+        create_colored_line(Colors.CYAN, "TLDR") == f"{Colors.CYAN}TLDR{Colors.RESET}"
+    )
+    assert create_colored_line("Hello", "TLDR") == f"HelloTLDR{Colors.RESET}"
+
+
+def create_argument_parser(description: str) -> argparse.ArgumentParser:
+    """
+    Create an argument parser that can be extended.
+
+    Parameters:
+    description (str): The description for the argument parser
+
+    Returns:
+    ArgumentParser: an argument parser.
+    """
+
+    parser = argparse.ArgumentParser(description=description)
+    parser.add_argument(
+        "-p",
+        "--page",
+        type=str,
+        default="",
+        help='page name in the format "platform/alias_command.md"',
+    )
+    parser.add_argument(
+        "-S",
+        "--sync",
+        action="store_true",
+        default=False,
+        help="synchronize each translation's alias page (if exists) with that of English page",
+    )
+    parser.add_argument(
+        "-l",
+        "--language",
+        type=str,
+        default="",
+        help='language in the format "ll" or "ll_CC" (e.g. "fr" or "pt_BR")',
+    )
+    parser.add_argument(
+        "-s",
+        "--stage",
+        action="store_true",
+        default=False,
+        help="stage modified pages (requires `git` to be on $PATH and TLDR_ROOT to be a Git repository)",
+    )
+    parser.add_argument(
+        "-n",
+        "--dry-run",
+        action="store_true",
+        default=False,
+        help="show what changes would be made without actually modifying the pages",
+    )
+
+    return parser
+
+
+def test_create_argument_parser():
+    description = "Test argument parser"
+    parser = create_argument_parser(description)
+
+    assert isinstance(parser, argparse.ArgumentParser)
+    assert parser.description == description
+
+    # Check if each expected argument is added with the correct configurations
+    arguments = [
+        ("-p", "--page", str, ""),
+        ("-l", "--language", str, ""),
+        ("-s", "--stage", None, False),
+        ("-S", "--sync", None, False),
+        ("-n", "--dry-run", None, False),
+    ]
+    for short_flag, long_flag, arg_type, default_value in arguments:
+        action = parser._option_string_actions[short_flag]  # Get action for short flag
+        assert action.dest.replace("_", "-") == long_flag.lstrip(
+            "-"
+        )  # Check destination name
+        assert action.type == arg_type  # Check argument type
+        assert action.default == default_value  # Check default value
+
+
+def stage(paths: list[Path]):
+    """
+    Stage the given paths using Git.
+
+    Parameters:
+    paths (list of Paths): the list of Path's to stage using Git.
+
+    """
+    subprocess.call(["git", "add", *(path.resolve() for path in paths)])
+
+
+@patch("subprocess.call")
+def test_stage(mock_subprocess_call):
+    paths = [Path("/path/to/file1"), Path("/path/to/file2")]
+
+    # Call the stage function
+    stage(paths)
+
+    # Verify that subprocess.call was called with the correct arguments
+    mock_subprocess_call.assert_called_once_with(["git", "add", *paths])

scripts/set-alias-page.py

@@ -41,57 +41,56 @@ Examples:
        python3 scripts/set-alias-page.py --sync --language pt_BR
 
     4. Read English alias pages, synchronize them into all translations and stage modified pages for commit:
-       python3 scripts/set-more-info-link.py -Ss
-       python3 scripts/set-more-info-link.py --sync --stage
+       python3 scripts/set-alias-page.py -Ss
+       python3 scripts/set-alias-page.py --sync --stage
 
     5. Read English alias pages and show what changes would be made:
        python3 scripts/set-alias-page.py -Sn
        python3 scripts/set-alias-page.py --sync --dry-run
 """
 
-import argparse
-import os
 import re
-import subprocess
 from pathlib import Path
-
-IGNORE_FILES = (".DS_Store", "tldr.md", "aria2.md")
-
-
-def get_tldr_root():
-    """
-    Get the path of local tldr repository for environment variable TLDR_ROOT.
-    """
-
-    # If this script is running from tldr/scripts, the parent's parent is the root
-    f = Path(__file__).resolve()
-    if (
-        tldr_root := next((path for path in f.parents if path.name == "tldr"), None)
-    ) is not None:
-        return tldr_root
-    elif "TLDR_ROOT" in os.environ:
-        return Path(os.environ["TLDR_ROOT"])
-    raise SystemExit(
-        "\x1b[31mPlease set TLDR_ROOT to the location of a clone of https://github.com/tldr-pages/tldr."
+from _common import (
+    IGNORE_FILES,
+    Colors,
+    get_tldr_root,
+    get_pages_dir,
+    get_target_paths,
+    get_locale,
+    get_status,
+    stage,
+    create_colored_line,
+    create_argument_parser,
+)
+
+IGNORE_FILES += ("tldr.md", "aria2.md")
+
+
+def test_ignore_files():
+    assert IGNORE_FILES == (
+        ".DS_Store",
+        "tldr.md",
+        "aria2.md",
     )
+    assert ".DS_Store" in IGNORE_FILES
+    assert "tldr.md" in IGNORE_FILES
 
 
-def get_templates(root):
+def get_templates(root: Path):
     """
     Get all alias page translation templates from
     TLDR_ROOT/contributing-guides/translation-templates/alias-pages.md.
 
     Parameters:
-    root (string): The path of local tldr repository, i.e., TLDR_ROOT.
+    root (Path): The path of local tldr repository, i.e., TLDR_ROOT.
 
     Returns:
     dict of (str, str): Language labels map to alias page templates.
     """
 
-    template_file = os.path.join(
-        root, "contributing-guides/translation-templates/alias-pages.md"
-    )
-    with open(template_file, encoding="utf-8") as f:
+    template_file = root / "contributing-guides/translation-templates/alias-pages.md"
+    with template_file.open(encoding="utf-8") as f:
         lines = f.readlines()
 
     # Parse alias-pages.md
@@ -122,156 +121,115 @@ def get_templates(root):
     return templates
 
 
-def get_alias_page(file):
-    """
-    Determine whether the given file is an alias page.
-
-    Parameters:
-    file (string): Path to a page
-
-    Returns:
-    str: "" If the file doesn't exit or is not an alias page,
-         otherwise return what command the alias stands for.
-    """
-
-    if not os.path.isfile(file):
-        return ""
-    with open(file, "r", encoding="utf-8") as f:
-        for line in f:
-            if match := re.search(r"^`tldr (.+)`", line):
-                return match[1]
-    return ""
-
-
-def set_alias_page(file, command, dry_run=False, language_to_update=""):
+def set_alias_page(
+    path: Path, command: str, dry_run: bool = False, language_to_update: str = ""
+) -> str:
     """
     Write an alias page to disk.
 
     Parameters:
-    file (string): Path to an alias page
+    path (string): Path to an alias page
     command (string): The command that the alias stands for.
     dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
     language_to_update (string): Optionally, the language of the translation to be updated.
 
     Returns:
     str: Execution status
-         "" if the alias page standing for the same command already exists.
-         "\x1b[36mpage added" if it's a new alias page.
-         "\x1b[34mpage updated" if the command updates.
+         "" if the alias page standing for the same command already exists or if the locale does not match language_to_update.
+         "\x1b[36mpage added"
+         "\x1b[34mpage updated"
+         "\x1b[36mpage would be added"
+         "\x1b[34mpage would updated"
     """
 
-    # compute locale
-    pages_dir = os.path.basename(os.path.dirname(os.path.dirname(file)))
-    if "." in pages_dir:
-        _, locale = pages_dir.split(".")
-    else:
-        locale = "en"
+    locale = get_locale(path)
     if locale not in templates or (
         language_to_update != "" and locale != language_to_update
     ):
+        # return empty status to indicate that no changes were made
         return ""
 
     # Test if the alias page already exists
-    orig_command = get_alias_page(file)
-    if orig_command == command:
+    original_command = get_alias_page(path)
+    if original_command == command:
         return ""
 
-    if orig_command == "":
-        status_prefix = "\x1b[36m"
-        action = "added"
-    else:
-        status_prefix = "\x1b[34m"
-        action = "updated"
-
-    if dry_run:
-        status = f"page would be {action}"
-    else:
-        status = f"page {action}"
-
-    status = f"{status_prefix}{status}\x1b[0m"
+    status = get_status(
+        "added" if original_command == "" else "updated", dry_run, "page"
+    )
 
-    if not dry_run:  # Only write to the file during a non-dry-run
-        alias_name, _ = os.path.splitext(os.path.basename(file))
+    if not dry_run:  # Only write to the path during a non-dry-run
+        alias_name = path.stem
         text = (
             templates[locale]
             .replace("example", alias_name, 1)
             .replace("example", command)
         )
-        os.makedirs(os.path.dirname(file), exist_ok=True)
-        with open(file, "w", encoding="utf-8") as f:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with path.open("w", encoding="utf-8") as f:
             f.write(text)
 
     return status
 
 
+def get_alias_page(path: Path) -> str:
+    """
+    Determine whether the given path is an alias page.
+
+    Parameters:
+    path (Path): Path to a page
+
+    Returns:
+    str: "" If the path doesn't exit or is not an alias page,
+         otherwise return what command the alias stands for.
+    """
+
+    if not path.exists():
+        return ""
+    with path.open(encoding="utf-8") as f:
+        for line in f:
+            # match alias (`tldr <alias>`)
+            if match := re.search(r"^`tldr (.+)`", line):
+                return match[1]
+    return ""
+
+
 def sync(
-    root, pages_dirs, alias_name, orig_command, dry_run=False, language_to_update=""
-):
+    root: Path,
+    pages_dirs: list[Path],
+    alias_name: str,
+    original_command: str,
+    dry_run: bool = False,
+    language_to_update: str = "",
+) -> list[Path]:
     """
     Synchronize an alias page into all translations.
 
     Parameters:
-    root (str): TLDR_ROOT
-    pages_dirs (list of str): Strings of page entry and platform, e.g. "page.fr/common".
+    root (Path): TLDR_ROOT
+    pages_dirs (list of Path's): Path's of page entry and platform, e.g. "page.fr/common".
     alias_name (str): An alias command with .md extension like "vi.md".
-    orig_command (string): An Original command like "vim".
+    original_command (str): An Original command like "vim".
     dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
-    language_to_update (string): Optionally, the language of the translation to be updated.
+    language_to_update (str): Optionally, the language of the translation to be updated.
 
     Returns:
-    list: A list of paths to be staged into git, using by --stage option.
+    list (list of Path's): A list of Path's to be staged into git, using by --stage option.
     """
-    rel_paths = []
+    paths = []
     for page_dir in pages_dirs:
-        path = os.path.join(root, page_dir, alias_name)
-        status = set_alias_page(path, orig_command, dry_run, language_to_update)
+        path = root / page_dir / alias_name
+        status = set_alias_page(path, original_command, dry_run, language_to_update)
         if status != "":
-            rel_path = path.replace(f"{root}/", "")
-            rel_paths.append(rel_path)
-            print(f"\x1b[32m{rel_path} {status}\x1b[0m")
-    return rel_paths
+            rel_path = "/".join(path.parts[-3:])
+            paths.append(rel_path)
+            print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
+    return paths
 
 
 def main():
-    parser = argparse.ArgumentParser(
-        description="Sets the alias page for all translations of a page"
-    )
-    parser.add_argument(
-        "-p",
-        "--page",
-        type=str,
-        required=False,
-        default="",
-        help='page name in the format "platform/alias_command.md"',
-    )
-    parser.add_argument(
-        "-S",
-        "--sync",
-        action="store_true",
-        default=False,
-        help="synchronize each translation's alias page (if exists) with that of English page",
-    )
-    parser.add_argument(
-        "-l",
-        "--language",
-        type=str,
-        required=False,
-        default="",
-        help='language in the format "ll" or "ll_CC" (e.g. "fr" or "pt_BR")',
-    )
-    parser.add_argument(
-        "-s",
-        "--stage",
-        action="store_true",
-        default=False,
-        help="stage modified pages (requires `git` to be on $PATH and TLDR_ROOT to be a Git repository)",
-    )
-    parser.add_argument(
-        "-n",
-        "--dry-run",
-        action="store_true",
-        default=False,
-        help="show what changes would be made without actually modifying the pages",
+    parser = create_argument_parser(
+        "Sets the alias page for all translations of a page"
     )
     parser.add_argument("command", type=str, nargs="?", default="")
     args = parser.parse_args()
@@ -281,51 +239,47 @@ def main():
     # A dictionary of all alias page translations
     global templates
     templates = get_templates(root)
-    pages_dirs = [d for d in os.listdir(root) if d.startswith("pages")]
-    rel_paths = []
+    pages_dirs = get_pages_dir(root)
+
+    target_paths = []
 
     # Use '--page' option
     if args.page != "":
-        if not args.page.lower().endswith(".md"):
-            args.page = f"{args.page}.md"
-
-        target_paths = [os.path.join(root, p, args.page) for p in pages_dirs]
-        target_paths.sort()
+        target_paths += get_target_paths(args.page, pages_dirs)
 
         for path in target_paths:
-            rel_path = path.replace(f"{root}/", "")
-            rel_paths.append(rel_path)
-            status = set_alias_page(path, args.command, args.language)
+            rel_path = "/".join(path.parts[-3:])
+            status = set_alias_page(path, args.command, args.dry_run, args.language)
             if status != "":
-                print(f"\x1b[32m{rel_path} {status}\x1b[0m")
+                print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
 
     # Use '--sync' option
     elif args.sync:
-        pages_dirs.remove("pages")
-        en_page = os.path.join(root, "pages")
-        platforms = [i for i in os.listdir(en_page) if i not in IGNORE_FILES]
+        pages_dirs.remove(root / "pages")
+        en_path = root / "pages"
+        platforms = [i.name for i in en_path.iterdir() if i.name not in IGNORE_FILES]
         for platform in platforms:
-            platform_path = os.path.join(en_page, platform)
+            platform_path = en_path / platform
             commands = [
-                f"{platform}/{p}"
-                for p in os.listdir(platform_path)
-                if p not in IGNORE_FILES
+                f"{platform}/{page.name}"
+                for page in platform_path.iterdir()
+                if page.name not in IGNORE_FILES
             ]
             for command in commands:
-                orig_command = get_alias_page(os.path.join(root, "pages", command))
-                if orig_command != "":
-                    rel_paths += sync(
+                original_command = get_alias_page(root / "pages" / command)
+                if original_command != "":
+                    target_paths += sync(
                         root,
                         pages_dirs,
                         command,
-                        orig_command,
+                        original_command,
                         args.dry_run,
                         args.language,
                     )
 
     # Use '--stage' option
-    if args.stage and not args.dry_run:
-        subprocess.call(["git", "add", *rel_paths], cwd=root)
+    if args.stage and not args.dry_run and len(target_paths) > 0:
+        stage(target_paths)
 
 
 if __name__ == "__main__":

scripts/set-more-info-link.py

@@ -5,16 +5,18 @@
 A Python script to add or update the "More information" link for all translations of a page.
 
 Note: If the current directory or one of its parents is called "tldr", the script will assume it is the tldr root, i.e., the directory that contains a clone of https://github.com/tldr-pages/tldr
-If you aren't, the script will use TLDR_ROOT as the tldr root. Also, ensure 'git' is available.
+If the script doesn't find it in the current path, the environment variable TLDR_ROOT will be used as the tldr root. Also, ensure 'git' is available.
 
 Usage:
-    python3 scripts/set-more-info-link.py [-p PAGE] [-S] [-s] [-n] [LINK]
+    python3 scripts/set-more-info-link.py [-p PAGE] [-S] [-l LANGUAGE] [-s] [-n] [LINK]
 
 Options:
     -p, --page PAGE
-        Specify the alias page in the format "platform/alias_command.md". This option allows setting the link for a specific page.
+        Specify the page in the format "platform/command". This option allows setting the link for a specific page.
     -S, --sync
-        Synchronize each translation's more information link (if exists) with that of the English page.
+        Synchronize each translation's "More information" link (if exists) with that of the English page.
+    -l, --language LANGUAGE
+        Specify the language, a POSIX Locale Name in the form of "ll" or "ll_CC" (e.g. "fr" or "pt_BR").
     -s, --stage
         Stage modified pages (requires 'git' on $PATH and TLDR_ROOT to be a Git repository).
     -n, --dry-run
@@ -25,27 +27,40 @@ Positional Argument:
 
 Examples:
     1. Set the link for a specific page:
-       python3 scripts/set-more-info-link.py -p common/tar.md https://example.com
-       python3 scripts/set-more-info-link.py --page common/tar.md https://example.com
+       python3 scripts/set-more-info-link.py -p common/tar https://example.com
+       python3 scripts/set-more-info-link.py --page common/tar https://example.com
 
-    2. Read English pages and synchronize more information links across translations:
+    2. Read English pages and synchronize the "More information" link across translations:
        python3 scripts/set-more-info-link.py -S
        python3 scripts/set-more-info-link.py --sync
 
-    3. Read English pages, synchronize more information links across translations and stage modified pages for commit:
+    3. Read English pages and synchronize the "More information" link for Brazilian Portuguese pages only:
+       python3 scripts/set-more-info-link.py -S -l pt_BR
+       python3 scripts/set-more-info-link.py --sync --language pt_BR
+
+    4. Read English pages, synchronize the "More information" link across translations and stage modified pages for commit:
        python3 scripts/set-more-info-link.py -Ss
        python3 scripts/set-more-info-link.py --sync --stage
 
-    4. Show what changes would be made across translations:
+    5. Show what changes would be made across translations:
        python3 scripts/set-more-info-link.py -Sn
        python3 scripts/set-more-info-link.py --sync --dry-run
 """
 
-import argparse
-import os
 import re
-import subprocess
 from pathlib import Path
+from _common import (
+    IGNORE_FILES,
+    Colors,
+    get_tldr_root,
+    get_pages_dir,
+    get_target_paths,
+    get_locale,
+    get_status,
+    stage,
+    create_colored_line,
+    create_argument_parser,
+)
 
 labels = {
     "en": "More information:",
@@ -87,28 +102,33 @@ labels = {
     "zh": "更多信息：",
 }
 
-IGNORE_FILES = (".DS_Store",)
-
 
-def get_tldr_root():
+def set_link(
+    path: Path, link: str, dry_run: bool = False, language_to_update: str = ""
+) -> str:
     """
-    Get the path of local tldr repository for environment variable TLDR_ROOT.
+    Write a "More information" link in a page to disk.
+
+    Parameters:
+    path (string): Path to a page
+    link (string): The "More information" link to insert.
+    dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
+    language_to_update (string): Optionally, the language of the translation to be updated.
+
+    Returns:
+    str: Execution status
+         "" if the page does not need an update or if the locale does not match language_to_update.
+         "\x1b[36mlink added"
+         "\x1b[34mlink updated"
+         "\x1b[36mlink would be added"
+         "\x1b[34mlink would updated"
     """
 
-    # If this script is running from tldr/scripts, the parent's parent is the root
-    f = Path(__file__).resolve()
-    if (
-        tldr_root := next((path for path in f.parents if path.name == "tldr"), None)
-    ) is not None:
-        return tldr_root
-    elif "TLDR_ROOT" in os.environ:
-        return Path(os.environ["TLDR_ROOT"])
-    raise SystemExit(
-        "\x1b[31mPlease set TLDR_ROOT to the location of a clone of https://github.com/tldr-pages/tldr."
-    )
-
+    locale = get_locale(path)
+    if language_to_update != "" and locale != language_to_update:
+        # return empty status to indicate that no changes were made
+        return ""
 
-def set_link(path: Path, link: str, dry_run=False) -> str:
     with path.open(encoding="utf-8") as f:
         lines = f.readlines()
 
@@ -123,13 +143,6 @@ def set_link(path: Path, link: str, dry_run=False) -> str:
             desc_end = i
             break
 
-    # compute locale
-    pages_dirname = path.parents[1].name
-    if "." in pages_dirname:
-        _, locale = pages_dirname.split(".")
-    else:
-        locale = "en"
-
     # build new line
     if locale in ["bn", "hi", "ne"]:
         new_line = f"> {labels[locale]} <{link}>।\n"
@@ -144,27 +157,18 @@ def set_link(path: Path, link: str, dry_run=False) -> str:
         # return empty status to indicate that no changes were made
         return ""
 
-    status_prefix = "\x1b[36m"  # Color code for pages
-
     if re.search(r"^>.*<.+>", lines[desc_end]):
         # overwrite last line
         lines[desc_end] = new_line
-        status_prefix = "\x1b[34m"
         action = "updated"
     else:
         # add new line
         lines.insert(desc_end + 1, new_line)
-        status_prefix = "\x1b[36m"
         action = "added"
 
-    if dry_run:
-        status = f"link would be {action}"
-    else:
-        status = f"link {action}"
-
-    status = f"{status_prefix}{status}\x1b[0m"
+    status = get_status(action, dry_run, "link")
 
-    if not dry_run:
+    if not dry_run:  # Only write to the path during a non-dry-run
         with path.open("w", encoding="utf-8") as f:
             f.writelines(lines)
 
@@ -172,6 +176,19 @@ def set_link(path: Path, link: str, dry_run=False) -> str:
 
 
 def get_link(path: Path) -> str:
+    """
+    Determine whether the given path has a "More information" link.
+
+    Parameters:
+    path (Path): Path to a page
+
+    Returns:
+    str: "" If the path doesn't exit or does not have a link,
+         otherwise return the "More information" link.
+    """
+
+    if not path.exists():
+        return ""
     with path.open(encoding="utf-8") as f:
         lines = f.readlines()
 
@@ -194,80 +211,60 @@ def get_link(path: Path) -> str:
 
 
 def sync(
-    root: Path, pages_dirs: list[str], command: str, link: str, dry_run=False
-) -> list[str]:
+    root: Path,
+    pages_dirs: list[Path],
+    command: str,
+    link: str,
+    dry_run: bool = False,
+    language_to_update: str = "",
+) -> list[Path]:
+    """
+    Synchronize a "More information" link into all translations.
+
+    Parameters:
+    root (Path): TLDR_ROOT
+    pages_dirs (list of Path's): Path's of page entry and platform, e.g. "page.fr/common".
+    command (str): A command like "tar".
+    link (str): A link like "https://example.com".
+    dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
+    language_to_update (str): Optionally, the language of the translation to be updated.
+
+    Returns:
+    list (list of Path's): A list of Path's to be staged into git, using by --stage option.
+    """
     paths = []
     for page_dir in pages_dirs:
         path = root / page_dir / command
         if path.exists():
-            rel_path = "/".join(path.parts[-3:])
-            status = set_link(path, link, dry_run)
+            status = set_link(path, link, dry_run, language_to_update)
             if status != "":
-                paths.append(path)
-                print(f"\x1b[32m{rel_path} {status}\x1b[0m")
+                rel_path = "/".join(path.parts[-3:])
+                paths.append(rel_path)
+                print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
     return paths
 
 
 def main():
-    parser = argparse.ArgumentParser(
-        description='Sets the "More information" link for all translations of a page'
-    )
-    parser.add_argument(
-        "-p",
-        "--page",
-        type=str,
-        required=False,
-        default="",
-        help='page name in the format "platform/command.md"',
-    )
-    parser.add_argument(
-        "-S",
-        "--sync",
-        action="store_true",
-        default=False,
-        help="synchronize each translation's more information link (if exists) with that of English page",
-    )
-    parser.add_argument(
-        "-s",
-        "--stage",
-        action="store_true",
-        default=False,
-        help="stage modified pages (requires `git` to be on $PATH and TLDR_ROOT to be a Git repository)",
-    )
-    parser.add_argument(
-        "-n",
-        "--dry-run",
-        action="store_true",
-        default=False,
-        help="show what changes would be made without actually modifying the pages",
+    parser = create_argument_parser(
+        'Sets the "More information" link for all translations of a page'
     )
     parser.add_argument("link", type=str, nargs="?", default="")
     args = parser.parse_args()
 
     root = get_tldr_root()
-    pages_dirs = [d for d in root.iterdir() if d.name.startswith("pages")]
+    pages_dirs = get_pages_dir(root)
 
     target_paths = []
 
     # Use '--page' option
     if args.page != "":
-        if not args.page.lower().endswith(".md"):
-            args.page = f"{args.page}.md"
-        arg_platform, arg_page = args.page.split("/")
-
-        for pages_dir in pages_dirs:
-            page_path = pages_dir / arg_platform / arg_page
-            if not page_path.exists():
-                continue
-            target_paths.append(page_path)
-
-        target_paths.sort()
+        target_paths += get_target_paths(args.page, pages_dirs)
 
         for path in target_paths:
             rel_path = "/".join(path.parts[-3:])
-            status = set_link(path, args.link)
+            status = set_link(path, args.link, args.dry_run, args.language)
             if status != "":
-                print(f"\x1b[32m{rel_path} {status}\x1b[0m")
+                print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
 
     # Use '--sync' option
     elif args.sync:
@@ -279,15 +276,18 @@ def main():
             commands = [
                 f"{platform}/{page.name}"
                 for page in platform_path.iterdir()
-                if page not in IGNORE_FILES
+                if page.name not in IGNORE_FILES
             ]
             for command in commands:
                 link = get_link(root / "pages" / command)
                 if link != "":
-                    target_paths += sync(root, pages_dirs, command, link, args.dry_run)
+                    target_paths += sync(
+                        root, pages_dirs, command, link, args.dry_run, args.language
+                    )
 
+    # Use '--stage' option
     if args.stage and not args.dry_run and len(target_paths) > 0:
-        subprocess.call(["git", "add", *target_paths], cwd=root)
+        stage(target_paths)
 
 
 if __name__ == "__main__":

scripts/set-page-title.py

@@ -5,16 +5,18 @@
 A Python script to add or update the page title for all translations of a page.
 
 Note: If the current directory or one of its parents is called "tldr", the script will assume it is the tldr root, i.e., the directory that contains a clone of https://github.com/tldr-pages/tldr
-If you aren't, the script will use TLDR_ROOT as the tldr root. Also, ensure 'git' is available.
+If the script doesn't find it in the current path, the environment variable TLDR_ROOT will be used as the tldr root. Also, ensure 'git' is available.
 
 Usage:
-    python3 scripts/set-page-title.py [-p PAGE] [-S] [-s] [-n] [TITLE]
+    python3 scripts/set-page-title.py [-p PAGE] [-S] [-l LANGUAGE] [-s] [-n] [TITLE]
 
 Options:
     -p, --page PAGE
-        Specify the page in the format "platform/command.md". This option allows setting the title for a specific page.
+        Specify the page in the format "platform/command". This option allows setting the title for a specific page.
     -S, --sync
         Synchronize each translation's title (if exists) with that of the English page.
+    -l, --language LANGUAGE
+        Specify the language, a POSIX Locale Name in the form of "ll" or "ll_CC" (e.g. "fr" or "pt_BR").
     -s, --stage
         Stage modified pages (requires 'git' on $PATH and TLDR_ROOT to be a Git repository).
     -n, --dry-run
@@ -25,49 +27,67 @@ Positional Argument:
 
 Examples:
     1. Set the title for a specific page:
-       python3 scripts/set-page-title.py -p common/tar.md tar.1
-       python3 scripts/set-page-title.py --page common/tar.md tar.1
+       python3 scripts/set-page-title.py -p common/tar tar
+       python3 scripts/set-page-title.py --page common/tar tar
 
     2. Synchronize titles across translations:
        python3 scripts/set-page-title.py -S
        python3 scripts/set-page-title.py --sync
 
-    3. Synchronize titles across translations and stage modified pages for commit:
+    3. Read English pages and synchronize the title for Brazilian Portuguese pages only:
+       python3 scripts/set-page-title.py -S -l pt_BR
+       python3 scripts/set-page-title.py --sync --language pt_BR
+
+    4. Synchronize titles across translations and stage modified pages for commit:
        python3 scripts/set-page-title.py -Ss
        python3 scripts/set-page-title.py --sync --stage
 
-    4. Show what changes would be made across translations:
+    5. Show what changes would be made across translations:
        python3 scripts/set-page-title.py -Sn
        python3 scripts/set-page-title.py --sync --dry-run
 """
 
-import argparse
-import os
-import subprocess
 from pathlib import Path
-
-IGNORE_FILES = (".DS_Store",)
-
-
-def get_tldr_root():
+from _common import (
+    IGNORE_FILES,
+    Colors,
+    get_tldr_root,
+    get_pages_dir,
+    get_target_paths,
+    get_locale,
+    get_status,
+    stage,
+    create_colored_line,
+    create_argument_parser,
+)
+
+
+def set_page_title(
+    path: Path, title: str, dry_run: bool = False, language_to_update: str = ""
+) -> str:
     """
-    Get the path of local tldr repository for environment variable TLDR_ROOT.
+    Write a title in a page to disk.
+
+    Parameters:
+    path (string): Path to a page
+    title (string): The title to insert.
+    dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
+    language_to_update (string): Optionally, the language of the translation to be updated.
+
+    Returns:
+    str: Execution status
+         "" if the page does not need an update or if the locale does not match language_to_update.
+         "\x1b[36mtitle added"
+         "\x1b[34mtitle updated"
+         "\x1b[36mtitle would be added"
+         "\x1b[34mtitle would updated"
     """
 
-    # If this script is running from tldr/scripts, the parent's parent is the root
-    f = Path(__file__).resolve()
-    if (
-        tldr_root := next((path for path in f.parents if path.name == "tldr"), None)
-    ) is not None:
-        return tldr_root
-    elif "TLDR_ROOT" in os.environ:
-        return Path(os.environ["TLDR_ROOT"])
-    raise SystemExit(
-        "\x1b[31mPlease set TLDR_ROOT to the location of a clone of https://github.com/tldr-pages/tldr."
-    )
-
+    locale = get_locale(path)
+    if language_to_update != "" and locale != language_to_update:
+        # return empty status to indicate that no changes were made
+        return ""
 
-def set_title(path: Path, title: str, dry_run=False) -> str:
     new_line = f"# {title}\n"
 
     # Read the content of the Markdown file
@@ -78,14 +98,9 @@ def set_title(path: Path, title: str, dry_run=False) -> str:
         # return empty status to indicate that no changes were made
         return ""
 
-    if dry_run:
-        status = "title would be updated"
-    else:
-        status = "title updated"
-
-    status = f"\x1b[34m{status}\x1b[0m"
+    status = get_status("updated", dry_run, "title")
 
-    if not dry_run:
+    if not dry_run:  # Only write to the path during a non-dry-run
         lines[0] = new_line
         with path.open("w", encoding="utf-8") as f:
             f.writelines(lines)
@@ -93,7 +108,20 @@ def set_title(path: Path, title: str, dry_run=False) -> str:
     return status
 
 
-def get_title(path: Path) -> str:
+def get_page_title(path: Path) -> str:
+    """
+    Determine whether the given path has a title.
+
+    Parameters:
+    path (Path): Path to a page
+
+    Returns:
+    str: "" If the path doesn't exit or does not have a title,
+         otherwise return the page title.
+    """
+
+    if not path.exists():
+        return ""
     with path.open(encoding="utf-8") as f:
         first_line = f.readline().strip()
 
@@ -101,80 +129,58 @@ def get_title(path: Path) -> str:
 
 
 def sync(
-    root: Path, pages_dirs: list[str], command: str, title: str, dry_run=False
+    root: Path,
+    pages_dirs: list[Path],
+    command: str,
+    title: str,
+    dry_run: bool = False,
+    language_to_update: str = "",
 ) -> list[str]:
+    """
+    Synchronize a page title into all translations.
+
+    Parameters:
+    root (Path): TLDR_ROOT
+    pages_dirs (list of Path's): Path's of page entry and platform, e.g. "page.fr/common".
+    command (str): A command like "tar".
+    title (str): A title like "tar".
+    dry_run (bool): Whether to perform a dry-run, i.e. only show the changes that would be made.
+    language_to_update (str): Optionally, the language of the translation to be updated.
+
+    Returns:
+    list (list of Path's): A list of Path's to be staged into git, using by --stage option.
+    """
     paths = []
     for page_dir in pages_dirs:
         path = root / page_dir / command
         if path.exists():
-            rel_path = "/".join(path.parts[-3:])
-            status = set_title(path, title, dry_run)
+            status = set_page_title(path, title, dry_run, language_to_update)
             if status != "":
-                paths.append(path)
-                print(f"\x1b[32m{rel_path} {status}\x1b[0m")
+                rel_path = "/".join(path.parts[-3:])
+                paths.append(rel_path)
+                print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
     return paths
 
 
 def main():
-    parser = argparse.ArgumentParser(
-        description="Sets the title for all translations of a page"
-    )
-    parser.add_argument(
-        "-p",
-        "--page",
-        type=str,
-        required=False,
-        default="",
-        help='page name in the format "platform/command.md"',
-    )
-    parser.add_argument(
-        "-s",
-        "--stage",
-        action="store_true",
-        default=False,
-        help="stage modified pages (requires `git` to be on $PATH and TLDR_ROOT to be a Git repository)",
-    )
-    parser.add_argument(
-        "-S",
-        "--sync",
-        action="store_true",
-        default=False,
-        help="synchronize each translation's title (if exists) with that of English page",
-    )
-    parser.add_argument(
-        "-n",
-        "--dry-run",
-        action="store_true",
-        default=False,
-        help="show what changes would be made without actually modifying the pages",
-    )
+    parser = create_argument_parser("Sets the title for all translations of a page")
     parser.add_argument("title", type=str, nargs="?", default="")
     args = parser.parse_args()
 
     root = get_tldr_root()
-    pages_dirs = [d for d in root.iterdir() if d.name.startswith("pages")]
+    pages_dirs = get_pages_dir(root)
 
     target_paths = []
 
     # Use '--page' option
     if args.page != "":
-        if not args.page.lower().endswith(".md"):
-            args.page = f"{args.page}.md"
-        arg_platform, arg_page = args.page.split("/")
-
-        for pages_dir in pages_dirs:
-            page_path = pages_dir / arg_platform / arg_page
-            if not page_path.exists():
-                continue
-            target_paths.append(page_path)
-
-        target_paths.sort()
+        target_paths += get_target_paths(args.page, pages_dirs)
 
         for path in target_paths:
             rel_path = "/".join(path.parts[-3:])
-            status = set_title(path, args.title)
+            status = set_page_title(path, args.title)
             if status != "":
-                print(f"\x1b[32m{rel_path} {status}\x1b[0m")
+                print(create_colored_line(Colors.GREEN, f"{rel_path} {status}"))
 
     # Use '--sync' option
     elif args.sync:
@@ -186,15 +192,18 @@ def main():
             commands = [
                 f"{platform}/{page.name}"
                 for page in platform_path.iterdir()
-                if page not in IGNORE_FILES
+                if page.name not in IGNORE_FILES
             ]
             for command in commands:
-                title = get_title(root / "pages" / command)
+                title = get_page_title(root / "pages" / command)
                 if title != "":
-                    target_paths += sync(root, pages_dirs, command, title, args.dry_run)
+                    target_paths += sync(
+                        root, pages_dirs, command, title, args.dry_run, args.language
+                    )
 
+    # Use '--stage' option
     if args.stage and not args.dry_run and len(target_paths) > 0:
-        subprocess.call(["git", "add", *target_paths], cwd=root)
+        stage(target_paths)
 
 
 if __name__ == "__main__":

scripts/test-requirements.txt

@@ -0,0 +1 @@
+pytest

scripts/test.sh

@@ -57,6 +57,20 @@ function run_flake8 {
   flake8 scripts
 }
 
+function run_pytest {
+  # skip pytest check if the command is not available in the system.
+  if [[ $CI != true ]] && ! exists pytest; then
+    echo "Skipping pytest check, command not available."
+    return 0
+  fi
+
+  errs=$(pytest scripts/*.py 2>&1 || true)
+  if [[ ${errs} == *"failed"* ]]; then
+    echo -e "${errs}" >&2
+    return 1
+  fi
+}
+
 # Default test function, run by `npm test`.
 function run_tests {
   find pages* -name '*.md' -exec markdownlint {} +
@@ -74,6 +88,7 @@ function run_tests {
   done
   run_black
   run_flake8
+  run_pytest
 }
 
 # Special test function for GitHub Actions pull request builds.