Add dev tools for live-reloading Anki's web views (#2151)

* Add dev tools for live-reloading the web stack while running Anki

* Handle CDP connection errors more graciously

* Include sass in web stack watchers

* Refactor monitored folder and event definition

* Switch to more specific build target

Thanks to @hikaru-y

* Add PyChromeDevTools to dev requirements

* Update rebuild-web for ninja

* Satisfy mypy

* Remove ts-watch

Superseded by web-watch (the version here was also still based around bazel)

* Simplify calls to other build tools

Given that `./ninja qt/aqt` has to be run from the project root anyways, it doesn't make sense to use calls relative to `rebuild-web` in an ill-guided effort to lower dependencies on hard-coded paths.

* Remove remaining script-relative tool path

python/requirements.dev.in

@@ -9,6 +9,7 @@ mypy-protobuf
 pip-tools
 pylint
 pytest
+PyChromeDevTools
 fluent.syntax
 types-decorator
 types-flask

python/requirements.dev.txt

@@ -368,6 +368,9 @@ protobuf==4.21.9 \
     # via
     #   -r requirements.bundle.txt
     #   mypy-protobuf
+pychromedevtools==0.4 \
+    --hash=sha256:453f889b11c58fed348206d1b6e91a0bbfe23a319365c586ae462214ecb513ce
+    # via -r requirements.dev.in
 pylint==2.15.5 \
     --hash=sha256:3b120505e5af1d06a5ad76b55d8660d44bf0f2fc3c59c2bdd94e39188ee3a4df \
     --hash=sha256:c2108037eb074334d9e874dc3c783752cc03d0796c88c9a9af282d0f161a1004
@@ -420,7 +423,9 @@ pytoml==0.1.21 \
 requests==2.28.1 \
     --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 \
     --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349
-    # via -r requirements.bundle.txt
+    # via
+    #   -r requirements.bundle.txt
+    #   pychromedevtools
 send2trash==1.8.0 \
     --hash=sha256:d2c24762fd3759860a0aff155e45871447ea58d2be6bdd39b5c8f966a0c99c2d \
     --hash=sha256:f20eaadfdb517eaca5ce077640cb261c7d2698385a6a0f072a4a5447fd49fa08
@@ -525,6 +530,10 @@ waitress==2.1.2 \
     --hash=sha256:7500c9625927c8ec60f54377d590f67b30c8e70ef4b8894214ac6e4cad233d2a \
     --hash=sha256:780a4082c5fbc0fde6a2fcfe5e26e6efc1e8f425730863c04085769781f51eba
     # via -r requirements.bundle.txt
+websocket-client==1.4.2 \
+    --hash=sha256:d6b06432f184438d99ac1f456eaf22fe1ade524c3dd16e661142dc54e9cba574 \
+    --hash=sha256:d6e8f90ca8e2dd4e8027c4561adeb9456b54044312dba655e7cae652ceb9ae59
+    # via pychromedevtools
 werkzeug==2.2.2 \
     --hash=sha256:7ea2d48322cc7c0f8b3a215ed73eabd7b5d75d0b50e31ab006286ccff9e00b8f \
     --hash=sha256:f979ab81f58d7318e064e99c4506445d60135ac5cd2e177a2de0089bfd4c9bd5

tools/rebuild-web

@@ -0,0 +1,12 @@
+#!/bin/bash
+# Copyright: Ankitects Pty Ltd and contributors
+# License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
+
+# Manually trigger a rebuild and reload of Anki's web stack
+
+# NOTE: This script needs to be run from the project root
+
+set -e
+
+./ninja qt/aqt
+./out/pyenv/bin/python tools/reload_webviews.py

tools/reload_webviews.py

@@ -0,0 +1,55 @@
+#!/usr/bin/env python
+# Copyright: Ankitects Pty Ltd and contributors
+# License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
+
+"""
+Trigger a reload of Anki's web views using QtWebEngine' Chromium
+Remote Debugging interface.
+"""
+
+import argparse
+import sys
+
+import PyChromeDevTools  # type: ignore[import]
+
+DEFAULT_HOST = "localhost"
+DEFAULT_PORT = 8080
+
+
+def print_error(message: str):
+    print(f"Error: {message}", file=sys.stderr)
+
+
+parser = argparse.ArgumentParser("reload_webviews")
+parser.add_argument(
+    "--host",
+    help=f"Host via which the Chrome session can be reached, e.g. {DEFAULT_HOST}",
+    type=str,
+    default=DEFAULT_HOST,
+    required=False,
+)
+parser.add_argument(
+    "--port",
+    help=f"Port via which the Chrome session can be reached, e.g. {DEFAULT_PORT}",
+    type=str,
+    default=DEFAULT_PORT,
+    required=False,
+)
+args = parser.parse_args()
+
+try:
+    chrome = PyChromeDevTools.ChromeInterface(host=args.host, port=args.port)
+except Exception as e:
+    print_error(
+        f"Could not establish connection to Chromium remote debugger. Exception:\n{e}"
+    )
+    exit(1)
+
+if chrome.tabs is None:
+    print_error("Was unable to get active web views.")
+    exit(1)
+
+for tab_index, tab_data in enumerate(chrome.tabs):
+    print(f"Reloading page: {tab_data['title']}")
+    chrome.connect(tab=tab_index, update_tabs=False)
+    chrome.Page.reload()

tools/ts-watch

@@ -1,13 +0,0 @@
-#!/bin/bash
-#
-# Monitor the ts folder and rebuild aqt's data each time
-# it changes, for testing pages locally.
-#
-# On a Mac, useful to combine with ts-run.
-
-# run once at startup
-cmd='printf \\033c\\n; bazel build qt:runanki'
-sh -c "$cmd"
-
-# then monitor for changes
-fswatch -r -o ts | xargs -n1 -I{} sh -c "$cmd"

tools/web-watch

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Copyright: Ankitects Pty Ltd and contributors
+# License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
+
+# Monitor all web-related folders and rebuild and reload Anki's web stack
+# when a change is detected.
+
+set -e
+
+MONITORED_FOLDERS=("ts/" "sass/" "qt/aqt/data/web/")
+MONITORED_EVENTS=("Created" "Updated" "Removed")
+
+on_change_detected="printf \\033c\\n; \"./tools/rebuild-web\""
+
+event_args=""
+for event in "${MONITORED_EVENTS[@]}"; do
+    event_args+="--event ${event} "
+done
+
+# poll_monitor comes with a slight performance penalty, but seems to more
+# reliably identify file system events across both macOS and Linux
+fswatch -r -o -m poll_monitor ${event_args[@]} \
+    "${MONITORED_FOLDERS[@]}" | xargs -n1 -I{} sh -c "$on_change_detected"