Add messages.pot update script to pre-commit. #8900
Conversation
rebecca-shoptaw
force-pushed
the
8818/feature/add-pot-updates-to-precommit
branch
from
40dff61 to
860b4c9
Compare
There was a problem hiding this comment.
I think the SyntaxWarning: invalid escape sequence '\ ' is coming from something that i18n-messages extract is calling, rather than from pre-commit.
I don't think this is the solution, but with the change to use pass_filenames: false instead of files, and the following, pre-commit shows Passed for Generate POT:
diff --git a/scripts/i18n-messages b/scripts/i18n-messages
index 98f28e5d6..4085c927c 100755
--- a/scripts/i18n-messages
+++ b/scripts/i18n-messages
@@ -4,6 +4,10 @@ macros and write to openlibrary/i18n/messages.pot file.
"""
import _init_path # noqa: F401 Imported for its side effect of setting PYTHONPATH
+import warnings
+
+warnings.simplefilter('ignore')
+
import sys
from openlibrary import i18nOf note, this SyntaxWarning only shows up with Python 3.12, and shows up regardless whether pre-commit is used. See also the second change listed here for Python 3.12: https://docs.python.org/3/whatsnew/3.12.html#other-language-changes.
rebecca-shoptaw
force-pushed
the
8818/feature/add-pot-updates-to-precommit
branch
3 times, most recently
from
b4876a7 to
9e0e962
Compare
There was a problem hiding this comment.
Awesome, thank you @rebecca-shoptaw !! I tested this in gitpod/locally, and:
- ✅ When no html/python files staged it doesn't run
- ✅ When HTML file staged with no i18n lines changed, it runs but doesn't save
- ✅ When HTML file staged with i18n lines moved but not modified, it runs but doesn't save
- ✅ When HTML file staged with i18n lines modified, it runs and saves
There's one case I realized would cause some issues; on my machine I have a bunch of untracked files, some of which are html/python! These are getting added to the messages.pot file when I do a commit, which is not what we want. I reckon we should try to exclude untracked/unstaged files when running from pre-commit. Maybe a --pre-commit option that adds these exclusions?
# Exclude untracked git files.
# git ls-files --others --exclude-standard
untracked_files = {
Path(line)
for dir in dirs
for line in subprocess.run(
['git', 'ls-files', '--others', '--exclude-standard', dir],
stdout=subprocess.PIPE,
text=True,
).stdout.split('\n')
if line.endswith(('.py', '.html'))
}
# Also exclude unstaged git files.
# git diff --name-only
unstaged_files = {
Path(line)
for line in subprocess.run(
['git', 'diff', '--name-only'],
stdout=subprocess.PIPE,
text=True,
).stdout.split('\n')
if line.endswith(('.py', '.html'))
}
excluded_files = untracked_files | unstaged_filesAnd then when it builds the catalog makes sure to toss these out?
Otherwise a few small code cleanups/fixes!
cdrini
added
the
Needs: Submitter Input
rebecca-shoptaw
force-pushed
the
8818/feature/add-pot-updates-to-precommit
branch
3 times, most recently
from
cae27a4 to
afec73d
Compare
rebecca-shoptaw
removed
the
Needs: Submitter Input
cdrini
added
the
Priority: 1
Co-authored-by: Scott Barnes <scottreidbarnes@gmail.com>
Co-authored-by: Scott Barnes <scottreidbarnes@gmail.com>
for more information, see https://pre-commit.ci
rebecca-shoptaw
force-pushed
the
8818/feature/add-pot-updates-to-precommit
branch
from
afa7d32 to
6564803
Compare
|
We talked Creation-Date potentially causing a few headaches via merge conflicts ; we decided to create a separate issue to fix the Creation-Date to the first day of the current year; this should hopefully not interfere with any translator flows, whilst reducing conflict headaches for us! |
Closes #8818.
Feature. Automates generation of new
messages.potfile for translation whenever Python or HTML files that affecti18nstrings are changed.Technical
The implementation is in two parts and mainly confined to two files:
pre-commit-config.yaml-- Added a newpre-commitlocal hook that runs the./scripts/i18n-messages extractscript to generate a newmessages.potfile and is activated by changes to Python or HTML files.Note: This hook has a few dependencies will need to be updated to match the current version when we update
Babel,multipart, and/orweb.py.__init__.pyinopenlibrary/i18n-- Wrote new logic into the message extraction function, using a straightforwardsymmetric_differencecompare between the old and newi18nmessages to make sure thatmessages.potonly gets updated wheni18nstrings have been added/modified/deleted.Note: This was done to prevent an infinite loop of failure within the pre-commit hook, as the hook will otherwise update the template time field in
messages.poton every run, and report a failed run as a result of the file modification, which prevents CI autofixing.This extra logic also had the nice side effect of ensuring that there will be no
messages.potupdates for totally unrelated files, as only actual relevant string changes will trigger the re-generation.Testing
i18n-formatted textpre-commitif not installedpre-commitshould automatically generate a newmessages.potfileStakeholders
@cdrini