Skip to content

Show long time warnings as GitHub annotations #40413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions src/sage/doctest/forker.py
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,8 @@
Since it needs to be able to read stdout, it should be called
while spoofing using :class:`SageSpoofInOut`.

INPUT: see :meth:`run`.

EXAMPLES::

sage: from sage.doctest.parsing import SageOutputChecker
Expand Down Expand Up @@ -628,6 +630,7 @@
check = self._checker.check_output

# Process each example.
example: doctest.Example
for examplenum, example in enumerate(test.examples):
if failures:
# If exitfirst is set, abort immediately after a
Expand Down Expand Up @@ -1185,7 +1188,7 @@
example.total_state = self.running_global_digest.hexdigest()
example.doctest_state = self.running_doctest_digest.hexdigest()

def _failure_header(self, test, example, message='Failed example:'):
def _failure_header(self, test, example, message='Failed example:', extra=None):
"""
We strip out ``sage:`` prompts, so we override
:meth:`doctest.DocTestRunner._failure_header` for better
Expand All @@ -1197,6 +1200,14 @@

- ``example`` -- a :class:`doctest.Example` instance in ``test``

- ``message`` -- a message to be shown. Must not have a newline

- ``extra`` -- an extra message to be shown in GitHub annotation

Note that ``message`` and ``extra`` are not accepted by
:meth:`doctest.DocTestRunner._failure_header`, as such by Liskov
substitution principle this method must be callable without passing those.

OUTPUT: string used for reporting that the given example failed

EXAMPLES::
Expand Down Expand Up @@ -1260,6 +1271,8 @@
message += ' [failed in baseline]'
else:
command = f'::error title={message}'
if extra:
message += f': {extra}'
if extra := getattr(example, 'extra', None):
message += f': {extra}'
if test.filename:
Expand Down Expand Up @@ -1561,12 +1574,12 @@
Test ran for 1.23s cpu, 2.50s wall
Check ran for 2.34s cpu, 3.12s wall
"""
out(self._failure_header(test, example, 'Warning: slow doctest:') +
('Test ran for %.2fs cpu, %.2fs wall\nCheck ran for %.2fs cpu, %.2fs wall\n'
% (example.cputime,
example.walltime,
check_timer.cputime,
check_timer.walltime)))
time_info = ('Test ran for %.2fs cpu, %.2fs wall\nCheck ran for %.2fs cpu, %.2fs wall\n'
% (example.cputime,
example.walltime,
check_timer.cputime,
check_timer.walltime))
out(self._failure_header(test, example, 'Warning: slow doctest:', time_info) + time_info)

def report_unexpected_exception(self, out, test, example, exc_info):
r"""
Expand Down Expand Up @@ -2195,7 +2208,7 @@
sage: W.start()
sage: DC = DocTestController(DD, filename)
sage: reporter = DocTestReporter(DC)
sage: W.join() # Wait for worker to finish

Check warning on line 2211 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.98s cpu, 20.43s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2211 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.01s cpu, 18.89s wall Check ran for 0.01s cpu, 0.00s wall

Check warning on line 2211 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 12.76s cpu, 16.80s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2211 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all, editable)

Warning: slow doctest:

slow doctest:: Test ran for 12.93s cpu, 15.63s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2211 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 12.54s cpu, 16.00s wall Check ran for 0.00s cpu, 0.00s wall
sage: result = W.result_queue.get()
sage: reporter.report(FDS, False, W.exitcode, result, "")
[... tests, ...s wall]
Expand Down Expand Up @@ -2322,7 +2335,7 @@
....: except OSError:
....: print("Write end of pipe successfully closed")
Write end of pipe successfully closed
sage: W.join() # Wait for worker to finish

Check warning on line 2338 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.55s cpu, 18.84s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2338 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.67s cpu, 17.85s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2338 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 12.93s cpu, 15.55s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2338 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all, editable)

Warning: slow doctest:

slow doctest:: Test ran for 13.30s cpu, 15.91s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2338 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 12.78s cpu, 15.28s wall Check ran for 0.00s cpu, 0.00s wall
"""
super().start()

Expand Down Expand Up @@ -2358,7 +2371,7 @@
sage: W.start()
sage: while W.rmessages is not None:
....: W.read_messages()
sage: W.join()

Check warning on line 2374 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.73s cpu, 0.01s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2374 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 9.31s cpu, 0.01s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2374 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 13.03s cpu, 0.04s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2374 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all, editable)

Warning: slow doctest:

slow doctest:: Test ran for 13.33s cpu, 0.04s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2374 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 13.56s cpu, 0.04s wall Check ran for 0.00s cpu, 0.00s wall
sage: len(W.messages) > 0
True
"""
Expand Down Expand Up @@ -2390,7 +2403,7 @@
sage: FDS = FileDocTestSource(filename, DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()
sage: W.join()

Check warning on line 2406 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 10.39s cpu, 19.45s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2406 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (macos, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 9.71s cpu, 18.43s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2406 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.11, all)

Warning: slow doctest:

slow doctest:: Test ran for 12.79s cpu, 16.11s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2406 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all, editable)

Warning: slow doctest:

slow doctest:: Test ran for 13.41s cpu, 15.96s wall Check ran for 0.00s cpu, 0.00s wall

Check warning on line 2406 in src/sage/doctest/forker.py

View workflow job for this annotation

GitHub Actions / Conda (ubuntu, Python 3.12, all)

Warning: slow doctest:

slow doctest:: Test ran for 13.57s cpu, 16.86s wall Check ran for 0.00s cpu, 0.00s wall
sage: W.save_result_output()
sage: sorted(W.result[1].keys())
['cputime', 'err', 'failures', 'optionals', 'tests', 'walltime', 'walltime_skips']
Expand Down
6 changes: 3 additions & 3 deletions src/sage/doctest/parsing.py
Original file line number Diff line number Diff line change
Expand Up @@ -829,7 +829,7 @@ def __ne__(self, other):
"""
return not (self == other)

def parse(self, string, *args):
def parse(self, string, *args) -> list[doctest.Example | str]:
r"""
A Sage specialization of :class:`doctest.DocTestParser`.

Expand Down Expand Up @@ -1015,8 +1015,8 @@ def parse(self, string, *args):
string = find_python_continuation.sub(r"\1" + ellipsis_tag + r"\2", string)
string = find_sage_prompt.sub(r"\1>>> sage: ", string)
string = find_sage_continuation.sub(r"\1...", string)
res = doctest.DocTestParser.parse(self, string, *args)
filtered = []
res: list[doctest.Example | str] = doctest.DocTestParser.parse(self, string, *args)
filtered: list[doctest.Example | str] = []
persistent_optional_tags = self.file_optional_tags
persistent_optional_tag_setter = None
persistent_optional_tag_setter_index = None
Expand Down
20 changes: 11 additions & 9 deletions src/sage/doctest/sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ def __ne__(self, other):
"""
return not (self == other)

def _process_doc(self, doctests, doc, namespace, start):
def _process_doc(self, doctests: list[doctest.DocTest], doc, namespace, start):
"""
Appends doctests defined in ``doc`` to the list ``doctests``.

Expand Down Expand Up @@ -266,7 +266,7 @@ def file_optional_tags(self):
"""
return set()

def _create_doctests(self, namespace, tab_okay=None):
def _create_doctests(self, namespace, tab_okay=None) -> tuple[list[doctest.DocTest], dict]:
"""
Create a list of doctests defined in this source.

Expand Down Expand Up @@ -314,7 +314,7 @@ def _create_doctests(self, namespace, tab_okay=None):
probed_tags=self.options.probe,
file_optional_tags=self.file_optional_tags)
self.linking = False
doctests = []
doctests: list[doctest.DocTest] = []
in_docstring = False
unparsed_doc = False
doc = []
Expand Down Expand Up @@ -480,7 +480,7 @@ def __iter__(self):
for lineno, line in enumerate(self.source.split('\n')):
yield lineno + self.lineno_shift, line + '\n'

def create_doctests(self, namespace):
def create_doctests(self, namespace) -> tuple[list[doctest.DocTest], dict]:
r"""
Create doctests from this string.

Expand All @@ -492,8 +492,8 @@ def create_doctests(self, namespace):

- ``doctests`` -- list of doctests defined by this string

- ``tab_locations`` -- either ``False`` or a list of linenumbers
on which tabs appear
- ``extras`` -- dictionary with ``extras['tab']`` either
``False`` or a list of linenumbers on which tabs appear

EXAMPLES::

Expand All @@ -503,10 +503,12 @@ def create_doctests(self, namespace):
sage: s = "'''\n sage: 2 + 2\n 4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource',(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, tabs = PSS.create_doctests({})
sage: dt, extras = PSS.create_doctests({})
sage: for t in dt:
....: print("{} {}".format(t.name, t.examples[0].sage_source))
<runtime> 2 + 2
sage: extras
{...'tab': []...}
"""
return self._create_doctests(namespace)

Expand Down Expand Up @@ -736,7 +738,7 @@ def file_optional_tags(self):
from .parsing import parse_file_optional_tags
return parse_file_optional_tags(self)

def create_doctests(self, namespace):
def create_doctests(self, namespace) -> tuple[list[doctest.DocTest], dict]:
r"""
Return a list of doctests for this file.

Expand Down Expand Up @@ -910,7 +912,7 @@ class SourceLanguage:

Currently supported languages include Python, ReST and LaTeX.
"""
def parse_docstring(self, docstring, namespace, start):
def parse_docstring(self, docstring, namespace, start) -> list[doctest.DocTest]:
"""
Return a list of doctest defined in this docstring.

Expand Down
36 changes: 36 additions & 0 deletions src/sage/doctest/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,42 @@
...
0

Check slow doctest warnings are correctly raised::

sage: subprocess.call(["sage", "-t", "--warn-long", # long time
....: "--random-seed=0", "--optional=sage", "sleep2.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long --random-seed=0 sleep2.rst
**********************************************************************
File "sleep2.rst", line 4, in sage.doctest.tests.sleep2
Warning: slow doctest:
while walltime(t) < 2: pass
Test ran for ...s cpu, ...s wall
Check ran for ...s cpu, ...s wall
[2 tests, ...s wall]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
...
0
sage: subprocess.call(["sage", "-t", "--format=github", "--warn-long", # long time
....: "--random-seed=0", "--optional=sage", "sleep2.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long --random-seed=0 sleep2.rst
**********************************************************************
::warning title=Warning: slow doctest:,file=sleep2.rst,line=4::slow doctest:: Test ran for ...s cpu, ...s wall%0ACheck ran for ...s cpu, ...s wall%0A
while walltime(t) < 2: pass
Test ran for ...s cpu, ...s wall
Check ran for ...s cpu, ...s wall
[2 tests, ...s wall]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
...
0

Check handling of tolerances::

sage: subprocess.call(["python3", "-m", "sage.doctest", "--warn-long", "0", # long time
Expand Down
4 changes: 4 additions & 0 deletions src/sage/doctest/tests/sleep2.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Test raising slow doctest warning (cputime instead of walltime is checked so we need busy loop)::

sage: t = walltime()
sage: while walltime(t) < 2: pass
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy