Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

Run CI-testing on GHA/Windows #337

Closed
wants to merge 20 commits into from
Closed

Run CI-testing on GHA/Windows #337

wants to merge 20 commits into from

Conversation

amotl
Copy link
Contributor

@amotl amotl commented Nov 13, 2020 •
edited
Loading

Dear Mathias,

while working on crate/crate-python#388, we faced some deadlock issues when tearing down the testlayer on GHA/Windows. So, we wanted to take the chance to also test the testlayer incorporated into cr8.

Please consider this as a work in progress / draft unless further notice.

With kind regards,
Andreas.

@amotl amotl force-pushed the amo/windows branch 2 times, most recently from 1e15689 to 20b6b8f Compare November 13, 2020 10:19
@amotl
Copy link
Contributor Author

amotl commented Nov 13, 2020 •
edited
Loading

Apparently, the deadlock issues on testlayer teardown also happen here.

======================================================================
ERROR: D:\a\cr8\cr8\tests\..\README.rst
Doctest: README.rst
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.6\x64\lib\doctest.py", line 2178, in tearDown
    self._dt_tearDown(test)
  File "D:\a\cr8\cr8\tests\test_integration.py", line 38, in teardown
    node.stop()
  File "D:\a\cr8\cr8\cr8\run_crate.py", line 395, in stop
    self.process.communicate(timeout=10)
  File "C:\hostedtoolcache\windows\Python\3.8.6\x64\lib\subprocess.py", line 1024, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "C:\hostedtoolcache\windows\Python\3.8.6\x64\lib\subprocess.py", line 1397, in _communicate
    raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command '['C:\\Users\\runneradmin\\.cache\\cr8\\crates\\crate-4.3.1\\bin\\crate.bat', '-Cdiscovery.initial_state_timeout=0', '-Cnetwork.host=127.0.0.1', '-Cudc.enabled=false', '-Ccluster.name=cr8-tests', '-Chttp.port=44200-44250', '-Cpath.data=C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp2yvi7ihz']' timed out after 10 seconds

-- https://github.com/mfussenegger/cr8/pull/337/checks?check_run_id=1395324258#step:6:237

@amotl
Copy link
Contributor Author

amotl commented Nov 13, 2020 •
edited
Loading

Another thing I would like to share in this context is that

echo "select name from sys.cluster" | cr8 timeit --hosts http://127.0.0.1:44200

will croak with

SqlException: SQLParseException[line 1:1: mismatched input '"select name from sys.cluster"' expecting {'SELECT', 'DEALLOCATE', 'CREATE', 'ALTER', 'KILL', 'BEGIN', 'COMMIT', 'ANALYZE', 'DISCARD', 'EXPLAIN', 'SHOW', 'OPTIMIZE', 'REFRESH', 'RESTORE', 'DROP', 'INSERT', 'VALUES', 'DELETE', 'UPDATE', 'SET', 'RESET', 'COPY', 'GRANT', 'DENY', 'REVOKE'}] occurred using: {"stmt": "\"select name from sys.cluster\""}

-- https://github.com/mfussenegger/cr8/pull/337/checks?check_run_id=1395743511#step:6:173

This happens even when running that on bash after 94ad6b2.

Apparently, the quotes are taken verbatim when echoing a string through a pipe. I believe [1] is discussing this very issue and others are experiencing similar pains, see also rundeck/rundeck#602 and rundeck/rundeck#1406.

[1] https://stackoverflow.com/questions/804646/how-do-you-strip-quotes-out-of-an-echoed-string-in-a-windows-batch-file

@amotl
Copy link
Contributor Author

amotl commented Dec 17, 2020

We have been able to dedicate some more minutes of research into the deadlocking issue on layer teardown when running on Windows.

Firstly, we have been able to discover very valuable resources at [1,2,3], which give many general insights into signal handling for child processes on Windows.

However, in the context of subprocess.Popen, we also just found [4,5,6], where [5] specifically states that the CREATE_NEW_PROCESS_GROUP creationflags parameter to subprocess.Popen is required for using os.kill() on the subprocess. We will try that within one of the next iterations.

[1] https://stackoverflow.com/questions/813086/can-i-send-a-ctrl-c-sigint-to-an-application-on-windows
[2] https://stackoverflow.com/questions/3342941/kill-child-process-when-parent-process-is-killed/4657392#4657392
[3] https://stanislavs.org/stopping-command-line-applications-programatically-with-ctrl-c-events-from-net/
[4] https://docs.python.org/3/library/subprocess.html#subprocess.Popen.send_signal
[5] https://docs.python.org/3/library/subprocess.html#subprocess.CREATE_NEW_PROCESS_GROUP
[6] https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.asyncio.subprocess.Process.send_signal

@amotl amotl mentioned this pull request Feb 19, 2021
Merged
5 tasks
@amotl amotl mentioned this pull request Apr 11, 2022
Merged
@amotl amotl force-pushed the amo/windows branch 2 times, most recently from 40bf147 to 5204550 Compare January 17, 2023 16:55
@amotl amotl mentioned this pull request Oct 9, 2023
Open
amotl added 9 commits February 10, 2024 16:39
@amotl
Run CI-testing on GHA/Windows
fef4ccf
@amotl
Adjust .gitignore file
bd760d0
@amotl
CI: Add support for extracting zip archives
dd90112
@amotl
CI: Mess with LANG or LC_ALL only on non-Windows systems
6f1e981
@amotl
CI: Mitigate nasty "Unrecognized Windows Sockets error: 10106"
ede5144
@amotl
CI: Don't invoke SourceBuildTest on Windows
bf79d43
@amotl
CI: Try to invoke unit tests explicitly with bash to work around Windows
87f9e5b
@amotl
Tests: Improve diff output at "test_geoshape_type_default"
a34b07e 
This test has an anomaly on Windows where the longitude
36.10457754779138 will be returned as 36.104577547791386,
i.e. with one more precision digit.

While investigating this, I found the diff output for comparing
before/after just the other way round than what I would have expected.
@amotl
CI: Skip "test_geoshape_type_default" test on Windows
4d13194
amotl added 3 commits February 10, 2024 19:18
@amotl
CI: Use LANG/LC_ALL/LANGUAGE also on Windows systems again
bc773b1
@amotl
CI: Set environment variable WINUTF8=1
4fb5e43
@amotl
Windows: Attempt to fix Unicode encoding error using chcp
ea1a42a 
UnicodeEncodeError: 'charmap' codec can't encode character '\u2192' in position 65: character maps to <undefined>
@amotl amotl force-pushed the amo/windows branch 2 times, most recently from 4b94495 to ce7399e Compare February 10, 2024 23:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@amotl @mfussenegger
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy