Skip to content

Commit 0eeb705

Browse files
dmitry-lipetskvshepard
dmitry-lipetsk
and
vshepard
authored
Tests are based on pytest (#192)
* Using pytest [pytest.raises] * Using pytest [pytest.skip] * Using pytest [assertIsNotNone] * Using pytest [assertFalse] * Using pytest [assertTrue] * Using pytest [assertEqual] * Using pytest [assertNotEqual] * Using pytest [assertGreaterEqual] * Using pytest [assertGreater] * Using pytest [assertIn] * Using pytest [assertListEqual] * unittest is not used * Code style (flake8) * Execution signature is removed * run_tests.sh installs pytest * run_tests.sh is updated run tests through pytest explicitly * Total refactoring of tests - TestgresRemoteTests does not use global variables and code - Explicit work with ..testgres folder * Code style (flake8) * Root __init__.py is added It is required for tests. * Code style (flake8) * pytest.ini is added * TestgresTests::test_ports_management is corrected Let's send warning about a garbage in the container "bound_ports" and continue working. * coding: utf-8 * Cleanup * CI runs all the tests of testgres. * Add install ssh (cherry picked from commit fec1e7a) * Revert "Add install ssh" This reverts commit 537a9ac. * Revert "CI runs all the tests of testgres." This reverts commit 2d2532c. * Test of probackup plugin is restored It works now (was runned with a fresh probackup2 and vanilla 18devel). * The test suite of a probackup plugin is based on pytest * Probackup plugin is updated Probackup plugin tests - They are skipped if PGPROBACKUPBIN is not defined Global variable init_params is None when PGPROBACKUPBIN is not defined or version is not processed * CI test use 4 cores * testgres.plugins.probackup2.Init was restored [thanks to Yuri Sokolov] * pytest.ini is updated [testpaths] Enumeration of all the known folders with tests. * test_child_pids (local, remote) is updated Multiple attempts and logging are added. * test_child_process_dies is updated Multiple attempts are added. --------- Co-authored-by: vshepard <v.shepard@postgrespro.ru>
1 parent 669e134 commit 0eeb705

File tree

13 files changed

+839
-617
lines changed

13 files changed

+839
-617
lines changed

‎__init__.py

Whitespace-only changes.

‎pytest.ini

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[pytest]
2+
testpaths = ["./tests", "./testgres/plugins/pg_probackup2/pg_probackup2/tests"]
3+
addopts = --strict-markers
4+
markers =
5+
#log_file = logs/pytest.log
6+
log_file_level = NOTSET
7+
log_file_format = %(levelname)8s [%(asctime)s] %(message)s
8+
log_file_date_format=%Y-%m-%d %H:%M:%S
9+

‎run_tests.sh

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ export VIRTUAL_ENV_DISABLE_PROMPT=1
2222
source $VENV_PATH/bin/activate
2323

2424
# install utilities
25-
$PIP install coverage flake8 psutil Sphinx
25+
$PIP install coverage flake8 psutil Sphinx pytest pytest-xdist psycopg2 six psutil
2626

2727
# install testgres' dependencies
2828
export PYTHONPATH=$(pwd)
29-
$PIP install .
29+
# $PIP install .
3030

3131
# test code quality
3232
flake8 .
@@ -38,21 +38,19 @@ rm -f $COVERAGE_FILE
3838

3939

4040
# run tests (PATH)
41-
time coverage run -a tests/test_simple.py
41+
time coverage run -a -m pytest -l -v -n 4 -k "TestgresTests"
4242

4343

4444
# run tests (PG_BIN)
4545
time \
4646
PG_BIN=$(dirname $(which pg_config)) \
47-
ALT_CONFIG=1 \
48-
coverage run -a tests/test_simple.py
47+
coverage run -a -m pytest -l -v -n 4 -k "TestgresTests"
4948

5049

5150
# run tests (PG_CONFIG)
5251
time \
5352
PG_CONFIG=$(which pg_config) \
54-
ALT_CONFIG=1 \
55-
coverage run -a tests/test_simple.py
53+
coverage run -a -m pytest -l -v -n 4 -k "TestgresTests"
5654

5755

5856
# show coverage

‎testgres/plugins/__init__.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
from pg_probackup2.gdb import GDBobj
2-
from pg_probackup2.app import ProbackupApp, ProbackupException
3-
from pg_probackup2.init_helpers import init_params
4-
from pg_probackup2.storage.fs_backup import FSTestBackupDir
1+
from .pg_probackup2.pg_probackup2.gdb import GDBobj
2+
from .pg_probackup2.pg_probackup2.app import ProbackupApp, ProbackupException
3+
from .pg_probackup2.pg_probackup2.init_helpers import init_params
4+
from .pg_probackup2.pg_probackup2.storage.fs_backup import FSTestBackupDir
55

66
__all__ = [
77
"ProbackupApp", "ProbackupException", "init_params", "FSTestBackupDir", "GDBobj"

‎testgres/plugins/pg_probackup2/pg_probackup2/init_helpers.py

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -121,8 +121,7 @@ def __init__(self):
121121
self.probackup_path = probackup_path_tmp
122122

123123
if not self.probackup_path:
124-
logging.error('pg_probackup binary is not found')
125-
exit(1)
124+
raise Exception('pg_probackup binary is not found')
126125

127126
if os.name == 'posix':
128127
self.EXTERNAL_DIRECTORY_DELIMITER = ':'
@@ -213,11 +212,15 @@ def __init__(self):
213212
if self.probackup_version.split('.')[0].isdigit():
214213
self.major_version = int(self.probackup_version.split('.')[0])
215214
else:
216-
logging.error('Can\'t process pg_probackup version \"{}\": the major version is expected to be a number'.format(self.probackup_version))
217-
sys.exit(1)
215+
raise Exception('Can\'t process pg_probackup version \"{}\": the major version is expected to be a number'.format(self.probackup_version))
218216

219217
def test_env(self):
220218
return self._test_env.copy()
221219

222220

223-
init_params = Init()
221+
try:
222+
init_params = Init()
223+
except Exception as e:
224+
logging.error(str(e))
225+
logging.warning("testgres.plugins.probackup2.init_params is set to None.")
226+
init_params = None

‎testgres/plugins/pg_probackup2/pg_probackup2/tests/__init__.py

Whitespace-only changes.

‎testgres/plugins/pg_probackup2/pg_probackup2/tests/basic_test.py

Lines changed: 0 additions & 80 deletions
This file was deleted.

‎testgres/plugins/pg_probackup2/pg_probackup2/tests/test_basic.py

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
from __future__ import annotations
2+
3+
import os
4+
import shutil
5+
import pytest
6+
7+
from ...... import testgres
8+
from ...pg_probackup2.app import ProbackupApp
9+
from ...pg_probackup2.init_helpers import Init, init_params
10+
from ..storage.fs_backup import FSTestBackupDir
11+
12+
13+
class ProbackupTest:
14+
pg_node: testgres.PostgresNode
15+
16+
@staticmethod
17+
def probackup_is_available() -> bool:
18+
p = os.environ.get("PGPROBACKUPBIN")
19+
20+
if p is None:
21+
return False
22+
23+
if not os.path.exists(p):
24+
return False
25+
26+
return True
27+
28+
@pytest.fixture(autouse=True, scope="function")
29+
def implicit_fixture(self, request: pytest.FixtureRequest):
30+
assert isinstance(request, pytest.FixtureRequest)
31+
self.helper__setUp(request)
32+
yield
33+
self.helper__tearDown()
34+
35+
def helper__setUp(self, request: pytest.FixtureRequest):
36+
assert isinstance(request, pytest.FixtureRequest)
37+
38+
self.helper__setup_test_environment(request)
39+
self.helper__setup_test_paths()
40+
self.helper__setup_backup_dir()
41+
self.helper__setup_probackup()
42+
43+
def helper__setup_test_environment(self, request: pytest.FixtureRequest):
44+
assert isinstance(request, pytest.FixtureRequest)
45+
46+
self.output = None
47+
self.cmd = None
48+
self.nodes_to_cleanup = []
49+
self.module_name, self.fname = request.node.cls.__name__, request.node.name
50+
self.test_env = Init().test_env()
51+
52+
def helper__setup_test_paths(self):
53+
self.rel_path = os.path.join(self.module_name, self.fname)
54+
self.test_path = os.path.join(init_params.tmp_path, self.rel_path)
55+
os.makedirs(self.test_path, exist_ok=True)
56+
self.pb_log_path = os.path.join(self.test_path, "pb_log")
57+
58+
def helper__setup_backup_dir(self):
59+
self.backup_dir = self.helper__build_backup_dir('backup')
60+
self.backup_dir.cleanup()
61+
62+
def helper__setup_probackup(self):
63+
self.pg_node = testgres.NodeApp(self.test_path, self.nodes_to_cleanup)
64+
self.pb = ProbackupApp(self, self.pg_node, self.pb_log_path, self.test_env,
65+
auto_compress_alg='zlib', backup_dir=self.backup_dir)
66+
67+
def helper__tearDown(self):
68+
if os.path.exists(self.test_path):
69+
shutil.rmtree(self.test_path)
70+
71+
def helper__build_backup_dir(self, backup='backup'):
72+
return FSTestBackupDir(rel_path=self.rel_path, backup=backup)
73+
74+
75+
@pytest.mark.skipif(not ProbackupTest.probackup_is_available(), reason="Check that PGPROBACKUPBIN is defined and is valid.")
76+
class TestBasic(ProbackupTest):
77+
def test_full_backup(self):
78+
# Setting up a simple test node
79+
node = self.pg_node.make_simple('node', pg_options={"fsync": "off", "synchronous_commit": "off"})
80+
81+
# Initialize and configure Probackup
82+
self.pb.init()
83+
self.pb.add_instance('node', node)
84+
self.pb.set_archiving('node', node)
85+
86+
# Start the node and initialize pgbench
87+
node.slow_start()
88+
node.pgbench_init(scale=100, no_vacuum=True)
89+
90+
# Perform backup and validation
91+
backup_id = self.pb.backup_node('node', node)
92+
out = self.pb.validate('node', backup_id)
93+
94+
# Check if the backup is valid
95+
assert f"INFO: Backup {backup_id} is valid" in out

‎tests/helpers/run_conditions.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
# coding: utf-8
12
import pytest
23
import platform
34

‎tests/test_local.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1+
# coding: utf-8
12
import os
23

34
import pytest
45
import re
56
import tempfile
67

7-
from testgres import ExecUtilException
8-
from testgres import InvalidOperationException
9-
from testgres import LocalOperations
8+
from ..testgres import ExecUtilException
9+
from ..testgres import InvalidOperationException
10+
from ..testgres import LocalOperations
1011

1112
from .helpers.run_conditions import RunConditions
1213

‎tests/test_remote.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
1+
# coding: utf-8
12
import os
23

34
import pytest
45
import re
56
import tempfile
67

7-
from testgres import ExecUtilException
8-
from testgres import InvalidOperationException
9-
from testgres import RemoteOperations
10-
from testgres import ConnectionParams
8+
from ..testgres import ExecUtilException
9+
from ..testgres import InvalidOperationException
10+
from ..testgres import RemoteOperations
11+
from ..testgres import ConnectionParams
1112

1213

1314
class TestRemoteOperations:

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy