Skip to content

Commit 1b17e85

Browse files
encukouencukou
committed
Add bytes_strictness to allow configuring behavior on bytes/text mismatch
Fixes: #166
1 parent e148184 commit 1b17e85

File tree

4 files changed

+152
-57
lines changed

4 files changed

+152
-57
lines changed

‎Doc/bytes_mode.rst

Lines changed: 31 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -43,37 +43,47 @@ Encoding/decoding to other formats – text, images, etc. – is left to the cal
4343
The bytes mode
4444
--------------
4545

46-
The behavior of python-ldap 3.0 in Python 2 is influenced by a ``bytes_mode``
47-
argument to :func:`ldap.initialize`.
48-
The argument can take these values:
46+
In Python 3, text values are represented as ``str``, the Unicode text type.
4947

50-
``bytes_mode=True``: backwards-compatible
48+
In Python 2, the behavior of python-ldap 3.0 is influenced by a ``bytes_mode``
49+
argument to :func:`ldap.initialize`:
5150

52-
Text values returned from python-ldap are always bytes (``str``).
53-
Text values supplied to python-ldap may be either bytes or Unicode.
54-
The encoding for bytes is always assumed to be UTF-8.
51+
``bytes_mode=True`` (backwards compatible):
52+
Text values are represented as bytes (``str``) encoded using UTF-8.
5553

56-
Not available in Python 3.
54+
``bytes_mode=False`` (future compatible):
55+
Text values are represented as ``unicode``.
5756

58-
``bytes_mode=False``: strictly future-compatible
57+
If not given explicitly, python-ldap will default to ``bytes_mode=True``,
58+
but if an ``unicode`` value supplied to it, if will warn and use that value.
5959

60-
Text values must be represented as ``unicode``.
61-
An error is raised if python-ldap receives a text value as bytes (``str``).
60+
Backwards-compatible behavior is not scheduled for removal until Python 2
61+
itself reaches end of life.
6262

63-
Unspecified: relaxed mode with warnings
6463

65-
Causes a warning on Python 2.
64+
Errors, warnings, and automatic encoding
65+
----------------------------------------
6666

67-
Text values returned from python-ldap are always ``unicode``.
68-
Text values supplied to python-ldap should be ``unicode``;
69-
warnings are emitted when they are not.
67+
While the type of values *returned* from python-ldap is always given by
68+
``bytes_mode``, the behavior for “wrong-type” values *passed in* can be
69+
controlled by the ``bytes_strictness`` argument to :func:`ldap.initialize`:
7070

71-
The warnings are of type :class:`~ldap.LDAPBytesWarning`, which
72-
is a subclass of :class:`BytesWarning` designed to be easily
73-
:ref:`filtered out <filter-bytes-warning>` if needed.
71+
``bytes_strictness='error'`` (default if ``bytes_mode`` is specified):
72+
A ``TypeError`` is raised.
7473

75-
Backwards-compatible behavior is not scheduled for removal until Python 2
76-
itself reaches end of life.
74+
``bytes_strictness='warn'`` (default when ``bytes_mode`` is not given explicitly):
75+
A warning is raised, and the value is encoded/decoded
76+
using the UTF-8 encoding.
77+
78+
The warnings are of type :class:`~ldap.LDAPBytesWarning`, which
79+
is a subclass of :class:`BytesWarning` designed to be easily
80+
:ref:`filtered out <filter-bytes-warning>` if needed.
81+
82+
``bytes_strictness='silent'``:
83+
The value is automatically encoded/decoded using the UTF-8 encoding.
84+
85+
When setting ``bytes_strictness``, an explicit value for ``bytes_mode`` needs
86+
to be given as well.
7787

7888

7989
Porting recommendations

‎Doc/reference/ldap.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Functions
2929

3030
This module defines the following functions:
3131

32-
.. py:function:: initialize(uri [, trace_level=0 [, trace_file=sys.stdout [, trace_stack_limit=None, [bytes_mode=None]]]]) -> LDAPObject object
32+
.. py:function:: initialize(uri [, trace_level=0 [, trace_file=sys.stdout [, trace_stack_limit=None, [bytes_mode=None, [bytes_strictness=None]]]]]) -> LDAPObject object
3333
3434
Initializes a new connection object for accessing the given LDAP server,
3535
and return an LDAP object (see :ref:`ldap-objects`) used to perform operations
@@ -53,7 +53,8 @@ This module defines the following functions:
5353
*trace_file* specifies a file-like object as target of the debug log and
5454
*trace_stack_limit* specifies the stack limit of tracebacks in debug log.
5555

56-
The *bytes_mode* argument specifies text/bytes behavior under Python 2.
56+
The *bytes_mode* and *bytes_strictness* arguments specify text/bytes
57+
behavior under Python 2.
5758
See :ref:`text-bytes` for a complete documentation.
5859

5960
Possible values for *trace_level* are

‎Lib/ldap/ldapobject.py

Lines changed: 52 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,8 @@ class SimpleLDAPObject:
9393

9494
def __init__(
9595
self,uri,
96-
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None
96+
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None,
97+
bytes_strictness=None,
9798
):
9899
self._trace_level = trace_level
99100
self._trace_file = trace_file or sys.stdout
@@ -107,20 +108,26 @@ def __init__(
107108
# Bytes mode
108109
# ----------
109110

110-
# By default, raise a TypeError when receiving invalid args
111-
self.bytes_mode_hardfail = True
112-
if bytes_mode is None and PY2:
113-
_raise_byteswarning(
114-
"Under Python 2, python-ldap uses bytes by default. "
115-
"This will be removed in Python 3 (no bytes for DN/RDN/field names). "
116-
"Please call initialize(..., bytes_mode=False) explicitly.")
117-
bytes_mode = True
118-
# Disable hard failure when running in backwards compatibility mode.
119-
self.bytes_mode_hardfail = False
120-
elif bytes_mode and not PY2:
121-
raise ValueError("bytes_mode is *not* supported under Python 3.")
122-
# On by default on Py2, off on Py3.
111+
if PY2:
112+
if bytes_mode is None:
113+
bytes_mode = True
114+
if bytes_strictness is None:
115+
_raise_byteswarning(
116+
"Under Python 2, python-ldap uses bytes by default. "
117+
"This will be removed in Python 3 (no bytes for "
118+
"DN/RDN/field names). "
119+
"Please call initialize(..., bytes_mode=False) explicitly.")
120+
bytes_strictness = 'warn'
121+
else:
122+
if bytes_strictness is None:
123+
bytes_strictness = 'error'
124+
else:
125+
if bytes_mode:
126+
raise ValueError("bytes_mode is *not* supported under Python 3.")
127+
bytes_mode = False
128+
bytes_strictness = 'error'
123129
self.bytes_mode = bytes_mode
130+
self.bytes_strictness = bytes_strictness
124131

125132
def _bytesify_input(self, arg_name, value):
126133
"""Adapt a value following bytes_mode in Python 2.
@@ -130,38 +137,46 @@ def _bytesify_input(self, arg_name, value):
130137
With bytes_mode ON, takes bytes or None and returns bytes or None.
131138
With bytes_mode OFF, takes unicode or None and returns bytes or None.
132139
133-
This function should be applied on all text inputs (distinguished names
134-
and attribute names in modlists) to convert them to the bytes expected
135-
by the C bindings.
140+
For the wrong argument type (unicode or bytes, respectively),
141+
behavior depends on the bytes_strictness setting.
142+
In all cases, bytes or None are returned (or an exception is raised).
136143
"""
137144
if not PY2:
138145
return value
139-
140146
if value is None:
141147
return value
148+
142149
elif self.bytes_mode:
143150
if isinstance(value, bytes):
144151
return value
152+
elif self.bytes_strictness == 'silent':
153+
pass
154+
elif self.bytes_strictness == 'warn':
155+
_raise_byteswarning(
156+
"Received non-bytes value for '{}' in bytes mode; "
157+
"please choose an explicit "
158+
"option for bytes_mode on your LDAP connection".format(arg_name))
145159
else:
146-
if self.bytes_mode_hardfail:
147160
raise TypeError(
148161
"All provided fields *must* be bytes when bytes mode is on; "
149162
"got type '{}' for '{}'.".format(type(value).__name__, arg_name)
150163
)
151-
else:
152-
_raise_byteswarning(
153-
"Received non-bytes value for '{}' with default (disabled) bytes mode; "
154-
"please choose an explicit "
155-
"option for bytes_mode on your LDAP connection".format(arg_name))
156-
return value.encode('utf-8')
164+
return value.encode('utf-8')
157165
else:
158-
if not isinstance(value, text_type):
166+
if isinstance(value, unicode):
167+
return value.encode('utf-8')
168+
elif self.bytes_strictness == 'silent':
169+
pass
170+
elif self.bytes_strictness == 'warn':
171+
_raise_byteswarning(
172+
"Received non-text value for '{}' with bytes_mode off and "
173+
"bytes_strictness='warn'".format(arg_name))
174+
else:
159175
raise TypeError(
160176
"All provided fields *must* be text when bytes mode is off; "
161177
"got type '{}' for '{}'.".format(type(value).__name__, arg_name)
162178
)
163-
assert not isinstance(value, bytes)
164-
return value.encode('utf-8')
179+
return value
165180

166181
def _bytesify_modlist(self, arg_name, modlist, with_opcode):
167182
"""Adapt a modlist according to bytes_mode.
@@ -1064,7 +1079,7 @@ class ReconnectLDAPObject(SimpleLDAPObject):
10641079
def __init__(
10651080
self,uri,
10661081
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None,
1067-
retry_max=1,retry_delay=60.0
1082+
bytes_strictness=None, retry_max=1, retry_delay=60.0
10681083
):
10691084
"""
10701085
Parameters like SimpleLDAPObject.__init__() with these
@@ -1078,7 +1093,9 @@ def __init__(
10781093
self._uri = uri
10791094
self._options = []
10801095
self._last_bind = None
1081-
SimpleLDAPObject.__init__(self,uri,trace_level,trace_file,trace_stack_limit,bytes_mode)
1096+
SimpleLDAPObject.__init__(self, uri, trace_level, trace_file,
1097+
trace_stack_limit, bytes_mode,
1098+
bytes_strictness=bytes_strictness)
10821099
self._reconnect_lock = ldap.LDAPLock(desc='reconnect lock within %s' % (repr(self)))
10831100
self._retry_max = retry_max
10841101
self._retry_delay = retry_delay
@@ -1097,6 +1114,11 @@ def __getstate__(self):
10971114

10981115
def __setstate__(self,d):
10991116
"""set up the object from pickled data"""
1117+
hardfail = d.get('bytes_mode_hardfail')
1118+
if hardfail:
1119+
d.setdefault('bytes_strictness', 'error')
1120+
else:
1121+
d.setdefault('bytes_strictness', 'warn')
11001122
self.__dict__.update(d)
11011123
self._last_bind = getattr(SimpleLDAPObject, self._last_bind[0]), self._last_bind[1], self._last_bind[2]
11021124
self._ldap_object_lock = self._ldap_lock()

‎Tests/t_ldapobject.py

Lines changed: 66 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -162,9 +162,9 @@ def test_search_keys_are_text(self):
162162
for value in values:
163163
self.assertEqual(type(value), bytes)
164164

165-
def _get_bytes_ldapobject(self, explicit=True):
165+
def _get_bytes_ldapobject(self, explicit=True, **kwargs):
166166
if explicit:
167-
kwargs = {'bytes_mode': True}
167+
kwargs.setdefault('bytes_mode', True)
168168
else:
169169
kwargs = {}
170170
return self._open_ldap_conn(
@@ -231,6 +231,68 @@ def test_unset_bytesmode_search_warns_bytes(self):
231231
l.search_s(base.encode('utf-8'), ldap.SCOPE_SUBTREE, b'(cn=Foo*)', ['*'])
232232
l.search_s(base, ldap.SCOPE_SUBTREE, b'(cn=Foo*)', [b'*'])
233233

234+
def _search_wrong_type(self, bytes_mode, strictness):
235+
if bytes_mode:
236+
l = self._get_bytes_ldapobject(bytes_strictness=strictness)
237+
else:
238+
l = self._open_ldap_conn(bytes_mode=False,
239+
bytes_strictness=strictness)
240+
base = 'cn=Foo1,' + self.server.suffix
241+
if not bytes_mode:
242+
base = base.encode('utf-8')
243+
result = l.search_s(base, scope=ldap.SCOPE_SUBTREE)
244+
return result[0][-1]['cn']
245+
246+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
247+
def test_bytesmode_silent(self):
248+
with warnings.catch_warnings(record=True) as w:
249+
warnings.resetwarnings()
250+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
251+
self._search_wrong_type(bytes_mode=True, strictness='silent')
252+
self.assertEqual(w, [])
253+
254+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
255+
def test_bytesmode_warn(self):
256+
with warnings.catch_warnings(record=True) as w:
257+
warnings.resetwarnings()
258+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
259+
self._search_wrong_type(bytes_mode=True, strictness='warn')
260+
self.assertEqual(len(w), 1)
261+
262+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
263+
def test_bytesmode_error(self):
264+
with warnings.catch_warnings(record=True) as w:
265+
warnings.resetwarnings()
266+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
267+
with self.assertRaises(TypeError):
268+
self._search_wrong_type(bytes_mode=True, strictness='error')
269+
self.assertEqual(w, [])
270+
271+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
272+
def test_textmode_silent(self):
273+
with warnings.catch_warnings(record=True) as w:
274+
warnings.resetwarnings()
275+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
276+
self._search_wrong_type(bytes_mode=True, strictness='silent')
277+
self.assertEqual(w, [])
278+
279+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
280+
def test_textmode_warn(self):
281+
with warnings.catch_warnings(record=True) as w:
282+
warnings.resetwarnings()
283+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
284+
self._search_wrong_type(bytes_mode=True, strictness='warn')
285+
self.assertEqual(len(w), 1)
286+
287+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
288+
def test_textmode_error(self):
289+
with warnings.catch_warnings(record=True) as w:
290+
warnings.resetwarnings()
291+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
292+
with self.assertRaises(TypeError):
293+
self._search_wrong_type(bytes_mode=True, strictness='error')
294+
self.assertEqual(w, [])
295+
234296
def test_search_accepts_unicode_dn(self):
235297
base = self.server.suffix
236298
l = self._ldap_conn
@@ -470,7 +532,7 @@ def test_ldapbyteswarning(self):
470532
self.assertIs(msg.category, ldap.LDAPBytesWarning)
471533
self.assertEqual(
472534
text_type(msg.message),
473-
"Received non-bytes value for 'base' with default (disabled) bytes "
535+
"Received non-bytes value for 'base' in bytes "
474536
"mode; please choose an explicit option for bytes_mode on your "
475537
"LDAP connection"
476538
)
@@ -632,7 +694,7 @@ def test103_reconnect_get_state(self):
632694
str('_trace_stack_limit'): 5,
633695
str('_uri'): self.server.ldap_uri,
634696
str('bytes_mode'): l1.bytes_mode,
635-
str('bytes_mode_hardfail'): l1.bytes_mode_hardfail,
697+
str('bytes_strictness'): l1.bytes_strictness,
636698
str('timeout'): -1,
637699
},
638700
)

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy