Skip to content

Commit 5ce6e74

Browse files
encukouencukou
committed
Add bytes_strictness to allow configuring behavior on bytes/text mismatch
Fixes: #166
1 parent dfbe523 commit 5ce6e74

File tree

4 files changed

+152
-57
lines changed

4 files changed

+152
-57
lines changed

‎Doc/bytes_mode.rst

Lines changed: 31 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -43,37 +43,47 @@ Encoding/decoding to other formats – text, images, etc. – is left to the cal
4343
The bytes mode
4444
--------------
4545

46-
The behavior of python-ldap 3.0 in Python 2 is influenced by a ``bytes_mode``
47-
argument to :func:`ldap.initialize`.
48-
The argument can take these values:
46+
In Python 3, text values are represented as ``str``, the Unicode text type.
4947

50-
``bytes_mode=True``: backwards-compatible
48+
In Python 2, the behavior of python-ldap 3.0 is influenced by a ``bytes_mode``
49+
argument to :func:`ldap.initialize`:
5150

52-
Text values returned from python-ldap are always bytes (``str``).
53-
Text values supplied to python-ldap may be either bytes or Unicode.
54-
The encoding for bytes is always assumed to be UTF-8.
51+
``bytes_mode=True`` (backwards compatible):
52+
Text values are represented as bytes (``str``) encoded using UTF-8.
5553

56-
Not available in Python 3.
54+
``bytes_mode=False`` (future compatible):
55+
Text values are represented as ``unicode``.
5756

58-
``bytes_mode=False``: strictly future-compatible
57+
If not given explicitly, python-ldap will default to ``bytes_mode=True``,
58+
but if an ``unicode`` value supplied to it, if will warn and use that value.
5959

60-
Text values must be represented as ``unicode``.
61-
An error is raised if python-ldap receives a text value as bytes (``str``).
60+
Backwards-compatible behavior is not scheduled for removal until Python 2
61+
itself reaches end of life.
6262

63-
Unspecified: relaxed mode with warnings
6463

65-
Causes a warning on Python 2.
64+
Errors, warnings, and automatic encoding
65+
----------------------------------------
6666

67-
Text values returned from python-ldap are always ``unicode``.
68-
Text values supplied to python-ldap should be ``unicode``;
69-
warnings are emitted when they are not.
67+
While the type of values *returned* from python-ldap is always given by
68+
``bytes_mode``, the behavior for “wrong-type” values *passed in* can be
69+
controlled by the ``bytes_strictness`` argument to :func:`ldap.initialize`:
7070

71-
The warnings are of type :class:`~ldap.LDAPBytesWarning`, which
72-
is a subclass of :class:`BytesWarning` designed to be easily
73-
:ref:`filtered out <filter-bytes-warning>` if needed.
71+
``bytes_strictness='error'`` (default if ``bytes_mode`` is specified):
72+
A ``TypeError`` is raised.
7473

75-
Backwards-compatible behavior is not scheduled for removal until Python 2
76-
itself reaches end of life.
74+
``bytes_strictness='warn'`` (default when ``bytes_mode`` is not given explicitly):
75+
A warning is raised, and the value is encoded/decoded
76+
using the UTF-8 encoding.
77+
78+
The warnings are of type :class:`~ldap.LDAPBytesWarning`, which
79+
is a subclass of :class:`BytesWarning` designed to be easily
80+
:ref:`filtered out <filter-bytes-warning>` if needed.
81+
82+
``bytes_strictness='silent'``:
83+
The value is automatically encoded/decoded using the UTF-8 encoding.
84+
85+
When setting ``bytes_strictness``, an explicit value for ``bytes_mode`` needs
86+
to be given as well.
7787

7888

7989
Porting recommendations

‎Doc/reference/ldap.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Functions
2929

3030
This module defines the following functions:
3131

32-
.. py:function:: initialize(uri [, trace_level=0 [, trace_file=sys.stdout [, trace_stack_limit=None, [bytes_mode=None]]]]) -> LDAPObject object
32+
.. py:function:: initialize(uri [, trace_level=0 [, trace_file=sys.stdout [, trace_stack_limit=None, [bytes_mode=None, [bytes_strictness=None]]]]]) -> LDAPObject object
3333
3434
Initializes a new connection object for accessing the given LDAP server,
3535
and return an LDAP object (see :ref:`ldap-objects`) used to perform operations
@@ -53,7 +53,8 @@ This module defines the following functions:
5353
*trace_file* specifies a file-like object as target of the debug log and
5454
*trace_stack_limit* specifies the stack limit of tracebacks in debug log.
5555

56-
The *bytes_mode* argument specifies text/bytes behavior under Python 2.
56+
The *bytes_mode* and *bytes_strictness* arguments specify text/bytes
57+
behavior under Python 2.
5758
See :ref:`text-bytes` for a complete documentation.
5859

5960
Possible values for *trace_level* are

‎Lib/ldap/ldapobject.py

Lines changed: 52 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,8 @@ class SimpleLDAPObject:
9393

9494
def __init__(
9595
self,uri,
96-
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None
96+
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None,
97+
bytes_strictness=None,
9798
):
9899
self._trace_level = trace_level
99100
self._trace_file = trace_file or sys.stdout
@@ -107,20 +108,26 @@ def __init__(
107108
# Bytes mode
108109
# ----------
109110

110-
# By default, raise a TypeError when receiving invalid args
111-
self.bytes_mode_hardfail = True
112-
if bytes_mode is None and PY2:
113-
_raise_byteswarning(
114-
"Under Python 2, python-ldap uses bytes by default. "
115-
"This will be removed in Python 3 (no bytes for DN/RDN/field names). "
116-
"Please call initialize(..., bytes_mode=False) explicitly.")
117-
bytes_mode = True
118-
# Disable hard failure when running in backwards compatibility mode.
119-
self.bytes_mode_hardfail = False
120-
elif bytes_mode and not PY2:
121-
raise ValueError("bytes_mode is *not* supported under Python 3.")
122-
# On by default on Py2, off on Py3.
111+
if PY2:
112+
if bytes_mode is None:
113+
bytes_mode = True
114+
if bytes_strictness is None:
115+
_raise_byteswarning(
116+
"Under Python 2, python-ldap uses bytes by default. "
117+
"This will be removed in Python 3 (no bytes for "
118+
"DN/RDN/field names). "
119+
"Please call initialize(..., bytes_mode=False) explicitly.")
120+
bytes_strictness = 'warn'
121+
else:
122+
if bytes_strictness is None:
123+
bytes_strictness = 'error'
124+
else:
125+
if bytes_mode:
126+
raise ValueError("bytes_mode is *not* supported under Python 3.")
127+
bytes_mode = False
128+
bytes_strictness = 'error'
123129
self.bytes_mode = bytes_mode
130+
self.bytes_strictness = bytes_strictness
124131

125132
def _bytesify_input(self, arg_name, value):
126133
"""Adapt a value following bytes_mode in Python 2.
@@ -130,32 +137,40 @@ def _bytesify_input(self, arg_name, value):
130137
With bytes_mode ON, takes bytes or None and returns bytes or None.
131138
With bytes_mode OFF, takes unicode or None and returns bytes or None.
132139
133-
This function should be applied on all text inputs (distinguished names
134-
and attribute names in modlists) to convert them to the bytes expected
135-
by the C bindings.
140+
For the wrong argument type (unicode or bytes, respectively),
141+
behavior depends on the bytes_strictness setting.
142+
In all cases, bytes or None are returned (or an exception is raised).
136143
"""
137144
if not PY2:
138145
return value
139-
140146
if value is None:
141147
return value
148+
142149
elif self.bytes_mode:
143150
if isinstance(value, bytes):
144151
return value
145-
else:
146-
if self.bytes_mode_hardfail:
147-
raise TypeError("All provided fields *must* be bytes when bytes mode is on; got %r" % (value,))
148-
else:
149-
_raise_byteswarning(
150-
"Received non-bytes value for '{}' with default (disabled) bytes mode; "
152+
elif self.bytes_strictness == 'silent':
153+
pass
154+
elif self.bytes_strictness == 'warn':
155+
_raise_byteswarning(
156+
"Received non-bytes value for '{}' in bytes mode; "
151157
"please choose an explicit "
152158
"option for bytes_mode on your LDAP connection".format(arg_name))
153-
return value.encode('utf-8')
159+
else:
160+
raise TypeError("All provided fields *must* be bytes when bytes mode is on; got %r" % (value,))
161+
return value.encode('utf-8')
154162
else:
155-
if not isinstance(value, text_type):
163+
if isinstance(value, unicode):
164+
return value.encode('utf-8')
165+
elif self.bytes_strictness == 'silent':
166+
pass
167+
elif self.bytes_strictness == 'warn':
168+
_raise_byteswarning(
169+
"Received non-text value for '{}' with bytes_mode off and "
170+
"bytes_strictness='warn'".format(arg_name))
171+
else:
156172
raise TypeError("All provided fields *must* be text when bytes mode is off; got %r" % (value,))
157-
assert not isinstance(value, bytes)
158-
return value.encode('utf-8')
173+
return value
159174

160175
def _bytesify_modlist(self, arg_name, modlist, with_opcode):
161176
"""Adapt a modlist according to bytes_mode.
@@ -1058,7 +1073,7 @@ class ReconnectLDAPObject(SimpleLDAPObject):
10581073
def __init__(
10591074
self,uri,
10601075
trace_level=0,trace_file=None,trace_stack_limit=5,bytes_mode=None,
1061-
retry_max=1,retry_delay=60.0
1076+
bytes_strictness=None, retry_max=1, retry_delay=60.0
10621077
):
10631078
"""
10641079
Parameters like SimpleLDAPObject.__init__() with these
@@ -1072,7 +1087,9 @@ def __init__(
10721087
self._uri = uri
10731088
self._options = []
10741089
self._last_bind = None
1075-
SimpleLDAPObject.__init__(self,uri,trace_level,trace_file,trace_stack_limit,bytes_mode)
1090+
SimpleLDAPObject.__init__(self, uri, trace_level, trace_file,
1091+
trace_stack_limit, bytes_mode,
1092+
bytes_strictness=bytes_strictness)
10761093
self._reconnect_lock = ldap.LDAPLock(desc='reconnect lock within %s' % (repr(self)))
10771094
self._retry_max = retry_max
10781095
self._retry_delay = retry_delay
@@ -1091,6 +1108,11 @@ def __getstate__(self):
10911108

10921109
def __setstate__(self,d):
10931110
"""set up the object from pickled data"""
1111+
hardfail = d.get('bytes_mode_hardfail')
1112+
if hardfail:
1113+
d.setdefault('bytes_strictness', 'error')
1114+
else:
1115+
d.setdefault('bytes_strictness', 'warn')
10941116
self.__dict__.update(d)
10951117
self._last_bind = getattr(SimpleLDAPObject, self._last_bind[0]), self._last_bind[1], self._last_bind[2]
10961118
self._ldap_object_lock = self._ldap_lock()

‎Tests/t_ldapobject.py

Lines changed: 66 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -126,9 +126,9 @@ def test_search_keys_are_text(self):
126126
for value in values:
127127
self.assertEqual(type(value), bytes)
128128

129-
def _get_bytes_ldapobject(self, explicit=True):
129+
def _get_bytes_ldapobject(self, explicit=True, **kwargs):
130130
if explicit:
131-
kwargs = {'bytes_mode': True}
131+
kwargs.setdefault('bytes_mode', True)
132132
else:
133133
kwargs = {}
134134
return self._open_ldap_conn(
@@ -195,6 +195,68 @@ def test_unset_bytesmode_search_warns_bytes(self):
195195
l.search_s(base.encode('utf-8'), ldap.SCOPE_SUBTREE, b'(cn=Foo*)', ['*'])
196196
l.search_s(base, ldap.SCOPE_SUBTREE, b'(cn=Foo*)', [b'*'])
197197

198+
def _search_wrong_type(self, bytes_mode, strictness):
199+
if bytes_mode:
200+
l = self._get_bytes_ldapobject(bytes_strictness=strictness)
201+
else:
202+
l = self._open_ldap_conn(bytes_mode=False,
203+
bytes_strictness=strictness)
204+
base = 'cn=Foo1,' + self.server.suffix
205+
if not bytes_mode:
206+
base = base.encode('utf-8')
207+
result = l.search_s(base, scope=ldap.SCOPE_SUBTREE)
208+
return result[0][-1]['cn']
209+
210+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
211+
def test_bytesmode_silent(self):
212+
with warnings.catch_warnings(record=True) as w:
213+
warnings.resetwarnings()
214+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
215+
self._search_wrong_type(bytes_mode=True, strictness='silent')
216+
self.assertEqual(w, [])
217+
218+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
219+
def test_bytesmode_warn(self):
220+
with warnings.catch_warnings(record=True) as w:
221+
warnings.resetwarnings()
222+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
223+
self._search_wrong_type(bytes_mode=True, strictness='warn')
224+
self.assertEqual(len(w), 1)
225+
226+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
227+
def test_bytesmode_error(self):
228+
with warnings.catch_warnings(record=True) as w:
229+
warnings.resetwarnings()
230+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
231+
with self.assertRaises(TypeError):
232+
self._search_wrong_type(bytes_mode=True, strictness='error')
233+
self.assertEqual(w, [])
234+
235+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
236+
def test_textmode_silent(self):
237+
with warnings.catch_warnings(record=True) as w:
238+
warnings.resetwarnings()
239+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
240+
self._search_wrong_type(bytes_mode=True, strictness='silent')
241+
self.assertEqual(w, [])
242+
243+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
244+
def test_textmode_warn(self):
245+
with warnings.catch_warnings(record=True) as w:
246+
warnings.resetwarnings()
247+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
248+
self._search_wrong_type(bytes_mode=True, strictness='warn')
249+
self.assertEqual(len(w), 1)
250+
251+
@unittest.skipUnless(PY2, "no bytes_mode under Py3")
252+
def test_textmode_error(self):
253+
with warnings.catch_warnings(record=True) as w:
254+
warnings.resetwarnings()
255+
warnings.simplefilter('always', ldap.LDAPBytesWarning)
256+
with self.assertRaises(TypeError):
257+
self._search_wrong_type(bytes_mode=True, strictness='error')
258+
self.assertEqual(w, [])
259+
198260
def test_search_accepts_unicode_dn(self):
199261
base = self.server.suffix
200262
l = self._ldap_conn
@@ -434,7 +496,7 @@ def test_ldapbyteswarning(self):
434496
self.assertIs(msg.category, ldap.LDAPBytesWarning)
435497
self.assertEqual(
436498
text_type(msg.message),
437-
"Received non-bytes value for 'base' with default (disabled) bytes "
499+
"Received non-bytes value for 'base' in bytes "
438500
"mode; please choose an explicit option for bytes_mode on your "
439501
"LDAP connection"
440502
)
@@ -596,7 +658,7 @@ def test103_reconnect_get_state(self):
596658
str('_trace_stack_limit'): 5,
597659
str('_uri'): self.server.ldap_uri,
598660
str('bytes_mode'): l1.bytes_mode,
599-
str('bytes_mode_hardfail'): l1.bytes_mode_hardfail,
661+
str('bytes_strictness'): l1.bytes_strictness,
600662
str('timeout'): -1,
601663
},
602664
)

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy