Skip to content

Commit 786d2d8

Browse files
mtsokolmtsokol
committed
np.array: call __array__ with copy keyword
1 parent 118f8e7 commit 786d2d8

33 files changed

+111
-91
lines changed

‎doc/source/reference/arrays.classes.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -305,7 +305,7 @@ NumPy provides several hooks that classes can customize:
305305
.. note:: For ufuncs, it is hoped to eventually deprecate this method in
306306
favour of :func:`__array_ufunc__`.
307307

308-
.. py:method:: class.__array__([dtype])
308+
.. py:method:: class.__array__(dtype=None, copy=None)
309309
310310
If defined on an object, should return an ``ndarray``.
311311
This method is called by array-coercion functions like np.array()

‎doc/source/reference/c-api/array.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -515,7 +515,8 @@ From other objects
515515
516516
Return an ndarray object from a Python object that exposes the
517517
:obj:`~numpy.class.__array__` method. The :obj:`~numpy.class.__array__`
518-
method can take 0, or 1 argument ``([dtype])``. ``context`` is unused.
518+
method can take 0, 1 or 2 arguments ``(dtype=None, copy=None)``.
519+
``context`` is unused.
519520
520521
.. c:function:: PyObject* PyArray_ContiguousFromAny( \
521522
PyObject* op, int typenum, int min_depth, int max_depth)

‎doc/source/user/basics.dispatch.rst

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ example that has rather narrow utility but illustrates the concepts involved.
2222
... self._i = value
2323
... def __repr__(self):
2424
... return f"{self.__class__.__name__}(N={self._N}, value={self._i})"
25-
... def __array__(self, dtype=None):
25+
... def __array__(self, dtype=None, copy=None):
2626
... return self._i * np.eye(self._N, dtype=dtype)
2727

2828
Our custom array can be instantiated like:
@@ -84,7 +84,7 @@ For this example we will only handle the method ``__call__``
8484
... self._i = value
8585
... def __repr__(self):
8686
... return f"{self.__class__.__name__}(N={self._N}, value={self._i})"
87-
... def __array__(self, dtype=None):
87+
... def __array__(self, dtype=None, copy=None):
8888
... return self._i * np.eye(self._N, dtype=dtype)
8989
... def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
9090
... if method == '__call__':
@@ -135,7 +135,7 @@ conveniently by inheriting from the mixin
135135
... self._i = value
136136
... def __repr__(self):
137137
... return f"{self.__class__.__name__}(N={self._N}, value={self._i})"
138-
... def __array__(self, dtype=None):
138+
... def __array__(self, dtype=None, copy=None):
139139
... return self._i * np.eye(self._N, dtype=dtype)
140140
... def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
141141
... if method == '__call__':
@@ -173,7 +173,7 @@ functions to our custom variants.
173173
... self._i = value
174174
... def __repr__(self):
175175
... return f"{self.__class__.__name__}(N={self._N}, value={self._i})"
176-
... def __array__(self, dtype=None):
176+
... def __array__(self, dtype=None, copy=None):
177177
... return self._i * np.eye(self._N, dtype=dtype)
178178
... def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
179179
... if method == '__call__':
@@ -306,4 +306,3 @@ Refer to the `dask source code <https://github.com/dask/dask>`_ and
306306
examples of custom array containers.
307307

308308
See also :doc:`NEP 18<neps:nep-0018-array-function-protocol>`.
309-

‎doc/source/user/basics.interoperability.rst

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,8 @@ relies on the existence of the following attributes or methods:
7575
- ``__array_interface__``: a Python dictionary containing the shape, the
7676
element type, and optionally, the data buffer address and the strides of an
7777
array-like object;
78-
- ``__array__()``: a method returning the NumPy ndarray view of an array-like
79-
object;
78+
- ``__array__()``: a method returning the NumPy ndarray copy or a view of an
79+
array-like object;
8080

8181
The ``__array_interface__`` attribute can be inspected directly:
8282

@@ -125,6 +125,16 @@ new ndarray object. This is not optimal, as coercing arrays into ndarrays may
125125
cause performance problems or create the need for copies and loss of metadata,
126126
as the original object and any attributes/behavior it may have had, is lost.
127127

128+
The signature of the method should be ``__array__(self, dtype=None, copy=None)``.
129+
If a passed ``dtype`` isn't ``None`` and different than the object's data type,
130+
a casting should happen to a specified type. If ``copy`` is ``None``, a copy
131+
should be made only if ``dtype`` argument enforces it. For ``copy=True``, a copy
132+
should always be made, where ``copy=False`` should raise an exception if a copy
133+
is needed.
134+
135+
If a class implements the old signature ``__array__(self)``, for ``np.array(a)``
136+
a warning will be raised saying that ``dtype`` and ``copy`` arguments are missing.
137+
128138
To see an example of a custom array implementation including the use of
129139
``__array__()``, see :ref:`basics.dispatch`.
130140

‎numpy/__init__.pyi

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1464,9 +1464,13 @@ class ndarray(_ArrayOrScalarCommon, Generic[_ShapeType, _DType_co]):
14641464
def __class_getitem__(self, item: Any) -> GenericAlias: ...
14651465

14661466
@overload
1467-
def __array__(self, dtype: None = ..., /) -> ndarray[Any, _DType_co]: ...
1467+
def __array__(
1468+
self, dtype: None = ..., /, *, copy: None | bool = ...
1469+
) -> ndarray[Any, _DType_co]: ...
14681470
@overload
1469-
def __array__(self, dtype: _DType, /) -> ndarray[Any, _DType]: ...
1471+
def __array__(
1472+
self, dtype: _DType, /, *, copy: None | bool = ...
1473+
) -> ndarray[Any, _DType]: ...
14701474

14711475
def __array_ufunc__(
14721476
self,
@@ -3715,9 +3719,9 @@ class poly1d:
37153719
__hash__: ClassVar[None] # type: ignore
37163720

37173721
@overload
3718-
def __array__(self, t: None = ...) -> NDArray[Any]: ...
3722+
def __array__(self, t: None = ..., copy: None | bool = ...) -> NDArray[Any]: ...
37193723
@overload
3720-
def __array__(self, t: _DType) -> ndarray[Any, _DType]: ...
3724+
def __array__(self, t: _DType, copy: None | bool = ...) -> ndarray[Any, _DType]: ...
37213725

37223726
@overload
37233727
def __call__(self, val: _ScalarLike_co) -> Any: ...

‎numpy/_core/defchararray.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1097,7 +1097,7 @@ class adds the following functionality:
10971097
10981098
copy : bool, optional
10991099
If true (default), then the object is copied. Otherwise, a copy
1100-
will only be made if __array__ returns a copy, if obj is a
1100+
will only be made if ``__array__`` returns a copy, if obj is a
11011101
nested sequence, or if a copy is needed to satisfy any of the other
11021102
requirements (`itemsize`, unicode, `order`, etc.).
11031103

‎numpy/_core/src/multiarray/ctors.c

Lines changed: 29 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1477,16 +1477,7 @@ _array_from_array_like(PyObject *op,
14771477
}
14781478
}
14791479

1480-
/*
1481-
* If op supplies the __array__ function.
1482-
* The documentation says this should produce a copy, so
1483-
* we skip this method if writeable is true, because the intent
1484-
* of writeable is to modify the operand.
1485-
* XXX: If the implementation is wrong, and/or if actual
1486-
* usage requires this behave differently,
1487-
* this should be changed!
1488-
*/
1489-
if (!writeable && tmp == Py_NotImplemented) {
1480+
if (tmp == Py_NotImplemented) {
14901481
tmp = PyArray_FromArrayAttr_int(op, requested_dtype, never_copy);
14911482
if (tmp == NULL) {
14921483
return NULL;
@@ -2418,9 +2409,8 @@ PyArray_FromInterface(PyObject *origin)
24182409
* @param descr The desired `arr.dtype`, passed into the `__array__` call,
24192410
* as information but is not checked/enforced!
24202411
* @param never_copy Specifies that a copy is not allowed.
2421-
* NOTE: Currently, this means an error is raised instead of calling
2422-
* `op.__array__()`. In the future we could call for example call
2423-
* `op.__array__(never_copy=True)` instead.
2412+
* NOTE: For false it passes `op.__array__(copy=None)`,
2413+
* for true: `op.__array__(copy=False)`.
24242414
* @returns NotImplemented if `__array__` is not defined or a NumPy array
24252415
* (or subclass). On error, return NULL.
24262416
*/
@@ -2438,15 +2428,6 @@ PyArray_FromArrayAttr_int(
24382428
}
24392429
return Py_NotImplemented;
24402430
}
2441-
if (never_copy) {
2442-
/* Currently, we must always assume that `__array__` returns a copy */
2443-
PyErr_SetString(PyExc_ValueError,
2444-
"Unable to avoid copy while converting from an object "
2445-
"implementing the `__array__` protocol. NumPy cannot ensure "
2446-
"that no copy will be made.");
2447-
Py_DECREF(array_meth);
2448-
return NULL;
2449-
}
24502431

24512432
if (PyType_Check(op) && PyObject_HasAttrString(array_meth, "__get__")) {
24522433
/*
@@ -2458,12 +2439,33 @@ PyArray_FromArrayAttr_int(
24582439
Py_DECREF(array_meth);
24592440
return Py_NotImplemented;
24602441
}
2461-
if (descr == NULL) {
2462-
new = PyObject_CallFunction(array_meth, NULL);
2463-
}
2464-
else {
2465-
new = PyObject_CallFunction(array_meth, "O", descr);
2442+
2443+
PyObject *copy = never_copy ? Py_False : Py_None;
2444+
PyObject *kwargs = PyDict_New();
2445+
PyDict_SetItemString(kwargs, "copy", copy);
2446+
PyObject *args = descr != NULL ? PyTuple_Pack(1, descr) : PyTuple_New(0);
2447+
2448+
new = PyObject_Call(array_meth, args, kwargs);
2449+
2450+
if (PyErr_Occurred()) {
2451+
PyObject *type, *value, *traceback;
2452+
PyErr_Fetch(&type, &value, &traceback);
2453+
if (PyUnicode_Check(value) && PyUnicode_CompareWithASCIIString(value,
2454+
"__array__() got an unexpected keyword argument 'copy'") == 0) {
2455+
Py_DECREF(type);
2456+
Py_XDECREF(value);
2457+
Py_XDECREF(traceback);
2458+
if (PyErr_WarnEx(PyExc_UserWarning,
2459+
"__array__ should implement 'dtype' and 'copy' keywords", 1) < 0) {
2460+
return NULL;
2461+
}
2462+
new = PyObject_Call(array_meth, args, NULL);
2463+
} else {
2464+
PyErr_Restore(type, value, traceback);
2465+
return NULL;
2466+
}
24662467
}
2468+
24672469
Py_DECREF(array_meth);
24682470
if (new == NULL) {
24692471
return NULL;

‎numpy/_core/src/multiarray/iterators.c

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1071,7 +1071,7 @@ static PyMappingMethods iter_as_mapping = {
10711071
* ignored.
10721072
*/
10731073
static PyArrayObject *
1074-
iter_array(PyArrayIterObject *it, PyObject *NPY_UNUSED(op))
1074+
iter_array(PyArrayIterObject *it, PyObject *NPY_UNUSED(args), PyObject *NPY_UNUSED(kwds))
10751075
{
10761076

10771077
PyArrayObject *ret;
@@ -1120,7 +1120,7 @@ static PyMethodDef iter_methods[] = {
11201120
/* to get array */
11211121
{"__array__",
11221122
(PyCFunction)iter_array,
1123-
METH_VARARGS, NULL},
1123+
METH_VARARGS | METH_KEYWORDS, NULL},
11241124
{"copy",
11251125
(PyCFunction)iter_copy,
11261126
METH_VARARGS, NULL},
@@ -1132,7 +1132,7 @@ iter_richcompare(PyArrayIterObject *self, PyObject *other, int cmp_op)
11321132
{
11331133
PyArrayObject *new;
11341134
PyObject *ret;
1135-
new = (PyArrayObject *)iter_array(self, NULL);
1135+
new = (PyArrayObject *)iter_array(self, NULL, NULL);
11361136
if (new == NULL) {
11371137
return NULL;
11381138
}

‎numpy/_core/src/multiarray/methods.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -928,7 +928,7 @@ array_getarray(PyArrayObject *self, PyObject *args, PyObject *kwds)
928928
{
929929
PyArray_Descr *newtype = NULL;
930930
NPY_COPYMODE copy = NPY_COPY_IF_NEEDED;
931-
static char *kwlist[] = {"", "copy", NULL};
931+
static char *kwlist[] = {"dtype", "copy", NULL};
932932
PyObject *ret;
933933

934934
if (!PyArg_ParseTupleAndKeywords(args, kwds, "|O&$O&:__array__", kwlist,
@@ -981,6 +981,7 @@ array_getarray(PyArrayObject *self, PyObject *args, PyObject *kwds)
981981
} else { // copy == NPY_COPY_NEVER
982982
PyErr_SetString(PyExc_ValueError,
983983
"Unable to avoid copy while creating an array from given array.");
984+
Py_DECREF(self);
984985
return NULL;
985986
}
986987
}

‎numpy/_core/src/multiarray/multiarraymodule.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1655,7 +1655,7 @@ _array_fromobject_generic(
16551655
if (copy == NPY_COPY_ALWAYS) {
16561656
flags = NPY_ARRAY_ENSURECOPY;
16571657
}
1658-
else if (copy == NPY_COPY_NEVER ) {
1658+
else if (copy == NPY_COPY_NEVER) {
16591659
flags = NPY_ARRAY_ENSURENOCOPY;
16601660
}
16611661
if (order == NPY_CORDER) {

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy