-
-
Notifications
You must be signed in to change notification settings - Fork 11.1k
FIX: np.insert fails with datetime64 and string input combination #29339 #29408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
FIX: np.insert fails with datetime64 and string input combination #29339 #29408
Conversation
…in np.insert (numpy#29339) - Fix np.insert to correctly convert 0-dim np.datetime64 arrays to strings when inserting into string arrays. This resolves RuntimeError when inserting datetime64 wrapped in 0-d arrays.
The correct place to fix this is almost certainly a lower-level C function rather than adding a special case to the high-level wrapper. The RuntimeError is coming from |
I would think that string casts just behave different from item coercion. FWIW, I am not sure I am convinced we should even make this work. Is it really useful to allow such unclear conversions to begin with? |
Apparently I made it work when I added support for It'd be nice if there wasn't this weird inconsistency between |
I am not sure I am following. I thought what is triggering is that one path for the old string dtype fails because the date can't be correctly inserted (it gets truncated), while the scalar assignment doesn't. But, I am not sure why that matters for |
Ah, that makes more sense, I didn't understand the subtlety that there was a truncation issue going on. I agree then, there might not be anything to fix. |
Hi @ngoldbaum and @seberg, Thank you for your feedback, and I apologize if my previous changes introduced any ambiguity. Let me clarify the core issue:
Although with unwrapped np.datetime64, the result is problematic due to truncation:
My initial approach was to make 0D arrays behave like scalars (to fix the inconsistency), but I understand the concern about whether this should be handled at the C level or avoided entirely. Is this logic acceptable as it is, or would it be better to move it to the C layer (e.g., NpyDatetime_MakeISO8601Datetime)? Or is this behavior inherently undesirable? Please let me know If we do fix it, should we also address the truncation issue, or deprecate such conversions altogether? |
Yes, exactly — since As @AnkitAhlawat7742 pointed out, the issue occurs specifically when inserting a 0D |
My point is that I generally don't like the truncating behavior much and I don't think it serves anyone making it work more often. The error seems usually like the better thing. This is a rare code path, data is lost, so forcing users to jump through hoops and casting themselves is good. I would be fine with deprecating the scalar assignment to strings here. While it is a discussion that we should probably have, it is a bigger discussion probably, because it is what we do with most scalar assignments to strings right now. Mainly, I am not convinced about moving towards being more forgiving here when being forgiving looks like a bug magnet. |
What was the Issue
Issue link: #29339
Bug Description: numpy.insert raised a RuntimeError when inserting a zero-dimensional np.datetime64 wrapped inside a zero-dimensional array into a string scalar or string array. This behavior is inconsistent because similar operations with datetime64 scalars work correctly, and the inserted values are expected to be converted to the string type of the destination array.
What This PR Fixes
This pull request fixes the issue for the case when inserting 0-dimensional np.datetime64 arrays into string scalars. It converts such 0-dim datetime64 arrays to their string representation before insertion, preventing the RuntimeError.
Notes
The fix ensures consistent behavior between inserting datetime64 scalars and 0-dim arrays containing datetime64.
This fix applies specifically to insertions into string-type arrays or scalars.
Additional tests have been added to cover these cases.