Skip to content

gh-116738: Make _csv module thread-safe #118344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Oct 11, 2024
Merged

gh-116738: Make _csv module thread-safe #118344

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
eeba1c2
Make the filed_limit thread safe
aisk Apr 27, 2024
40e71a8
Make dialects fields on module state thread safe
aisk Apr 27, 2024
c77750f
Utilize FT_ATOMIC_LOAD_INT32
aisk Apr 27, 2024
6fbcb9f
Remove critical section on `dialects` field
aisk Apr 27, 2024
60753f1
Totally revert the change to csv_register_dialect
aisk Apr 29, 2024
64b7d20
Revert field_limit to long, and add atomic api for it
aisk Apr 29, 2024
1de333e
Add _Py_atomic_load_long for other platforms
aisk Apr 29, 2024
7bbfd36
Merge branch 'main' into csv-thread-safe
aisk May 15, 2024
0a41927
Revert the type change to old_limit
aisk May 15, 2024
f413092
Using atomic_store instead of critical_section
aisk May 15, 2024
6c40b59
Update clinic generated files
aisk May 15, 2024
a79cf88
Apply suggestions from code review
aisk May 17, 2024
1f03de1
Merge branch 'main' into csv-thread-safe
aisk Jul 10, 2024
3739c46
Change field_limit to Py_ssize_t
aisk Jul 10, 2024
850bf2a
Remove atomic functions for long type
aisk Jul 10, 2024
bfbfc56
Fix types
aisk Jul 10, 2024
d73b827
Fix the return type of field_size_limit
aisk Jul 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert field_limit to long, and add atomic api for it
  • Loading branch information
@aisk
aisk committed Apr 29, 2024
commit 64b7d209bf8bb05c17739793770b256f4330ef20
4 changes: 4 additions & 0 deletions Include/cpython/pyatomic_gcc.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,10 @@ static inline int
_Py_atomic_load_int(const int *obj)
{ return __atomic_load_n(obj, __ATOMIC_SEQ_CST); }

static inline long
_Py_atomic_load_long(const long *obj)
{ return __atomic_load_n(obj, __ATOMIC_SEQ_CST); }

static inline int8_t
_Py_atomic_load_int8(const int8_t *obj)
{ return __atomic_load_n(obj, __ATOMIC_SEQ_CST); }
Expand Down
4 changes: 2 additions & 2 deletions Include/internal/pycore_pyatomic_ft_wrappers.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ extern "C" {
_Py_atomic_store_ssize_relaxed(&value, new_value)
#define FT_ATOMIC_STORE_UINT8_RELAXED(value, new_value) \
_Py_atomic_store_uint8_relaxed(&value, new_value)
#define FT_ATOMIC_LOAD_INT32(value) _Py_atomic_load_int32(&value)
#define FT_ATOMIC_LOAD_LONG(value) _Py_atomic_load_long(&value)

#else
#define FT_ATOMIC_LOAD_PTR(value) value
Expand All @@ -54,7 +54,7 @@ extern "C" {
#define FT_ATOMIC_STORE_UINTPTR_RELEASE(value, new_value) value = new_value
#define FT_ATOMIC_STORE_SSIZE_RELAXED(value, new_value) value = new_value
#define FT_ATOMIC_STORE_UINT8_RELAXED(value, new_value) value = new_value
#define FT_ATOMIC_LOAD_INT32(value) value
#define FT_ATOMIC_LOAD_LONG(value) value

#endif

Expand Down
6 changes: 3 additions & 3 deletions Modules/_csv.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ module instead.
#endif

#include "Python.h"
#include "pycore_pyatomic_ft_wrappers.h" // FT_ATOMIC_LOAD_INT32
#include "pycore_pyatomic_ft_wrappers.h" // FT_ATOMIC_LOAD_LONG

#include <stddef.h> // offsetof()
#include <stdbool.h>
Expand All @@ -35,7 +35,7 @@ typedef struct {
PyTypeObject *dialect_type;
PyTypeObject *reader_type;
PyTypeObject *writer_type;
int32_t field_limit; /* max parsed field size */
long field_limit; /* max parsed field size */
PyObject *str_write;
} _csvstate;

Expand Down Expand Up @@ -703,7 +703,7 @@ parse_grow_buff(ReaderObj *self)
static int
parse_add_char(ReaderObj *self, _csvstate *module_state, Py_UCS4 c)
{
uint32_t field_limit = FT_ATOMIC_LOAD_INT32(module_state->field_limit);
long field_limit = FT_ATOMIC_LOAD_LONG(module_state->field_limit);
if (self->field_len >= field_limit) {
PyErr_Format(module_state->error_obj,
"field larger than field limit (%ld)",
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy