Split initialize function and add post-initialize phase #1467

KFilipek · 2025-07-25T09:25:04Z

Description

Checklist

Code compiles without errors locally
All tests pass locally
CI workflows execute properly

CI workflows, not executed per PR (e.g. Nightly), execute properly
New tests added, especially if they will fail without my changes
Added/extended example(s) to cover this functionality
Extended the README/documentation
All newly added source files have a license
All newly added source files are referenced in CMake files
Logger (with debug/info/... messages) is used
All API changes are reflected in docs and def/map files, and are tested

lplewa · 2025-08-01T10:05:29Z

include/umf/memory_pool_ops.h

+    /// @param provider memory provider that will be used for coarse-grain allocations.
+    ///        Should contain at least one memory provider.
+    /// @param numProvider number of elements in the providers array
+    /// @param params pool-specific params, or NULL for defaults
+    /// @param pool [out] returns pointer to the pool
+    /// @return UMF_RESULT_SUCCESS on success or appropriate error code on failure.


Write better documentation

lplewa · 2025-08-01T10:44:43Z

src/memory_provider.c

@@ -250,6 +276,16 @@ umf_result_t umfMemoryProviderCreate(const umf_memory_provider_ops_t *ops,

    provider->provider_priv = provider_priv;

+    if (provider->ops.ext_post_initialize != NULL) {


this if is not needed

lplewa · 2025-08-01T10:46:59Z

src/pool/pool_disjoint.c

-static umf_result_t CTL_READ_HANDLER(name)(void *ctx,
-                                           umf_ctl_query_source_t source,
-                                           void *arg, size_t size,
-                                           umf_ctl_index_utlist_t *indexes) {
+static umf_result_t CTL_READ_HANDLER(name)(void* ctx,
+    umf_ctl_query_source_t source,
+    void* arg, size_t size,
+    umf_ctl_index_utlist_t* indexes) {
    (void)source, (void)indexes;

-    disjoint_pool_t *pool = (disjoint_pool_t *)ctx;
+    disjoint_pool_t* pool = (disjoint_pool_t*)ctx;

    if (arg == NULL) {
        return UMF_RESULT_ERROR_INVALID_ARGUMENT;
    }

    if (size > 0) {
-        strncpy((char *)arg, pool->params.name, size - 1);
-        ((char *)arg)[size - 1] = '\0';
+        strncpy((char*)arg, pool->params.name, size - 1);
+        ((char*)arg)[size - 1] = '\0';
    }


Corrected. Funny fact is that this is ignored by code check, I suppose that #if 0 is here a perpetrator.

lplewa · 2025-08-01T10:47:15Z

src/pool/pool_disjoint.c

    (void)source, (void)indexes, (void)size;
-    disjoint_pool_t *pool = (disjoint_pool_t *)ctx;


I suppose it's fixed

lplewa · 2025-08-01T10:50:30Z

src/memory_pool.c

@@ -261,7 +276,16 @@ static umf_result_t umfPoolCreateInternal(const umf_memory_pool_ops_t *ops,
        }
    }

+    if (ops->ext_post_initialize != NULL) {
+        ret = ops->ext_post_initialize(pool->provider, params, pool->pool_priv);


document what happens if post initialize fails. Who is responsible for clearing things initialized in initialize bunction

lplewa · 2025-08-01T10:51:20Z

src/pool/pool_disjoint.c

-
-err_free_disjoint_pool:
-    umf_ba_global_free(disjoint_pool);
-


is it memleak? Who is respoinsible for this free?

Reverted, thanks

lplewa · 2025-08-01T10:54:42Z

test/pools/jemalloc_pool.cpp

@@ -161,7 +161,7 @@ TEST_F(test, jemallocPoolParams) {
    EXPECT_EQ(ret, UMF_RESULT_SUCCESS);
 }

-TEST_F(test, jemallocPoolParamsInvalid) {
+TEST_F(test, jemallocPoolParamsInvalid) { // TODO FIX THIS


lukaszstolarczuk · 2025-08-04T09:02:19Z

include/umf/memory_pool_ops.h

+    /// @brief Post-initializes memory pool.
+    /// @param provider memory provider that will be used for coarse-grain allocations.
+    ///        Should contain at least one memory provider.
+    /// @param numProvider number of elements in the providers array


rm numProvider + Should contain at least one memory provider.

lukaszstolarczuk · 2025-08-04T09:10:08Z

src/memory_pool.c

+            memcpy(&compatible_ops, ops,
+                   offsetof(umf_memory_pool_ops_t, ext_post_initialize));
+        } else {
+            LOG_ERR("Memory Pool ops unknown version, which \"%d\" is not "


\"%d\" should be moved right after unknown version, I guess

e.g. Memory Pool ops unknown version (%d) - it is not supported or Unsupported Memory Pool ops version: %d

Done - rebased.

lukaszstolarczuk · 2025-08-04T09:12:03Z

src/memory_pool.c

+            memcpy(&compatible_ops, ops,
+                   offsetof(umf_memory_pool_ops_t, ext_post_initialize));
+        } else {
+            LOG_ERR("Memory Pool ops unknown version, which \"%d\" is not "


what if we have new version 1.2 - it won't be supported?

we need to keep backward compatibility, not future :)

soo, I gues it will just fail whenever someone bumps the version, which is not ideal - just saying ;)

lukaszstolarczuk · 2025-08-04T09:13:00Z

include/umf/memory_pool_ops.h

@@ -166,6 +166,18 @@ typedef struct umf_memory_pool_ops_t {
                            const char *name, void *arg, size_t size,
                            umf_ctl_query_type_t queryType, va_list args);

+    ///
+    /// @brief Post-initializes memory pool.


can you please describe it a little more, what does it mean "post-init"? Reading the code I can see something for CTL is set up between init and post-init, but that's really not obvious reading the docs.

lukaszstolarczuk · 2025-08-04T09:13:44Z

.github/workflows/reusable_compatibility.yml

@@ -98,7 +98,7 @@ jobs:
      run: >
        UMF_LOG="level:warning;flush:debug;output:stderr;pid:no"
        LD_LIBRARY_PATH=${{github.workspace}}/latest_version/build/lib/
-        ctest --verbose


whole test file should be excluded? At least add a TODO info why it's disabled.

+1. Also, if only specific test cases fail, please use gtest to test the rest

Comment added and splited in two steps

lukaszstolarczuk · 2025-08-04T09:25:34Z

src/provider/provider_cuda.c

+                                                       void *provider) {
+    (void)params;
+    (void)provider;
+    // For initial version, just return success


since this is ext function, can't we just omit it, when not needed?

+1 - this would be replaced by "default" handler during provider creation

bratpiorka · 2025-08-04T13:40:14Z

.github/workflows/reusable_compatibility.yml

@@ -98,7 +98,7 @@ jobs:
      run: >
        UMF_LOG="level:warning;flush:debug;output:stderr;pid:no"
        LD_LIBRARY_PATH=${{github.workspace}}/latest_version/build/lib/
-        ctest --verbose


+1. Also, if only specific test cases fail, please use gtest to test the rest

bratpiorka · 2025-08-04T14:40:26Z

include/umf/memory_pool_ops.h

@@ -36,6 +36,10 @@ typedef struct umf_memory_pool_ops_t {

    ///
    /// @brief Initializes memory pool.
+    /// /details
+    /// * This function *must* be called before any other memory pool operation.


Please remove this. This API is not intended to be called by the user; it is only used by the internal UMF functions

bratpiorka · 2025-08-04T14:41:10Z

include/umf/memory_pool_ops.h

+    /// @brief Post-initializes and set up memory pool.
+    ///
+    /// \details
+    /// * This function *must* be called before any other memory pool operation.


Please remove notes about calling this func. This API is not intended to be called by the user; it is only used by the internal UMF functions

bratpiorka · 2025-08-04T14:46:15Z

include/umf/memory_pool_ops.h

+    ///
+    /// @param provider memory provider that will be used for coarse-grain allocations
+    /// @param params pool-specific params, or NULL for defaults
+    /// @param pool returns pointer to the pool


This function doesn't set the "pool" param - this is the in not out. Next, the "pool" param should go first, as this is the operation on an existing object. Finally, why do we need provider and params args? What if they are different from args used in initialize()?

bratpiorka · 2025-08-04T14:50:18Z

src/pool/pool_disjoint.c

@@ -1146,7 +1152,7 @@ static umf_memory_pool_ops_t UMF_DISJOINT_POOL_OPS = {
    .get_last_allocation_error = disjoint_pool_get_last_allocation_error,
    .get_name = disjoint_pool_get_name,
    .ext_ctl = disjoint_pool_ctl,
-};
+    .ext_post_initialize = disjoint_pool_post_initialize};


nit: add ',' after the last member

Initializer list doesn't require it; it's not a struct declaration. Sometimes ends with ,, sometimes not (vide defaults structure in the same file). I will leave it as is, if it's not a problem.

bratpiorka · 2025-08-04T14:51:53Z

include/umf/memory_pool_ops.h

+    /// @param pool returns pointer to the pool
+    /// @return UMF_RESULT_SUCCESS on success or appropriate error code on failure
+    ///
+    umf_result_t (*ext_post_initialize)(umf_memory_provider_handle_t provider,


please remove params and provider - they should be stored in the pool already

bratpiorka · 2025-08-04T14:57:55Z

src/provider/provider_cuda.c

+                                                       void *provider) {
+    (void)params;
+    (void)provider;
+    // For initial version, just return success


+1 - this would be replaced by "default" handler during provider creation

bratpiorka · 2025-08-04T14:58:48Z

src/memory_provider.c

+                                             void *provider) {
+    (void)params;
+    (void)provider;
+    return UMF_RESULT_SUCCESS;


please return UMF_RESULT_ERROR_NOT_SUPPORTED here

bratpiorka · 2025-08-04T15:16:01Z

include/umf/memory_pool_ops.h

@@ -166,6 +170,24 @@ typedef struct umf_memory_pool_ops_t {
                            const char *name, void *arg, size_t size,
                            umf_ctl_query_type_t queryType, va_list args);

+    ///
+    /// @brief Post-initializes and set up memory pool.


add "this function must be implemented if the pool/provider supports CTL that overrides defaults"

Even with this clarification, for me it is not clear why we need a separate function.

bratpiorka · 2025-08-05T12:37:55Z

include/umf/memory_pool_ops.h

@@ -166,6 +169,24 @@ typedef struct umf_memory_pool_ops_t {
                            const char *name, void *arg, size_t size,
                            umf_ctl_query_type_t queryType, va_list args);

+    ///


add a comment above "The following operations were added in ops version 1.1"

bratpiorka · 2025-08-05T12:38:35Z

include/umf/memory_provider_ops.h

@@ -278,6 +278,14 @@ typedef struct umf_memory_provider_ops_t {
                            const char *name, void *arg, size_t size,
                            umf_ctl_query_type_t queryType, va_list args);

+    ///


add a comment above "The following operations were added in ops version 1.1"

Split between initialize and post-initialize function is necessary for properly handling CTL defaults.

vinser52 · 2025-08-05T18:06:50Z

include/umf/memory_pool_ops.h

+    /// \details
+    /// * This function *must* be implemented if the pool/provider supports CTL that overrides defaults.
+    /// * This function *must* free any resources allocated in the function.
+    /// * This function *must* be called after the memory pool has been allocated in initialize function


This function *must* be called after the memory pool has been allocated in initialize function

Must be called by whom? If UMF calls it, then the comment is missleading.

vinser52 · 2025-08-05T18:08:02Z

include/umf/memory_pool_ops.h

@@ -166,6 +170,24 @@ typedef struct umf_memory_pool_ops_t {
                            const char *name, void *arg, size_t size,
                            umf_ctl_query_type_t queryType, va_list args);

+    ///
+    /// @brief Post-initializes and set up memory pool.


Even with this clarification, for me it is not clear why we need a separate function.

KFilipek self-assigned this Jul 25, 2025

KFilipek requested a review from a team as a code owner July 25, 2025 09:25

bratpiorka marked this pull request as draft July 28, 2025 09:22

KFilipek force-pushed the post_initialize branch 17 times, most recently from 54df467 to 40c6dfc Compare August 1, 2025 08:46

KFilipek marked this pull request as ready for review August 1, 2025 10:14

KFilipek requested review from lplewa and bratpiorka August 1, 2025 10:15

KFilipek force-pushed the post_initialize branch from 40c6dfc to 1f76309 Compare August 1, 2025 10:52

lplewa requested changes Aug 1, 2025

View reviewed changes

lplewa reviewed Aug 1, 2025

View reviewed changes

lukaszstolarczuk reviewed Aug 4, 2025

View reviewed changes

KFilipek force-pushed the post_initialize branch 2 times, most recently from a5f6195 to 7aca774 Compare August 4, 2025 11:53

bratpiorka requested changes Aug 4, 2025

View reviewed changes

KFilipek force-pushed the post_initialize branch 3 times, most recently from 7af81de to e3b372c Compare August 5, 2025 11:14

bratpiorka reviewed Aug 5, 2025

View reviewed changes

KFilipek force-pushed the post_initialize branch 3 times, most recently from 4675e67 to fdf2ac3 Compare August 5, 2025 16:42

add post-initialize function to pools and providers

5b35410

Split between initialize and post-initialize function is necessary for properly handling CTL defaults.

KFilipek force-pushed the post_initialize branch 3 times, most recently from dcb8c16 to 172429a Compare August 5, 2025 17:15

NEW

1fcad79

KFilipek force-pushed the post_initialize branch from 172429a to 1fcad79 Compare August 5, 2025 17:19

vinser52 reviewed Aug 5, 2025

View reviewed changes

		@@ -250,6 +276,16 @@ umf_result_t umfMemoryProviderCreate(const umf_memory_provider_ops_t *ops,

		provider->provider_priv = provider_priv;

		if (provider->ops.ext_post_initialize != NULL) {

		(void)source, (void)indexes, (void)size;
		disjoint_pool_t pool = (disjoint_pool_t )ctx;


		err_free_disjoint_pool:
		umf_ba_global_free(disjoint_pool);

Split initialize function and add post-initialize phase #1467

Are you sure you want to change the base?

Split initialize function and add post-initialize phase #1467

Uh oh!

Conversation

KFilipek commented Jul 25, 2025

Description

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment