4224 openldap entryuuid confusion #4344

Firstyear · 2020-09-28T05:41:05Z

Bug Description: OpenLDAP server as a syncrepl consumed enforces
the condition that syncUUID in ldap messages must match the entryuuid
of the entry. This is not in the RFC but it affects this one situation.

Fix Description: To resolve this, we enforce that entryuuid is a
requirement to the openldap syncrepl mode. Only entries with an
entryuuid can be sent to openldap. Additionally, this mode is disabled
by default by a configuration parameter "syncrepl-allow-openldap" in
the content sync plugin config.

NOTE: It looks like the commits from 51260 which were previously merged seem to be missing after the migration, so this branch includes those.

Thanks!

Fixes: #4313 #4224

tbordaz

@Firstyear the overall patch looks very nice. I have minor comments. However I would appreciate a design (port389) describing the solution and the choices

ldap/servers/plugins/sync/sync_refresh.c

ldap/servers/plugins/sync/sync_init.c

ldap/servers/plugins/sync/sync_persist.c

ldap/servers/plugins/sync/sync_util.c

ldap/servers/plugins/sync/sync_refresh.c

Firstyear · 2020-10-06T01:28:31Z

@tbordaz Thanks for your review, I have updated all the areas you commented on, and I have updated https://www.port389.org/docs/389ds/design/openldap2ds.html with details about the syncrepl changes.

Firstyear · 2020-10-12T03:16:40Z

@tbordaz, @mreynolds389 Review reminder :)

Firstyear · 2020-10-19T02:26:11Z

Ping @tbordaz and @mreynolds389 :)

Firstyear · 2020-11-20T02:47:02Z

Ping @tbordaz and @mreynolds389,

mreynolds389 · 2020-11-20T13:25:32Z

@tbordaz has been doing a lot of work around sync repl lately. I'd prefer him to do this review as it could impact what he is working on

Firstyear · 2020-11-22T23:13:16Z

Yeah, that's fair @mreynolds389. I'll poke him some more then about the matter :)

tbordaz · 2020-12-02T10:52:41Z

@Firstyear except those two last comments/questions, the overall patch looks nice to me

Firstyear · 2020-12-03T01:06:22Z

Hey mate, I don't see the two comments/questions you are referring to? All the comments I can see are your older ones which have been addressed?

dirsrvtests/tests/suites/syncrepl_plugin/__init__.py

ldap/servers/plugins/sync/sync_refresh.c

tbordaz · 2020-12-03T06:44:57Z

@Firstyear , definitely something I do not understand with PR. Some comments can remain "hidden" and some others are public.. I do not know what to do to make all my comments public. Can you see my two comments (sync_create_state_control, null cookie) ?

Firstyear · 2020-12-04T00:59:10Z

@Firstyear , definitely something I do not understand with PR. Some comments can remain "hidden" and some others are public.. I do not know what to do to make all my comments public. Can you see my two comments (sync_create_state_control, null cookie) ?

If you start a review, your comments are "staged" in your browser, and only if you click "complete review" in the top right do they become visible to me and others. Maybe that's what is going on?

Firstyear · 2020-12-04T01:06:44Z

@tbordaz I think your python tests are failing because of this python-ldap/python-ldap#351

I indicated in a comment how to check your python ldap version for this :)

tbordaz · 2020-12-04T07:09:30Z

@tbordaz I think your python tests are failing because of this python-ldap/python-ldap#351

I indicated in a comment how to check your python ldap version for this :)

Exactly !! you can not imagine the amount of time I spent to identify that the failure was due to incorrect python-ldap failing to decode syncInfo. As I am still testing with prefix_install I was using an old version of python-ldap. Thanks for having fixed that.
This python-ldap bug impacted test_syncrepl_basic but others sync_repl tests were failing because of invalid testcase. Now all is fixed upstream

ldap/servers/plugins/sync/sync_util.c

dirsrvtests/tests/suites/syncrepl_plugin/openldap_test.py

Firstyear · 2020-12-06T23:46:15Z

@tbordaz Okay, I've updated this - I've fixed the null deref, and added flags to the tests to prevent them running if python ldap is too old, as well as checkingf for enable-rust which gives the entry uuid support. I've run all the sync repl tests and they all pass.

tbordaz · 2020-12-07T09:54:38Z

@tbordaz Okay, I've updated this - I've fixed the null deref, and added flags to the tests to prevent them running if python ldap is too old, as well as checkingf for enable-rust which gives the entry uuid support. I've run all the sync repl tests and they all pass.

Sorry, I can not see the changes in the patch. Am I missing something ?

Firstyear · 2020-12-07T23:52:55Z

Sorry, I can not see the changes in the patch. Am I missing something ?

Nope, I made a mistake in my push - I pushed the branch to the upstream project, rather than my fork. No damage done to upstream since it was a unique branch name, so I have deleted the mistake, and now pushed to the correct branch. You should now see the changes. Sorry about that :(

tbordaz

Thanks for the patch. LGTM. Ack

Firstyear · 2020-12-08T09:07:40Z

Thanks @tbordaz I might ask @mreynolds389 getting a second look over as well. Really appreciate your help :)

mreynolds389 · 2020-12-09T23:27:47Z

LGTM

@tbordaz

Bug Description: The cookie encodes which changelog entries we have seen up to and including. However, the sync process would then re-send the cl item from the cookie number. This could cause corruption in some cases as some combinations of actions between two points are no-oped in the server. Fix Description: Fix the changelog search to always process that entries of the CL must be greater than, but not equal to the already seen CL items from the cookie. Fixes: 389ds#4313 Author: William Brown <william@blackhats.net.au> Review by: @tbordaz

@tbordaz

Bug Description: This is a supplement to 51260. Fix Description: This expands the test cases to be able to detect the subsequent data corruption of 51260. This also improves documentation around the rfc, and some todo comments for future work with entryuuid + openldap. Fixes: 389ds#4313 Author: William Brown <william@blackhats.net.au> Review by: @tbordaz

@tbordaz

Bug Description: OpenLDAP server as a syncrepl consumed enforces the condition that syncUUID in ldap messages must match the entryuuid of the entry. This is not in the RFC but it affects this one situation. Fix Description: To resolve this, we enforce that entryuuid is a requirement to the openldap syncrepl mode. Only entries with an entryuuid can be sent to openldap. Additionally, this mode is disabled by default by a configuration parameter "syncrepl-allow-openldap" in the content sync plugin config. Fixes: 389ds#4224 Author: William Brown <william@blackhats.net.au> Review by: @tbordaz (Thanks!)

Firstyear requested a review from tbordaz September 28, 2020 05:41

tbordaz reviewed Oct 2, 2020

View reviewed changes

Firstyear force-pushed the 4224-openldap-entryuuid-confusion branch from b456330 to 5eb10cc Compare October 6, 2020 01:27

Firstyear closed this Oct 19, 2020

Firstyear reopened this Oct 19, 2020

Firstyear added this to the 1.4.4 milestone Nov 20, 2020

Firstyear force-pushed the 4224-openldap-entryuuid-confusion branch from 5eb10cc to da51939 Compare December 1, 2020 02:52

tbordaz reviewed Dec 3, 2020

View reviewed changes

tbordaz reviewed Dec 4, 2020

View reviewed changes

ldap/servers/plugins/sync/sync_util.c Show resolved Hide resolved

tbordaz reviewed Dec 4, 2020

View reviewed changes

dirsrvtests/tests/suites/syncrepl_plugin/openldap_test.py Show resolved Hide resolved

Firstyear force-pushed the 4224-openldap-entryuuid-confusion branch from da51939 to 7320dd6 Compare December 7, 2020 23:52

Firstyear force-pushed the 4224-openldap-entryuuid-confusion branch from 7320dd6 to 47e245a Compare December 8, 2020 00:52

tbordaz approved these changes Dec 8, 2020

View reviewed changes

Firstyear added 3 commits December 10, 2020 09:29

Firstyear force-pushed the 4224-openldap-entryuuid-confusion branch from 47e245a to ce7ec36 Compare December 9, 2020 23:29

Firstyear merged commit 5a3a6e5 into 389ds:master Dec 9, 2020

Firstyear deleted the 4224-openldap-entryuuid-confusion branch December 9, 2020 23:29

4224 openldap entryuuid confusion #4344

4224 openldap entryuuid confusion #4344

Uh oh!

Conversation

Firstyear commented Sep 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tbordaz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Firstyear commented Oct 6, 2020

Uh oh!

Firstyear commented Oct 12, 2020

Uh oh!

Firstyear commented Oct 19, 2020

Uh oh!

Firstyear commented Nov 20, 2020

Uh oh!

mreynolds389 commented Nov 20, 2020

Uh oh!

Firstyear commented Nov 22, 2020

Uh oh!

tbordaz commented Dec 2, 2020

Uh oh!

Firstyear commented Dec 3, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tbordaz commented Dec 3, 2020

Uh oh!

Firstyear commented Dec 4, 2020

Uh oh!

Firstyear commented Dec 4, 2020

Uh oh!

tbordaz commented Dec 4, 2020

Uh oh!

Uh oh!

Uh oh!

Firstyear commented Dec 6, 2020

Uh oh!

tbordaz commented Dec 7, 2020

Uh oh!

Firstyear commented Dec 7, 2020

Uh oh!

tbordaz left a comment

Choose a reason for hiding this comment

Uh oh!

Firstyear commented Dec 8, 2020

Uh oh!

mreynolds389 commented Dec 9, 2020

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Firstyear commented Sep 28, 2020 •

edited

Loading