Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#6499 commit order must consider non-nullability of join columns as prioritised over nullable join columns #6533

Closed
wants to merge 3 commits into from

Conversation

Frikkle
Copy link
Contributor

@Frikkle Frikkle commented Jun 29, 2017

As suggested by @lcobucci, I created a PR with a minimal test example.

While working on the test example I looked into the code and found the (probable) cause for this issue and proposed a fix for it in a second commit.

The issue seems to originate from the part of the code in UnitOfWork which calculates the weight for new dependencies in the graph based on whether or not the nullable clause on join columns is set and to which value.

In particular the part (int)empty($joinColumns['nullable']) which to me seems incorrect. With the current implementation in master the case where the nullable property on the join column is set to false (so, the relation is mandatory) yields the same weight as when the nullable property is not defined. But when the nullable property is not defined, the documention states that the default would be nullable === true.

Below, you find a simple truth table breakdown of the specific part of the code (for the possible values for $joinColumns['nullable'].

(int) empty ( $joinColumns['nullable'] )
0 false true
1 true false
1 true unset/null/etc.

I tried to keep the (in-line) fix as compact as possible. All test keep running and the added test for this issue also succeeds after applying the fix. Before applying the fix, it throws a ConstraintViolationException and tries to insert null for a mandatory foreign key.

Since I'm not completely familiar with the codebase, please look into whether this does not influence other parts of the code. If there's anything I can do to assist, let me know.

/**
* DDC6499A constructor.
*/
public function __construct()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop all API that isn't strictly needed? Getters, setters, etc

*/
public function __construct()
{
$this->bs = new ArrayCollection();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use = [] as default property value (test is simple enough) ;-)

/**
* Test for the bug described in issue #6499.
*/
public function testIssue()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mark as : void

/**
* {@inheritDoc}
*/
protected function setUp()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mark as : void

$this->_em->flush();

// Issue #6499 will result in a Integrity constraint violation before reaching this point
$this->assertEquals(true, true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be dropped - instead, try a find after a clear, and verify that the collection and the association are correctly populated. Put the comment in the assertion message

private $id;

/**
* @OneToMany(targetEntity="DDC6499B", mappedBy="a", cascade={"persist", "remove"}, orphanRemoval=true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop all cascade operations, orphan removals and properties/mappings that aren't strictly related to the tests

@@ -1154,7 +1154,7 @@ private function getCommitOrder(array $entityChangeSet = null)

$joinColumns = reset($assoc['joinColumns']);

$calc->addDependency($targetClass->name, $class->name, (int)empty($joinColumns['nullable']));
$calc->addDependency($targetClass->name, $class->name, (int) ! ( ! isset($joinColumns['nullable']) || $joinColumns['nullable'] === true));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of negations. Also, this is only checking the first join column... Possibly requiring to add tests around this area. :-\

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to also cover this on the unit level (UnitOfWorkTest)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep it on integration tests for now - unit testing the UoW is a bit more messy, and we can add that just before merging, rather than early during the fixing process

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just fixed the other requested changes, causing the example to be a lot smaller. This proposed change fixes our broken test, but consequently breaks two other tests. I will look a bit further into this when I have some spare time.

@Ocramius Do you agree that there is a logic inconsistency in the the original line in UnitOfWork.php?

@Frikkle
Copy link
Contributor Author

Frikkle commented Jun 29, 2017

I'll adjust the above suggestions, fix Scrutinizer improvements and run some more tests locally.

-- Processed Ocramius' feedback.
@guilliamxavier
Copy link
Contributor

Just for the record: the file's history shows that the current logic

(int)empty($joinColumns['nullable'])

originates from 27e9b49#diff-6e8c1c1e78b054ba05e20ea09d877865L1158 as a simplification of

(isset($joinColumns['nullable']) ? $joinColumns['nullable'] : false) ? 0 : 1

(actually with an intermediate variable $isNullable for the parenthesized ternary expression)
which was originally introduced in 8ea62b9#diff-6e8c1c1e78b054ba05e20ea09d877865L1165

so the present PR is like saying that the false should actually have been a true, or (in PHP 7):

($joinColumns['nullable'] ?? true) ? 0 : 1

or alternatively

(int) ! ($joinColumns['nullable'] ?? true)
(int) ! ( ! isset($joinColumns['nullable']) || $joinColumns['nullable'])
(int) (isset($joinColumns['nullable']) && ! $joinColumns['nullable'])

all being equivalent to the currently proposed fix just without the === true strict equality.

Now, as you already noted, this is unfortunately causing exceptions in an existing test, and the original logic was precisely a fix for the affected test.

@Frikkle
Copy link
Contributor Author

Frikkle commented Jun 30, 2017

@guilliamxavier Thanks for clarifying the file's history. As said, I'm not that familiar with the codebase and when digging into it I also could not find out what causes the other two tests to fail.

It might be that this proposed fix isn't correct for this issue, but then the question remains: what causes issue #6499 to arise? Or is #6499 intended behavior?

@Frikkle
Copy link
Contributor Author

Frikkle commented Aug 11, 2017

Any news on this issue?

@Ocramius Ocramius added this to the 2.5.7 milestone Aug 11, 2017
@Ocramius Ocramius self-assigned this Aug 11, 2017
@Ocramius Ocramius added the Bug label Aug 11, 2017
@Ocramius
Copy link
Member

@triasinformatica-gabe adding it to my bucket of stuff to check

@Frikkle
Copy link
Contributor Author

Frikkle commented Aug 11, 2017

@Ocramius thanks.

If there's anything we (me or my colleagues) can do to help, please let us know. But currently we're unfamiliar with the history of this part of the doctrine codebase.

Ocramius added a commit that referenced this pull request Aug 11, 2017
Ocramius added a commit that referenced this pull request Aug 11, 2017
Ocramius added a commit that referenced this pull request Aug 11, 2017
Ocramius added a commit that referenced this pull request Aug 11, 2017
Ocramius added a commit that referenced this pull request Aug 11, 2017
…r makes it *SOMEWHAT* more readable (no miracles)
@Ocramius Ocramius closed this in 2a58645 Aug 11, 2017
@Ocramius Ocramius changed the title #6499 test case + fix #6499 commit order must consider non-nullability of join columns as prioritised over nullable join columns Aug 11, 2017
@Ocramius Ocramius modified the milestones: 2.6.0, 2.5.7 Aug 11, 2017
@Ocramius
Copy link
Member

@triasinformatica-gabe I manually merged after some cleanups and moving stuff around.

@lcobucci I couldn't find a proper way to test this in isolation, as it's deep in UnitOfWork internals, sadly.

This patch can only land in 2.6.0, since the commit order calculator doesn't even know about weights of dependencies before that.

@Ocramius Ocramius reopened this Aug 11, 2017
Ocramius added a commit that referenced this pull request Aug 11, 2017
…nsider-all-join-column-fields'"

This reverts commit 2a58645, reversing
changes made to 6d428c9.
@Ocramius
Copy link
Member

@triasinformatica-gabe sorry, I had to revert the merge - this patch introduces errors when deleting foreign key referenced data.

I think this is much more complex than what it looks. To be clear, I know as much as you do about this issue. The problem area is exactly the one you worked on, except that the commit order isn't simply "reversible" for deletes (current approach).

@Ocramius Ocramius removed this from the 2.6.0 milestone Aug 11, 2017
@Ocramius Ocramius assigned Frikkle and unassigned Ocramius Aug 11, 2017
@bendavies
Copy link

bendavies commented Feb 5, 2018

@triasinformatica-gabe @Ocramius is this waiting on anything now? Or was this the commit that was reverted?
This fixes #7006 for me

@Ocramius
Copy link
Member

Ocramius commented Feb 5, 2018

@bendavies see #6533 (comment) - this was reverted

@bendavies
Copy link

bendavies commented Feb 5, 2018

@Ocramius yeah thanks. I got confused as this PR was still open. was CascadeRemoveOrderTest the failure? It seems to be passing on 2.6 now, with this patch.

@Ocramius
Copy link
Member

Ocramius commented Feb 5, 2018

@bendavies no, we are lacking tests about cascade-deleted data, from what I remember (sorry, mind is also a bit clouded over here)

@Frikkle
Copy link
Contributor Author

Frikkle commented Feb 5, 2018

@bendavies It's passing the tests, but only in SQLite, which is ignoring the foreign keys. If you run the test set in MySQL or another database that does enforce this, some other tests will fail. There seems to be a non-trivial issue in building up the relationship dependency graph for cascades. I didn't have time so far to look into it, but it's still somewhere on my list.

The fix this PR originates from was a bit too rigid and therefore not solving the issue.

@bendavies
Copy link

bendavies commented Feb 5, 2018 via email

@Frikkle
Copy link
Contributor Author

Frikkle commented Feb 5, 2018 via email

@bendavies
Copy link

hey @triasinformatica-gabe. Any chance to look at this yet?

@Frikkle
Copy link
Contributor Author

Frikkle commented Feb 21, 2018

@bendavies Sorry for the late response, haven't had time to feedback on this before since I was abroad for a while. Any way; I did run all the tests on top of the latest changes and for me all the same tests keep failing the same way as they did last summer, so I'm not sure how you managed to get it working running Postgress and MySQL.

Could you elaborate?

@bendavies
Copy link

Hmm, I'll try again.

@bendavies
Copy link

@triasinformatica-gabe which test is failing for you please?

@lcobucci
Copy link
Member

we are lacking tests about cascade-deleted data

@Ocramius can you give more context about that? Maybe creating an issue for us to track these tests (specially considering the changes in v3.0)?

@Ocramius
Copy link
Member

@Ocramius can you give more context about that?

The problem is:

(sorry, mind is also a bit clouded over here)

I also don't remember the details.

@lcobucci
Copy link
Member

I also don't remember the details.

@Ocramius I BELIEVE IN YOU, you can do it!!!! Didn't help, did it? HAHAHA alright, thanks!

@bendavies
Copy link

@triasinformatica-gabe trying to pick this up again....which test fails for you?

@greg0ire greg0ire closed this Feb 23, 2021
@greg0ire greg0ire deleted the branch doctrine:master February 23, 2021 08:19
mpdude added a commit to mpdude/doctrine2 that referenced this pull request Jun 2, 2023
@Frikkle was the first to propose these tests in doctrine#6533.
@rvanlaak followed up in doctrine#8703, making some adjustments.
mpdude added a commit to mpdude/doctrine2 that referenced this pull request Jun 2, 2023
@Frikkle was the first to propose these tests in doctrine#6533.
@rvanlaak followed up in doctrine#8703, making some adjustments.

Co-authored-by: Gabe van der Weijde <[email protected]>
Co-authored-by: Richard van Laak <[email protected]>
Co-authored-by: Grégoire Paris <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants