Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix data flow checking. #4025

Merged
merged 5 commits into from
Mar 17, 2022

Conversation

Shylock-Hg
Copy link
Contributor

@Shylock-Hg Shylock-Hg commented Mar 14, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

Thie happens when I try to push filter down to AppendVertices, the ldbc query(as below) report broken promise error when I add this rule. After debugging, I found it's caused by mistake of checkDataFlow:

  1. AppendVertices read by Argument operator and Argument doesn't depend on it.
  2. In this case, checkDataFlow should return failure, but don't for the empty dependencies of Argument operator.
  3. Then after applying this optimize rule, the origin readby relationship is erased, and new readby relationship is rebuild by dependencies relationship. But as said before, Argument operator don't depends on AppendVertices, so now Argument don't read AppendVertices too!
  4. Scheduler will set dependencies of Argument by read by relationship, but now Argument lost these, so lead to broken promise error.
      MATCH
        (tagClass1:TagClass)<-[:HAS_TYPE]-(:`Tag`)<-[:HAS_TAG]-
        (forum1:Forum)-[:HAS_MEMBER]->(stranger:Person)
      WHERE id(tagClass1)=="MusicalArtist"
      WITH DISTINCT stranger
      MATCH
        (tagClass2:TagClass)<-[:HAS_TYPE]-(:`Tag`)<-[:HAS_TAG]-
        (forum2:Forum)-[:HAS_MEMBER]->(stranger)
      WHERE id(tagClass2)=="OfficeHolder"
      WITH DISTINCT stranger
      MATCH
        (person:Person)<-[:HAS_CREATOR]-(`comment`:`Comment`)-[:REPLY_OF*]->(message:Message)-[:HAS_CREATOR]->(stranger)
      WHERE person.Person.birthday > 19890101 AND
            person <> stranger
      OPTIONAL MATCH p1 = (person)-[:KNOWS]-(stranger)
      OPTIONAL MATCH p2       =(message)-[:REPLY_OF*]->(:Message)-[:HAS_CREATOR]->(stranger)
      WITH person.Person.id AS personId, stranger.Person.id AS strangerId, `comment`.`Comment`.length AS commentLength, p1, p2
      WHERE p1 IS NULL AND
            p2 IS NULL
      RETURN
        personId,
        count(DISTINCT strangerId) AS strangersCount,
        count(commentLength) AS interactionCount
      ORDER BY
        interactionCount DESC,
        personId ASC
      LIMIT 100

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

@Shylock-Hg Shylock-Hg added ready-for-testing PR: ready for the CI test type/bug Type: something is unexpected labels Mar 14, 2022
@yixinglu
Copy link
Contributor

Then after applying this optimize rule, the origin readby relationship is erased,

I want to know why to erase the readby relationship?

@Shylock-Hg
Copy link
Contributor Author

Shylock-Hg commented Mar 14, 2022

Then after applying this optimize rule, the origin readby relationship is erased,

I want to know why to erase the readby relationship?

I'm not sure, it's code in OptGroup::explore. Maybe because readBy is incorrect after transforming (PlanNode is cloned).

@Sophie-Xie Sophie-Xie requested review from nevermore3 and removed request for czpmango and CPWstatic March 15, 2022 13:00
@Shylock-Hg Shylock-Hg requested a review from CPWstatic March 16, 2022 02:01
Copy link
Contributor

@yixinglu yixinglu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember why to add this check here. Maybe we should find the root cause later. At present, LGTM.

@Shylock-Hg
Copy link
Contributor Author

I don't remember why to add this check here. Maybe we should find the root cause later. At present, LGTM.

As I known, it's aimed to keep input/output relationship is subset of dependencies relationship.

@Sophie-Xie Sophie-Xie merged commit 43a447f into vesoft-inc:master Mar 17, 2022
@Shylock-Hg Shylock-Hg deleted the fix/strict-dataflow-checking branch March 17, 2022 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review ready-for-testing PR: ready for the CI test type/bug Type: something is unexpected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants