Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format first-party source according to .clang-format #3178

Closed
wants to merge 4 commits into from

Conversation

thejohnfreeman
Copy link
Collaborator

@thejohnfreeman thejohnfreeman commented Dec 5, 2019

This is not a PR I expect to be merged. I'm just creating it to give us all a chance to look at what clang-format would do to our code based on the .clang-format off the branch develop PR for 1.5.0-b1 (#3167). The diff is huge. I suggest you pick out your favorite files and see if you like the outcome.

I ran clang-format on all the source files ending in .h or .cpp under src/ripple and src/test. I left the rest alone. There's plenty to see here without getting into special case files, and I don't think we want to worry about formatting third-party code.

The top ten most-changed files, where a "change" is an added or removed line:

changes path
7362 src/test/rpc/RPCCall_test.cpp
4867 src/test/app/Offer_test.cpp
3478 src/ripple/beast/container/detail/aged_unordered_container.h
2851 src/test/protocol/STTx_test.cpp
2744 src/ripple/app/misc/NetworkOPs.cpp
2116 src/ripple/beast/container/detail/aged_ordered_container.h
1752 src/test/json/json_value_test.cpp
1637 src/test/app/Check_test.cpp
1631 src/ripple/overlay/impl/PeerImp.cpp
1627 src/test/rpc/JSONRPC_test.cpp

clang-format detected these files as Objective-C and reported them with the warning "Configuration file(s) do(es) not support Objective-C: rippled/.clang-format". It left them unchanged. I could not find an option to clang-format to force the language, and the documentation suggests it is based off the filename. These are all headers, so it probably digs into the code to try to disambiguate.

path
src/ripple/consensus/Validations.h
src/ripple/consensus/Consensus.h
src/ripple/consensus/LedgerTrie.h
src/ripple/consensus/DisputedTx.h
src/ripple/app/misc/Manifest.h
src/ripple/app/consensus/RCLCxTx.h
src/test/csf/Histogram.h
src/test/csf/Digraph.h

@ximinez
Copy link
Collaborator

ximinez commented Dec 5, 2019

The first thing I notice is that this breaks the build. Fortunately, it was only a couple of quick changes to fix it: ximinez@466dfb4

@thejohnfreeman
Copy link
Collaborator Author

I merged @ximinez's fixes. This PR is only 3 commits ahead of ripple:develop (not 19), which you can see at develop...thejohnfreeman:clang-format. Fingers crossed that GitHub will recalculate that difference soon.

@codecov-io
Copy link

codecov-io commented Dec 16, 2019

Codecov Report

Merging #3178 into develop will decrease coverage by 0.19%.
The diff coverage is 56.05%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3178      +/-   ##
===========================================
- Coverage    70.41%   70.21%   -0.20%     
===========================================
  Files          683      683              
  Lines        54775    54621     -154     
===========================================
- Hits         38570    38354     -216     
- Misses       16205    16267      +62     
Impacted Files Coverage Δ
src/ripple/app/consensus/RCLConsensus.h 80.00% <ø> (ø)
src/ripple/app/consensus/RCLCxLedger.h 78.94% <ø> (ø)
src/ripple/app/consensus/RCLCxPeerPos.cpp 30.00% <0.00%> (+0.96%) ⬆️
src/ripple/app/consensus/RCLCxPeerPos.h 0.00% <ø> (ø)
src/ripple/app/consensus/RCLCxTx.h 16.66% <0.00%> (ø)
src/ripple/app/consensus/RCLValidations.h 85.71% <ø> (ø)
src/ripple/app/ledger/AbstractFetchPackContainer.h 100.00% <ø> (ø)
src/ripple/app/ledger/AccountStateSF.cpp 33.33% <0.00%> (ø)
src/ripple/app/ledger/BookListeners.h 100.00% <ø> (ø)
src/ripple/app/ledger/ConsensusTransSetSF.cpp 0.00% <0.00%> (ø)
... and 799 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 020b285...bc01a9d. Read the comment docs.

@thejohnfreeman
Copy link
Collaborator Author

@nbougalis raised the issue of multi-line if-conditions branching to non-compound bodies. That is, an if-statement with a multi-line condition and a "single statement" body with no curlies. Here's an example, as our .clang-format formats it, from line 1769 in src/ripple/app/misc/NetworkOps.cpp:

if (loadFactorServer != b.loadFactorServer ||
    loadBaseServer != b.loadBaseServer || baseFee != b.baseFee ||
    em.is_initialized() != b.em.is_initialized())
    return true;

Nik and I both find it undesirable that the first column of the body statement is aligned with the first columns of the continued lines of the condition expression. Let's call this the Binary Operator Scenario.

Note that there are two scenarios in which these columns do not perfectly align:

  1. If the continued lines are broken inside a function call argument list, after an assignment operator, or after the opening curly of a lambda body. Breaking here indents the continued lines by one "level" (4 spaces in our configuration). Call this the Function Call Scenario. Example from line 200 of src/ripple/shamap/impl/SHAMapDelta.cpp
if (!otherMap.walkBranch(
        other, ours->peekItem(), false, differences, maxCount))
    return false;
  1. If the continued lines are broken inside a parenthetical expression. These continued lines are aligned with the first column after the opening parenthesis (I don't like this). Call this the Subexpression Scenario. An example appears at line 1699 in src/ripple/app/misc/NetworkOps.cpp (I couldn't find one with a non-compound body):
if (app_.timeKeeper().now() < (current->info().parentCloseTime +
                               2 * current->info().closeTimeResolution))
{

The only clang-format option that has any effect on the Binary Operator Scenario is BreakBeforeBinaryOperators, which is presently false.

My personal preference is to see every if-statement with a multi-line condition use a compound statement for its body. The opening curly does a good job of marking the end of the condition and the start of the body. However, clang-format never adds any non-whitespace tokens; it will never change a non-compound statement to a compound statement. To make this kind of change automatically, we would have to use clang-tidy.

Lastly, for a particularly egregious example that would look much better with a compound statement, take a look at lines 314-323 in src/ripple/protocol/impl/STParsedJSON.cpp:

if (!(0u <= type &&
      type <=
          std::min<unsigned>(
              std::numeric_limits<
                  std::uint16_t>::max(),
              std::numeric_limits<
                  std::underlying_type_t<
                      LedgerEntryType>>::max())))
    Throw<std::runtime_error>(
        "Invalid ledger entry type: out of range");

@cjcobb23
Copy link
Contributor

My personal preference is to see every if-statement with a multi-line condition use a compound statement for its body.

I would take this a step further even. My preference is that every if statement, whether the condition is multi-line or not, use a compound body, even when it's just a one line body. I know it's not necessary, but just a personal preference.

@cjcobb23
Copy link
Contributor

@thejohnfreeman can we update this and actually merge it? My past two PR's have extraneous reformatting that is a pain to avoid (and a pain for reviewers). Would like to merge this if possible. If we decide we want to change the .clang-format file in the future, we can just reformat again and add the commit to .git-blame-ignore-revs.
There doesn't seem to be any blockers and this would make my life easier if this was merged.

@ximinez
Copy link
Collaborator

ximinez commented Feb 12, 2020

I'm in favor of merging. It might be worth doing a beta with this all by itself so we can get it over with quickly.

@thejohnfreeman
Copy link
Collaborator Author

I'll prepare a new commit based on 1.5.0-b5 and update this PR. If I can get it passing tests quickly enough, maybe we can talk @manojsdoshi into creating a b6 beta soon.

@thejohnfreeman thejohnfreeman force-pushed the clang-format branch 2 times, most recently from fe9c825 to 1ac91cc Compare February 13, 2020 15:28
@seelabs
Copy link
Collaborator

seelabs commented Feb 19, 2020

Not all sources seem to be formatted. For example, AccountCurrencies.cpp isn't formatted.

@seelabs
Copy link
Collaborator

seelabs commented Feb 19, 2020

It would be good to add a CI flags any mis-formatted sources. We should probably do that in this PR.

@seelabs
Copy link
Collaborator

seelabs commented Feb 19, 2020

Since we're reformatting everything, we should consider:

  1. Switching to a vanilla format like "google" instead of a rippled custom one
  2. Tweak any of our current settings, for example, we could change the indent width from 4 to 2 or turn on bin pack parameters and arguments.
  3. We should check if the current style causes problems in people's preferred code editor, and if we can address that with a style change.

@seelabs
Copy link
Collaborator

seelabs commented Feb 19, 2020

The reformatting changes are still showing up in my git blame.

Edit:
I got this working. There's a couple of steps:

  1. I had to update git on my system. It looks like this functionality was introduced in git version 2.23, which is newer than the one I'm using (2.17).
  2. It looks like the wrong commit is specified in .git-blame-ignore-revs it should be c10f44774a9932150f6a3970b88ca75d51317482
  3. I thought .git-blame-ignore-revs would be picked up automatically. I needed to edit my .gitconfig to pick up this file.

@seelabs
Copy link
Collaborator

seelabs commented Feb 19, 2020

I'd prefer to make this the first (and only) commit of a 1.6.0-b1 rather than put this into 1.5.

@thejohnfreeman
Copy link
Collaborator Author

Not sure how so many source files slipped past without getting formatted. Those changes are incoming.

I put the Git version requirement in the log message, but I'll add it and some instructions to the ignore file.

@carlhua
Copy link
Contributor

carlhua commented Feb 20, 2020

Most coding style guides I have seen in the past - including Google's are way too big for anyone to fully absorb. I think this is great that we are converging to a style via .clang-format. My comments are as follows, note they do not necessitate any action items for this PR, perhaps just some thoughts,

  1. Consider reviewing the formatter's configuration to make sure we all agree on the style.
  2. Adopt running clang formatter as part of the commit process.
  3. Inject checks into CI to make sure we are running the formatter as part of the commit process. This way we continue to maintain the style overtime.

@thejohnfreeman
Copy link
Collaborator Author

I had put this branch up two months ago hoping it would give people a chance to easily check it out and see if they dislike the style or if it screws up their editor, but I'm not sure anyone took the opportunity. It might be better to ask forgiveness than permission in a sense; it might be the only way to get folks to check it out. We can always make fixes after, or even roll back the change entirely, as part of the beta refinement process.

I would like to see a CI job as well, but it can be developed separately. I'll open an issue and link it to this review.

Hooks are local to a repository, so I'm not sure how we can deploy a clang-format hook for everyone to use.

@MarkusTeufelberger
Copy link
Collaborator

You can use the https://pre-commit.com/ tool to make sure everyone uses the same version of clang-format (you can also run it in CI) and then it is up to everyone individually to install a pre-commit hook already locally or have faster commits and have the CI shout at you in case you didn't do it right in the first place. People tend to prefer different work styles and this enables both.

@cjcobb23
Copy link
Contributor

I dont know if we need to really do anything in terms of deploying a clang-format hook. There are many different ways to use clang format correctly. Most editors provide clang-format integration. You can run git-clang-format from the command line as well, which reformats any staged changes and adds them to the working tree. We could provide an example pre-commit hook in the repo if you want, that people can use if they wish (by copying to local .git/hooks). As @MarkusTeufelberger said, people tend to prefer different work styles, and I am not sure we really need to provide a way to use it. There is extensive documentation online for clang-format as well.

I found a way to add this to CI by the way, via github actions: https://github.com/cjcobb23/rippled/blob/develop/.github/workflows/ccpp.yml

@thejohnfreeman
Copy link
Collaborator Author

thejohnfreeman commented Feb 21, 2020

I added the GitHub action workflow. It ran as expected on my fork: https://github.com/thejohnfreeman/rippled/runs/465667385

@thejohnfreeman
Copy link
Collaborator Author

This just needs some approvals before @manojsdoshi can cut a beta with it. What do we think? @nbougalis @seelabs

@cjcobb23
Copy link
Contributor

@thejohnfreeman @manojsdoshi There is a proposed b6: #3270 . This PR looks great to me, but if that b6 goes in, we have to rebase and reformat.
Or, we could cut this as b6, and push the proposed b6 to b7.
Or, we could add this PR into the proposed b6, as the last commit.

I actually vote for tacking this on as the last commit in the proposed b6. If we push the proposed b6 to b7, and cut this as a b6, all of the PR's in the proposed b6, which are all approved, are going to have to rebase and reformat. I'd also rather not wait to merge this until a b7, or 1.6, because this PR has been open for quite some time now, yet merging it keeps getting pushed back. @seelabs Since this doesn't actually introduce any functionality changes, I don't think we need to have this as the sole commit for a beta; I think it's fine to include this in a beta with other commits. What do we think?

@thejohnfreeman
Copy link
Collaborator Author

Fair enough, though that means b1 will not be an ancestor of b2. Not sure if that bothers anyone, but I do think it would be new.

@intelliot
Copy link
Collaborator

Feel free to ignore me since I won't be involved with any of this. But, I would be wary of rebasing on top of the formatting PR. I could be wrong, but git doesn't always handle this in the best way and it could end up being a little more difficult than anticipated. There's also a risk of things going awry when rebasing. John's plan seems, at least intuitively, a bit safer: include Tickets in 1.6.0-b1, and then have b2 include this PR exclusively. Formatting is just running an automated tool; rebasing is manual and at risk of human error.

@thejohnfreeman
Copy link
Collaborator Author

Re-reading my last comment, I see now that it probably made no sense to anyone else because I misunderstood @cjcobb23's last comment. Here's what was going through my mind, which may be an option for (a) merging this format PR ASAP and (b) making it easy to merge the Tickets PR:

  1. Merge this PR as 1.6.0-b1.
  2. Roll back step 1.
  3. Merge the Tickets PR.
  4. Format the code and rebase the tail commits from this PR. Presently, there are 4 commits in this PR's tail (two fixes for the build, a .git-blame-ignore-revs file, and a GitHub Action configuration). The .git-blame-ignore-revs file would have to be changed to reflect the new history.
  5. Cut this new history as 1.6.0-b2. It will not have 1.6.0-b1 as an ancestor.

@thejohnfreeman thejohnfreeman force-pushed the clang-format branch 4 times, most recently from 18d83ef to 85e8e2f Compare April 17, 2020 15:23
@seelabs
Copy link
Collaborator

seelabs commented Apr 22, 2020

@thejohnfreeman I just checked the PR out and ran clang format against src/ripple and looked for any files that changed. It looks like we missed some of the ipp files. So we need to format these and change the github action to check for these files as well.

Here's a patch that should work: seelabs@3173fe0

Once we take care of these ipp files I think this is ready to go and we'll do a beta.

@thejohnfreeman thejohnfreeman force-pushed the clang-format branch 2 times, most recently from 76236aa to fcd322f Compare April 22, 2020 03:03
# This feature requires Git >= 2.24
# To use it by default in git blame:
# git config blame.ignoreRevsFile .git-blame-ignore-revs
09a4679b9eaafb430d1c19a83e5c3143ea34b745
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed this, but I think this needs to be updated now too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I pushed the fix. Can't wait for this to go in so I can stop recreating this changeset...

@thejohnfreeman thejohnfreeman force-pushed the clang-format branch 2 times, most recently from 199bda6 to bc01a9d Compare April 22, 2020 22:25
seelabs
seelabs previously approved these changes Apr 22, 2020
Copy link
Collaborator

@seelabs seelabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

cjcobb23
cjcobb23 previously approved these changes Apr 22, 2020
thejohnfreeman and others added 4 commits April 23, 2020 09:40
- Add missing `#include` in `ripple/core/JobTypeInfo.h`
- Protect version string from clang-format in
  `ripple/protocol/impl/BuildInfo.cpp`.
  `Builds/CMake/RippledVersion.cmake` searches for this line by pattern.
@thejohnfreeman thejohnfreeman dismissed stale reviews from cjcobb23 and seelabs via 331472d April 23, 2020 14:43
@thejohnfreeman thejohnfreeman force-pushed the clang-format branch 2 times, most recently from bc01a9d to 331472d Compare April 23, 2020 14:43
This was referenced Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants