Add emit extension block symbols option to dump-symbol-graph and SymbolGraphOptions

[theMomax]

Adds support for emitting extension block symbols to the dump-symbol-graph command and other SymbolGraphOptions.

Motivation:

Extension block symbols with the respective command line flags -emit-extension-block-symbols and -omit-extension-block-symbols were introduced in swiftlang/swift#59047 and swiftlang/swift#61637 as part of an ongoing effort to solve swiftlang/swift-docc#210. These flags should be exposed in higher-level commands such as swift package dump-symbol-graph as well as the package plugin API, so that users and package manager plugins such as apple/swift-docc-plugin can access them more easily.

Modifications:

This PR adds the flag-pair --emit-extension-block-symbols/--omit-extension-block-symbols to the DumpSymbolGraph command and exposes the emitExtensionBlocks option in PluginToHostMessage.SymbolGraphOptions. SymbolGraphExtract was also extended by an emitExtensionBlockSymbols option and its extractSymbolGraph function adds the respective compiler command line flag accordingly.

Result:

swift package dump-symbol-graph --emit-extension-block-symbols dumps symbol graph files that extension block symbol format. The default behavior does not change. The --omit-extension-block-symbols flag will be used to explicitly disable the feature once the default behavior has been changed to --emit-extension-block-symbols in the future.

Package Plugins can specify the emitExtensionBlocks property requesting symbol graphs, e.g.:

import PackagePlugin

// ...
try getSymbolGraph(for: target, options: PackageManager.SymbolGraphOptions(
            minimumAccessLevel: targetMinimumAccessLevel,
            includeSynthesized: true,
            includeSPI: false,
            emitExtensionBlocks: true
        ))

The default behavior for package plugins also does not change.

[neonichu]

@swift-ci please smoke test

[tomerd]

looks good once CI is green

[tomerd]

should this be Boolean instead?

[theMomax]

AFAIK this is the only way to achieve a flag inversion with a custom name, i.e. with a prefix other than enable/disable or no. See the ArgumentParser docs here:

To create a flag with custom names for a Boolean value[...], define an enumeration that conforms to the EnumerableFlag protocol.

IMO the emit/omit prefixes describe the semantics best and I wouldn't want to have entirely different names for SwiftPM and compiler commands.

[tomerd]

ok

[neonichu]

@swift-ci please smoke test

[theMomax]

@neonichu Test failures are expected until the CI uses a toolchain that was built after Dec. 2nd (and therefore includes this code adding the -omit-extension-block-symbols flag to swift-driver). This last run used one from Dec. 1st.

[theMomax]

@neonichu Could you run tests again, please? They should pass now.

[franklinsch]

@swift-ci please smoke test

[theMomax]

It seems like the self hosted macOS runner uses Swift 5.5, which doesn't know the flags this PR introduces. I'm a little confused as to why a contribution to the main branch (which is supposed to land in Swift 5.8) must be compatible with Swift 5.5. I couldn't find any CLI flags in SwiftPM that are hidden based on the Swift version, so I assumed each version of SwiftPM only needs to be compatible with the compiler/driver version of the respective toolchain it is part of. However, looking at the branches, it seems like you're currently maintaining compatibility down to Swift 5.3 from the main branch.

Should I therefore wrap all my changes in Swift 5.8 guards so that one can actually run the Swift 5.5 compiler's swift-symbolgraph-extract with the resulting version of SwiftPM? Or would it be better to just make a Swift 5.7 branch before this PR lands to avoid messing up the code-base?

@tomerd or @neonichu could you please tell me what is the best strategy for this?

[daniel-grumberg]

@swift-ci please test

[neonichu]

@theMomax It is intentional that we are supporting older versions, we have an API to check which flags are available in the used compiler to guard availability of newer features like this. 5.5 is a little bit older than we’d like, but generally speaking we do want to retain the ability of using and developing SwiftPM with toolchains that aren’t the latest.

A compilation conditional is not sufficient to handle this since we do want people to mix a newer SwiftPM with older compilers at runtime as much as possible. You can look at the use of this API to see how conditionality could be done based on the availability of the flags you need rather than versions.

[theMomax]

@neonichu thank you for the explanation! I pushed a fix that basically just drops the flag with a warning if it is not supported by the driver. Is that how it's supposed to be done? I couldn't find a precedent in the code where a user-provided command argument had to be dropped.

let extensionBlockSymbolsFlag = emitExtensionBlockSymbols ? "-emit-extension-block-symbols" : "-omit-extension-block-symbols"
if DriverSupport.checkSupportedFrontendFlags(flags: [extensionBlockSymbolsFlag.trimmingCharacters(in: ["-"])], fileSystem: fileSystem) {
    commandLine += [extensionBlockSymbolsFlag]
} else {
    print("warning: dropped \(extensionBlockSymbolsFlag) flag because it is not supported by this compiler version")
}

People would still see the flag in the help dialog of SwiftPM even if they use it with a compiler version that doesn't support it. Since they manually configured their setup to use SwiftPM/compiler versions from different toolchain versions I guess that's ok, though. I also wouldn't know an intended way to conditionally disable an ArgumentParser flag at runtime, but if necessary I'm sure it's possible somehow.

[QuietMisdreavus]

@swift-ci Please smoke test

[neonichu]

@swift-ci please smoke test

[neonichu]

@theMomax emitting a warning like you're doing seems like the right thing to do

[theMomax]

Looks like this is ready to go. Thanks for your help @neonichu !

[compnerd]

@shahmishal why did this not trigger the Windows build? This seems to have broken the windows build.