Memory Usage explosion in parse command in version 0.2.37

[hernancorigliano]

Hi, I'd like to report an issue after upgrading from 0.2.36 to 0.2.37.

The parse command started having an issue in which it starts taking up massive amounts of system memory (RAM) indefinitely until the OS collapses.

The issue is not happening for previous versions where the parse command finishes instantly.

I have tried using the mentioned flags --omit_notes --omit_warnings to simplify the parsing of a large log, but this won't stop the issue.

Steps to reproduce:

• Build your app in Xcode 15.0.1
• Run parse command as xclogparser parse --project MyProject --reporter flatJson --output log.json
• The xclogparser process will slowly start taking up all available system memory. (Check the growing amount in Activity Monitor Memory tab)
• (Manually kill the process before the OS crashes)

Context:

• MacOS: Sonoma 14.1.2
• Device: MacbookPro M1
• Xcode: 15.0.1
• XCLogParser installation: 0.2.37 (Homebrew)

[Parabak]

+1

[seyfeddin]

+1

[BalestraPatrick]

Hello, unfortunately I don't have time to debug exactly what's wrong, but it must be one of the recently added contributions between the last two versions: v0.2.36...v0.2.37

I have a feeling it must be one of these 3 changes:

• Remove occurrences of single quotes in filenames #181
• Add SwiftCompile as detailType #182
• Add IDEActivityLogActionMessage support #189

If you can locally try to revert each of those commits and see which one fixes it, that would be helpful.

[Parabak]

I can confirm that rolling back to version 0.2.36 worked for me

[bcylin]

Tested locally. Reverting #182 avoids the massive RAM usage. But the underlying issue seems to be the BuildStep struct that has too many subSteps.

---

With SwiftCompile as a detail type, it includes each file's compilation, such as:

SwiftCompile normal arm64 /opt/homebrew/var/agent/builds/ip-123-45-67-890-internal/project/pipeline/.../Sources/.../file.swift (in target '...' from project '...') ... etc

I tried it in a project with ~7,000 Swift files, it adds more than 400,000 sub steps into the parsed result, nested in 5 levels down.

In both use cases of --reporter json and --reporter flatJson, the struct becomes too large that JSONEncoder can't handle. The memory usage shoots up on line 42:

XCLogParser/Sources/XCLogParser/reporter/JsonReporter.swift

Lines 39 to 44 in cd41cfc

	private func report<T: Encodable>(encodable: T, output: ReporterOutput) throws {
	let encoder = JSONEncoder()
	encoder.outputFormatting = .prettyPrinted
	let json = try encoder.encode(encodable)
	try output.write(report: json)
	}

An alternative workaround is to use --reporter summaryJson as it drops all the sub steps in the SummaryJsonReporter.

[BalestraPatrick]

Reverting in #199 and then will cut a new release. Thanks for reporting!

[InspektorKek]

Same problem with JSON, but not only on last version. Actually on each version I face that issue(

Context:
MacOS: Sonoma 14.2.1
Device: MacbookPro M3 Pro
Xcode: 15.1
XCLogParser installation: rake build v 0.2.36

[daltonclaybrook]

Any update on this? It's unfortunate that SwiftCompile is not supported.