report package history in PkgEval summaries

[LilithHafner]

In theory, the result should look like this:

I changed from list to table because aligning histories can elucidate when historical failures are a package's fault vs. Julia's fault.

Also in theory, it should be possible to load historical results from local storage on the Nanosoldier machines. However, I am not familiar with those machines, so I implemented a portable version that downloads them from github so that I can test my implementation locally.

[maleadt]

Really nice, thanks!

Also in theory, it should be possible to load historical results from local storage on the Nanosoldier machines.

Yeah I'll probably rework it so that we re-use the local storage, since the AWS buckets have a fairly limited lifetime associated with them.

[LilithHafner]

This doesn't download from aws buckets, only https://github.com/JuliaCI/NanosoldierReports/tree/master/pkgeval/by_date/.

Is it possible to view the markdown summary generated in CI?

[maleadt]

This doesn't download from aws buckets, only https://github.com/JuliaCI/NanosoldierReports/tree/master/pkgeval/by_date/.

Oh, right. Still, doesn't seem ideal to hit GitHub for these, especially when we have the reports repository cloned locally. In fact, we already use the local clone to find the previous date:

Nanosoldier.jl/src/jobs/PkgEvalJob.jl

Lines 291 to 302 in 26ce3fb

	function retrieve_daily_pkgeval_data!(cfg, date)
	dailydir = joinpath(reportdir(cfg), "pkgeval", "by_date", datedirname(date))
	isdir(dailydir) || return nothing
	# NOTE: we don't actually use the data from the previous day, just the
	# build properties, since packages upgrades might cause failures too.
	dbpath = joinpath(dailydir, "db.json")
	isfile(dbpath) || return nothing
	db = JSON.parsefile(dbpath)
	return db
	end

Is it possible to view the markdown summary generated in CI?

It's currently not published anywhere, no.

[maleadt]

Great, let's try this out!

[LilithHafner]

The markdown report in CI did not successfully show a plot, but I suppose we'll see if it works in production

Edit: it did not JuliaLang/julia@3c64bc3#commitcomment-123412580