Omit unnecessary matrix variables in the CI workflow #1420

Job	Run time
dist	1m 26s
windows (amd64, UCRT64)	3m 57s
macos (amd64, x86_64-apple-darwin)	2m 11s
windows (i386, MINGW32)	3m 54s
macos (arm64, arm64-apple-darwin)	2m 0s
linux (amd64, x86_64-linux-gnu)	58s
linux (arm64, aarch64-linux-gnu)	1m 23s
linux (armel, arm-linux-gnueabi)	1m 0s
linux (armhf, arm-linux-gnueabihf)	1m 1s
linux (i386, i686-linux-gnu)	1m 0s
linux (mips, mips-linux-gnu)	54s
linux (mips64, mips64-linux-gnuabi64)	58s
linux (mips64el, mips64el-linux-gnuabi64)	54s
linux (mips64r6, mipsisa64r6-linux-gnuabi64)	1m 11s
linux (mips64r6el, mipsisa64r6el-linux-gnuabi64)	56s
linux (mipsel, mipsel-linux-gnu)	54s
linux (mipsr6, mipsisa32r6-linux-gnu)	53s
linux (mipsr6el, mipsisa32r6el-linux-gnu)	54s
linux (powerpc, powerpc-linux-gnu)	1m 1s
linux (ppc64el, powerpc64le-linux-gnu)	1m 1s
linux (riscv64, riscv64-linux-gnu)	51s
linux (s390x, s390x-linux-gnu)	58s
docker	22s
release	0s
	30m 37s

Provide feedback