cmd/go: go list -m -u -json all does not produce valid JSON

[przglo]

When go list -m -u -json all is issued it produces invalid JSON.

What version of Go are you using (go version)?

go1.11

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

linux/amd64

What did you do?

Created the following two files in a directory.

foo.go:

package testpackage

import (
	"fmt"
	humanize "github.com/dustin/go-humanize"
)

func TestFun() {
  fmt.Printf("%s\n", humanize.Bytes(1234567))
}

go.mod:

module testmodule

require github.com/dustin/go-humanize v0.0.0-20180713052910-9f541cc9db5d // indirect

Issued this from within the directory: go list -m -u -json all

What did you expect to see?

Anything that could be parsed using the encoding/json package. A JSON array would be nice:

[{
	"Path": "testmodule",
	"Main": true,
	"Dir": "[some path]",
	"GoMod": "[some path]"
},
{
	"Path": "github.com/dustin/go-humanize",
	"Version": "v0.0.0-20180713052910-9f541cc9db5d",
	"Time": "2018-07-13T05:29:10Z",
	"Indirect": true,
	"Dir": "[some path]",
	"GoMod": "[some path]"
}]

What did you see instead?

{
	"Path": "testmodule",
	"Main": true,
	"Dir": "[some path]",
	"GoMod": "[some path]"
}
{
	"Path": "github.com/dustin/go-humanize",
	"Version": "v0.0.0-20180713052910-9f541cc9db5d",
	"Time": "2018-07-13T05:29:10Z",
	"Indirect": true,
	"Dir": "[some path]",
	"GoMod": "[some path]"
}

which is a concatenation of JSONs and cannot be easily parsed using encoding/json.

[myitcv]

@przglo go list has always behaved in this way to my knowledge (arguably because it allows, if required, results to be consumed as they become available, instead of needing to wait for the end of the input)

You can easily use the following approach to parse the output (which does wait for all of the output; the alternative would be to connect the stdout of the command to the decoder):

type Module struct { 
	// ... 
}

// ...

out, err := exec.Command("go", "list", "-m", "all").Output()

// ...

dec := json.NewDecoder(bytes.NewReader(out))
for {
	var m Module
	if err := dec.Decode(&m); err != nil {
		if err == io.EOF {
			break
		}
		log.Fatalf("reading go list output: %v", err)
	}

	// do something with m
}

[dominikh]

This is indeed working as intended. It's a stream of JSON objects, which is easy to parse with encoding/json (as @myitcv demonstrates).

Besides, this isn't going to be changed now, tools are depending on the existing format.