Server exception when using enum in schema

[phemmer]

Describe the bug
When trying to create a topic with a schema containing an enum, an exception is thrown on the server.

To Reproduce
Steps to reproduce the behavior:

1. Run the following go code:

package main

import (
	"fmt"

	"github.com/apache/pulsar/pulsar-client-go/pulsar"
)

var schema = `
{ "name": "test", "type": "record", "fields": [
	{ "name": "testfield", "type": "enum", "symbols": ["one","two"] }
]}
`

func main() {
	client, _ := pulsar.NewClient(pulsar.ClientOptions{URL: "pulsar://localhost:6650"})
	_, err := client.CreateProducerWithSchema(pulsar.ProducerOptions{Topic: "test"}, pulsar.NewJsonSchema(schema, nil))
	fmt.Printf("Error: %s\n", err)
}

Expected behavior
No error

Actual behavior
On client:

2020/04/19 00:23:46.004 c_client.go:68: [info] WARN  | ClientConnection:925 | [127.0.0.1:43932 -> 127.0.0.1:6650] Received error response from server: 1 -- req_id: 0
2020/04/19 00:23:46.004 c_client.go:68: [info] ERROR | ProducerImpl:219 | [persistent://public/default/test, ] Failed to create producer: UnknownError
2020/04/19 00:23:46.004 c_client.go:68: [info] INFO  | ProducerImpl:474 | Producer - [persistent://public/default/test, ] , [batching  = off]
Error: Failed to create Producer: UnknownError

On server:

00:23:46.003 [bookkeeper-ml-workers-OrderedExecutor-2-0] ERROR org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:43932] Failed to create topic persistent://public/default/test
java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: Can not resolve JsonSchema 'type' id of "record", not recognized as one of standard values: [STRING, NUMBER, INTEGER, BOOLEAN, OBJECT, ARRAY, NULL, ANY]
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:673) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_242]
	at org.apache.pulsar.broker.service.BrokerService$2.lambda$openLedgerComplete$1(BrokerService.java:868) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:719) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.uniRunStage(CompletableFuture.java:731) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.thenRun(CompletableFuture.java:2023) ~[?:1.8.0_242]
	at org.apache.pulsar.broker.service.BrokerService$2.openLedgerComplete(BrokerService.java:861) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$7(ManagedLedgerFactoryImpl.java:341) ~[org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_242]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_242]
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeComplete(ManagedLedgerFactoryImpl.java:329) ~[org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$3.operationComplete(ManagedLedgerImpl.java:446) ~[org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$3.operationComplete(ManagedLedgerImpl.java:436) ~[org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper.lambda$null$5(MetaStoreImplZookeeper.java:223) ~[org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [org.apache.pulsar-managed-ledger-2.5.0.jar:2.5.0]
	at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.43.Final.jar:4.1.43.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: java.lang.IllegalArgumentException: Can not resolve JsonSchema 'type' id of "record", not recognized as one of standard values: [STRING, NUMBER, INTEGER, BOOLEAN, OBJECT, ARRAY, NULL, ANY]
	at com.fasterxml.jackson.module.jsonSchema.JsonSchemaIdResolver.typeFromId(JsonSchemaIdResolver.java:66) ~[com.fasterxml.jackson.module-jackson-module-jsonSchema-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:156) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:97) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:254) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:68) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4202) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3266) ~[com.fasterxml.jackson.core-jackson-databind-2.10.1.jar:2.10.1]
	at org.apache.pulsar.broker.service.schema.validator.StructSchemaDataValidator.validate(StructSchemaDataValidator.java:60) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.pulsar.broker.service.schema.validator.SchemaDataValidator.validateSchemaData(SchemaDataValidator.java:42) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.pulsar.broker.service.schema.validator.SchemaRegistryServiceWithSchemaDataValidator.putSchemaIfAbsent(SchemaRegistryServiceWithSchemaDataValidator.java:92) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.pulsar.broker.service.AbstractTopic.addSchema(AbstractTopic.java:194) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.pulsar.broker.service.ServerCnx.tryAddSchema(ServerCnx.java:1489) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$19(ServerCnx.java:934) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0]
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_242]
	... 23 more

Desktop (please complete the following information):

• OS: Linux, Fedora 31

Additional context
Pulsar version: 2.5.0

And can we please get some better error handling in that section of the code on the server? That exception is extremely confusing. It took me way too long to figure out the problem was with the enum.

[phemmer]

Sigh. I just ran across another issue in the same vein.
The following schema also generates an exception:

{ "name": "test", "type": "record", "fields": [
	{ "name": "data", "type": "record", "fields": [] }
]}

Do schemas even work?

[codelipenghui]

Looks you have to specify a namespace for the schema definition. You can check here https://github.com/apache/pulsar/blob/master/pulsar-client-go/pulsar/schema_test.go#L42.

[phemmer]

@codelipenghui specifying a namespace has no effect. Namespace is optional in avro.
The code you reference uses neither enum nor an embedded record, which are the types documented as non-functional in this issue report.

[phemmer]

So it looks like I had the syntax for enums wrong. It's just records and booleans which don't appear to be working right. I'm going to close this issue since it's deviated quite a bit. I'll see about a new issue with clean reproduction case.