Kafka - Avro - Value serialization error

The default serialization format for Kafka is Avro. I mentioned how to use this in C# yesterday.

Today I got into trouble when trying to send a specific message.

	// Construct the message to send
	var message = new Message<Null, PageViewEvent>
	{
	Value = new PageViewEvent() { pageUrl=$"http://kafkaexample/{value}", browser=null}
	};
	await producer.ProduceAsync("kafka-topic-name", message);

view raw PublishMessageWithNullValue.cs hosted with ❤ by GitHub

I changed the example a little bit to explicitly point out the issue. You see in the code above that I set the value for ‘Browser’ to ‘null’.

When trying to send this message it failed with the following error:

Local: Value serialization error

Let’s have a look at the related avro schema:

The problem is that in the schema is specified that the Browser field should a value of type string. ‘Null’ is not a valid value for string. This explains why it fails.

To solve this I have two options;

1) Either change the code to send an empty string instead of null:

	// Construct the message to send
	var message = new Message<Null, PageViewEvent>
	{
	Value = new PageViewEvent() { pageUrl=$"http://kafkaexample/{value}", browser=String.Empty}
	};
	await producer.ProduceAsync("kafka-topic-name", message);

view raw PublishMessageFixed.cs hosted with ❤ by GitHub

2) Either update the schema to allow null values for the browser field:

	{
	"namespace": "MessageTypes",
	"type": "record",
	"doc": "A simple message type.",
	"name": "PageViewEvent",
	"fields": [
	{
	"name": "pageUrl",
	"type": "string"
	},
	{
	"name": "browser",
	"type": ["null","string"]
	}
	]
	}

view raw PageViewEventNull.avsc hosted with ❤ by GitHub

More about Avro: https://en.wikipedia.org/wiki/Apache_Avro