LocalFileSink

Behavior

LocalFileSink is writing its messages into the file in the local file system. Its name is formatted as <datetime when the file is created><hostname><random UUID>

datetime is formatted as PyyyyMMddTHHmmss. LocalFileSink is asynchronous sink and it contains internal queue for messages. We can specify which type of message queue to use, maximum time to buffer data and the maximum number of queued messages.

properties

If the default value is not specified, it is required.

Properties	Description	type	Default
outputDir	directory path where files are stored	String
writer	Its type can be 'text' or 'sequence'. 'text' means text file and 'sequence' means Hadoop sequence file. It is getting codec argument denoting Hadoop codec class.	FileWriter	TextFileWriter without any compression
maxFileSize	when the file size reaches to this value, local file sink will rotate the file - close current file, create a new file and write message set to the new file.	long	104857600(100MB)
rotationPeriod	local file sink will rotate the file with this period. It can be ISO period string such as PT10m or milliseconds value.	String	PT1m
minPercentFreeDisk	when the disk is filled and the free disk space percentage is getting bigger than this threshold value, local file sink will abort the operation and it will make Suro server stop taking the traffic.	int	50
notify	It describes Notify interface for how to notify its progress.	Notify	QueueNotify
queue4Sink	Which queue would be used	MessageQueue4Sink	Memory based queue with the capacity as 10,000 messages
batchSize	Maximum number of queued messages	int	200
batchTimeout	Maximum time to buffer data	int	1000(ms)