LocalFileSink

Behavior

LocalFileSink writes messages into local file system. The name of each written file is formatted as <datetime when the file is created><hostname><random UUID>

datetime is formatted as PyyyyMMddTHHmmss. LocalFileSink is an asynchronous sink, and employs an internal queue for buffering messages. We can specify which type of message queue to use, maximum time to buffer data, and the batch size of queued messages.

Properties

If the default value is not specified, it is required.

Properties	Description	type	Default
outputDir	directory path where files are stored	String
writer	Its type can be 'text' or 'sequence'. 'text' means text file and 'sequence' means Hadoop sequence file. User can specify Hadoop codec class.	FileWriter	TextFileWriter without any compression
maxFileSize	when the file size reaches to this value, local file sink will rotate the file: closing current file, creating a new file, and starting to write messages to the new file.	long	104857600(100MB)
rotationPeriod	local file sink will rotate files with this period. It can be an ISO-formatted period string such as PT10m, or milliseconds value.	String	PT1m
minPercentFreeDisk	The minimum percentage of free disk space. When the free disk space drops below this threshold, local file sink will abort the operation, and notifies Suro server, which in turn will stop taking traffic.	int	50
notice	It describes Notice type for how to notify its progress.	Notice	QueueNotice
queue4Sink	Which queue would be used	MessageQueue4Sink	Memory based queue with the capacity as 10,000 messages
batchSize	Maximum number of queued messages	int	200
batchTimeout	Maximum time to buffer data	int	1000(ms)