Apache Flume 1.6.0 发布,日志服务器 - 开源中国社区
Apache Flume 1.6.0 发布,日志服务器
oschina 2015年06月03日

Apache Flume 1.6.0 发布,日志服务器

oschina oschina 发布于2015年06月03日 收藏 26 评论 2

【腾讯云】如何快速搭建微信小程序?>>>  

Apache Flume 1.6.0 发布,此版本现已提供下载:

http://flume.apache.org/download.html  

更新内容:

** Bug 修复

  • [FLUME-1793] - Unit test TestElasticSearchLogStashEventSerializer fails with IBM JDK

  • [FLUME-1934] - Spoolingdir source exception when reading multiple   zero size files

  • [FLUME-2024] - Add ExecSource flush timeout to Flume User Guide

  • [FLUME-2126] - Problem in elasticsearch sink when the event body is a complex field

  • [FLUME-2148] - Windows : Add  flume-env.ps1

  • [FLUME-2214] - FileChannelIntegrityTool needs documentation in user guide

  • [FLUME-2245] - HDFS files with errors unable to close

  • [FLUME-2337] - export JAVA_HOME in flume-env.sh.template and increase heap size

  • [FLUME-2346] - idLogFileMap in Log can lose track of file ids

  • [FLUME-2404] - Default maxReadBufferBytes might cause OOM and cause scribe source exit

  • [FLUME-2408] - Remove FLUME-1899 from Flume 1.5.0 change log.

  • [FLUME-2416] - Use CodecPool in compressed stream to prevent leak of direct buffers

  • [FLUME-2420] - HDFSEventSink#stop might throw ConcurrentModificationException

  • [FLUME-2425] - FileChannel should trim data and checkpoint directories

  • [FLUME-2432] - Update Kite dependency to 0.15.0

  • [FLUME-2438] - Make Syslog source message body configurable

  • [FLUME-2450] - Improve replay index insertion speed.

  • [FLUME-2466] - Embedded agent name cannot contain spaces

  • [FLUME-2470] - Kafka Sink and Source must use camel case for all configs.

  • [FLUME-2472] - SyslogUtils fails when system locale is not English

  • [FLUME-2475] - toLowerCase/toUpperCase used without Locale parameter to manipulate Enum

  • [FLUME-2479] - Kafka property auto.commit.enable is incorrect for KafkaSource

  • [FLUME-2480] - Typo in file channel exception: “Usable space exhaused”

  • [FLUME-2481] - TestFileChannelEncryption fails with System locale other than en_US

  • [FLUME-2482] - Race condition in File Channels’ Log.removeOldLogs

  • [FLUME-2486] - TestExecSource fails on some environments

  • [FLUME-2487] - SyslogParser has rounding errors in timestamp parsing

  • [FLUME-2488] - TestElasticSearchRestClient fails on Oracle JDK 8

  • [FLUME-2489] - Upgrade Apache Curator to 2.6.0

  • [FLUME-2492] - Flume’s Kafka Source doesn’t account time correctly

  • [FLUME-2495] - Kafka Source may miss events when channel is not available

  • [FLUME-2497] - TCP and UDP syslog sources parsing the timestamp incorrectly

  • [FLUME-2500] - Add a channel that uses Kafka

  • [FLUME-2505] - Test added in FLUME-2502 is flaky

  • [FLUME-2517] - Performance issue: SimpleDateFormat constructor takes 30% of HDFSEventSink.process()

  • [FLUME-2525] - flume should handle a zero byte .flumespool-main.meta file for the spooldir source

  • [FLUME-2538] - TestResettableFileInputStream fails on JDK 8

  • [FLUME-2556] - TestBucketPath.testDateFormatHours fails intermittently

  • [FLUME-2557] - DatasetSink doesn’t work with Kerberos

  • [FLUME-2560] - Kafka channel entries missing for output jar generation

  • [FLUME-2568] - Additional fix for TestReliableSpoolingFileEventReader

  • [FLUME-2578] - Kafka source throws NPE if Kafka record has null key

  • [FLUME-2586] - HDFS Sink should have an option to try rename even if close fails

  • [FLUME-2592] - Specify main manifest attribute in flume tools jar

  • [FLUME-2594] - Close Async HBase Client if there are large number of consecutive timeouts

  • [FLUME-2595] - Add option to checkpoint on file channel shutdown

  • [FLUME-2624] - Improve Hive Sink performance

  • [FLUME-2626] - Remove trustmanager-type from Thrift RPC client

  • [FLUME-2633] - Update Kite dependency to 1.0.0

  • [FLUME-2639] - SecureThriftRpcClient client privileged calls throw IllegalAccessError

  • [FLUME-2645] - ipFilter.rules property name is wrong

  • [FLUME-2654] - Memory channel error message related to byteCapacity needs refining

  • [FLUME-2657] - Upgrade to Hive 1.0

  • [FLUME-2658] - Upgrade to Hbase 1.0

  • [FLUME-2664] - Site profile fails due to dependency issues

  • [FLUME-2665] - Update documentation for hdfs.closeTries based on FLUME-2586

  • [FLUME-2670] - Modify dependencies to not pull hadoop/hive related jars

  • [FLUME-2673] - Remove unused import in TestLog

  • [FLUME-2678] - Upgrade xalan to 2.7.2 to take care of CVE-2014-0107 vulnerability

  • [FLUME-2679] - Make hbase-1 the default profile

** 依赖升级

  • [FLUME-2443] - org.apache.hadoop.fs.FSDataOutputStream.sync() is deprecated in hadoop 2.4

** 文档

  • [FLUME-1594] - Document JMX options

  • [FLUME-2509] - Add description for properties in Failover Sink Processor

  • [FLUME-2630] - Update documentation for Thrift Src/Sink SSL support and Kerberos support

  • [FLUME-2635] - Documentation for zookeeper dynamic configuration in flume

  • [FLUME-2668] - Document SecureThriftRpcClient/SecureRpcClientFactory in Flume Developer Guide

** 改进

  • [FLUME-1334] - Write an startscript for flume agents on Windows

  • [FLUME-1491] - Dynamic configuration from Zookeeper watcher

  • [FLUME-1521] - Document the StressSource

  • [FLUME-1710] - JSONEvent.getBody should not return null

  • [FLUME-1920] - Test case TestFileChannel fails when flume is built from paths containing the string “hadoop”

  • [FLUME-2237] - Example config in User Guide to use same notation for Agent name

  • [FLUME-2246] - event body data size can make it configurable for logger sinker

  • [FLUME-2273] - ElasticSearchSink: Add handling for header substitution in indexName

  • [FLUME-2373] - Enable configuration to switch thrift source serialization protocol

  • [FLUME-2385] - Flume spans log file with “Spooling Directory Source runner has shutdown” messages at INFO level

  • [FLUME-2401] - Optionally Compress Backup Checkpoint

  • [FLUME-2439] - Update DatasetSink for Kite 0.15.0

  • [FLUME-2462] - Remove use of deprecated methods in DatasetSink

  • [FLUME-2463] - Add support for Hive and HBase datasets to DatasetSink

  • [FLUME-2469] - DatasetSink should load dataset when needed, not at startup

  • [FLUME-2499] - Include Kafka Message Key in Event Header, Updated Comments

  • [FLUME-2502] - Spool source’s directory listing is inefficient

  • [FLUME-2558] - Update javadoc for StressSource

  • [FLUME-2562] - Metrics for Flafka components

  • [FLUME-2591] - Kite DatasetSink 2.0

  • [FLUME-2613] - Tool/script for deleting individual message from queue

  • [FLUME-2642] - Limit kerberos relogin attempt

** 新特性

  • [FLUME-1734] - Create a Hive Sink based on the new Hive Streaming support

  • [FLUME-2242] - Flume Sink and Source for Apache Kafka

  • [FLUME-2426] - Support interceptors in the Embedded Agent

  • [FLUME-2431] - Add simple regex search-and-replace interceptor

  • [FLUME-2570] - Add option to not pad date fields

  • [FLUME-2574] - SSL Support for Thrift Rpc

  • [FLUME-2631] - End to End authentication in Flume

更新日志和文档:

http://flume.apache.org/releases/1.6.0.html 

Flume 是一个分布式、可靠和高可用的服务,用于收集、聚合以及移动大量日志数据,使用一个简单灵活的架构,就流数据模型。这是一个可靠、容错的服务。

Agent component diagram

本站文章除注明转载外,均为本站原创或编译。欢迎任何形式的转载,但请务必注明出处,尊重他人劳动共创开源社区。
转载请注明:文章转载自 开源中国社区 [http://www.oschina.net]
本文标题:Apache Flume 1.6.0 发布,日志服务器
分享
评论(2)
最新评论
0
应该是日志收集器
0
Great!
顶部