Alluxio 1.2.0 发布,提升集成对象存储可用性

发布于 2016年07月24日
收藏 10

Alluxio 1.2.0 发布了,Alluxio 是一个高容错的分布式文件系统,允许文件以内存的速度在集群框架中进行可靠的共享,类似Spark和 MapReduce。通过利用lineage信息,积极地使用内存,Alluxio的吞吐量要比HDFS高300多倍。Alluxio都是在内存中处理缓 存文件,并且让不同的 Jobs/Queries以及框架都能内存的速度来访问缓存文件。


  • Richer feature set and improved performance for S3 under storage:

    • Support for Amazon’s official s3 client library, aws-sdk through the s3a:// scheme.

    • S3a achieves up to 4x higher write throughput compared to s3n.

    • S3a achieves up to 2x higher metadata throughput compared to s3n.

    • Support for advanced features including instance profile login, server side encryption, and secure http protocol.

    • Jets3t client library is still available under the s3n:// scheme.

  • Improved performance for Swift under storage: Alluxio 1.2 streamlines operations to Swift under storages, cutting down the number of requests to Swift per command by up to 4x compared to previous versions of Alluxio.

  • Transparent ACLs between Alluxio and under storages with ACL support:Automatically propagates user, group, and permissions information from Alluxio to the under storage. Automatically fetches user, group, and permissions information from the under storage into Alluxio when new file metadata are discovered. Enable this feature by setting the flag to true.

  • Improved file system metadata management: Alluxio 1.2 manages metadata over 3x more effectively compared to Alluxio 1.1.1, allowing users to store and access triple the amount of metadata compared to before.

  • More flexible under storage metadata management: Alluxio now provides finer grained control over metadata loaded from the under storage. For each operation which may need to consult the under storage, users can specify if they want to always sync new data, only sync new data on the first access, or never sync new data.


转载请注明:文章转载自 开源中国社区 []
本文标题:Alluxio 1.2.0 发布,提升集成对象存储可用性