Onyx 0.9.0 发布,分布式计算系统

发布于 2016年03月24日
收藏 10

Onyx 0.9.0 发布了。

Onyx 是什么?

  • Onyx 是一个无中心、支持云、容错的分布式计算系统

  • 使用 Clojure 编写

  • 支持批处理和流处理混合

  • 提供信息模型用于描述和构建分布式工作流

  • 竞争对手: Storm, Cascading, Map/Reduce, Dryad, Apache Sqoop, Twitter Crane 等等


  • 实时的事件流处理

  • 连续计算

  • ETL

  • 在 MapReduce 进行数据转换

  • 数据清理

  • 数据摄入和存储媒介传输


  • API breaking change: onyx/id in peer-config and env-config was renamed to :onyx/tenancy-id

  • API breaking change: :aggregation/fn was renamed to :aggregation/create-state-update.

  • API breaking change: changed the signatures of: :aggregation/init, :aggregation/create-state-update, :aggregation/apply-state-update, and :aggregation/super-aggregation-fn. All now take the window as the first argument.

  • API breaking change: internal messaging functions no longer take the event map as an argument. Note, this will break any plugins that manually manage the ack count, such as plugins using async callbacks.

  • API breaking change: triggers and refinements functions are now resolved to vars via namespace lookup e.g. Trigger example: :trigger/on :segment -> :trigger/on :onyx.triggers/segment, Refinement example: :trigger/refinement :accumulating -> :trigger/refinement :onyx.refinements/accumulating,

  • Breaking change in onyx-metrics: :metrics/workflow-name was deprecated in favor of metadata map. See onyx-metrics changes

  • Bug fix: fixed exceptions losing their main .getCause message when thrown inside a lifecycle

  • New feature: Jobs now support metadata by including a metadata map with job submission data e.g. {:workflow ... :catalog ... :job-metadata {:name "myjobname"}}. This makes it easier to correlate information about jobs, query the cluster for jobs, etc.

  • Design change: Implemented trigger refinements in terms of state updates. In order to implement a refinement, one must now implement a refinement calls map, analagous to the aggregation calls map. This takes the form {:refinement/create-state-update (fn [trigger state state-event]) :refinement/apply-state-update (fn [trigger state entry])}

  • Enhancement: Lifecycles can now catch flow condition exceptions through :lifeycycle/handle-exception from the :lifecycle/execute-flow-conditions and :lifecycle/apply-fn phases.

  • Enhancement: loosened the peer configuration schema needed for log subscription

  • Dependency change: Upgraded org.clojure/clojure to 1.8.0


转载请注明:文章转载自 OSCHINA 社区 [http://www.oschina.net]
本文标题:Onyx 0.9.0 发布,分布式计算系统