kafka-connect-github-source 正在参加 2021 年度 OSC 中国开源项目评选,请投票支持!
kafka-connect-github-source 在 2021 年度 OSC 中国开源项目评选 中已获得 {{ projectVoteCount }} 票,请投票支持!
2021 年度 OSC 中国开源项目评选 正在火热进行中,快来投票支持你喜欢的开源项目!
2021 年度 OSC 中国开源项目评选 >>> 中场回顾
kafka-connect-github-source 获得 2021 年度 OSC 中国开源项目评选「最佳人气项目」 !
授权协议 MIT License
开发语言 Java
操作系统 跨平台
软件类型 开源软件
所属分类 云计算云原生
开源组织
地区 不详
投 递 者 首席测试
适用人群 未知
收录时间 2021-12-09

软件简介

Learning

This project is a companion repository to the Apache Kafka Connect course on Udemy.

https://links.datacumulus.com/kafka-connect-coupon

Kafka Connect Source GitHub

This connector allows you to get a stream of issues and pull requests from your GitHub repository, using the GitHub Api: https://developer.github.com/v3/issues/#list-issues-for-a-repository

Issues are pulled based on updated_at field, meaning any update to an issue or pull request will appear in the stream.

The connector writes to topic that is great candidate to demonstrate log compaction. It's also a fun way to automate your GitHub workflow.

It's finally aimed to be an educative example to demonstrate how to write a Source Connector a little less trivial than the FileStreamSourceConnector provided in Kafka.

Contributing

This connector is not perfect and can be improved, please feel free to submit any PR you deem useful.

Configuration

name=GitHubSourceConnectorDemo
tasks.max=1
connector.class=com.simplesteph.kafka.GitHubSourceConnector
topic=github-issues
github.owner=kubernetes
github.repo=kubernetes
since.timestamp=2017-01-01T00:00:00Z
# I heavily recommend you set those two fields:
auth.username=your_username
auth.password=your_password

Running in development

Note: Java 8 is required for this connector. Make sure config/worker.properties is configured to wherever your kafka cluster is

./build.sh
./run.sh 

The simplest way to run run.sh is to have docker installed. It will pull a Dockerfile and run the connector in standalone mode above it.

Deploying

Note: Java 8 is required for this connector.

TODO

展开阅读全文

代码

评论

点击引领话题📣
暂无内容
发表了博客
{{o.pubDate | formatDate}}

{{formatAllHtml(o.title)}}

{{parseInt(o.replyCount) | bigNumberTransform}}
{{parseInt(o.viewCount) | bigNumberTransform}}
没有更多内容
暂无内容
发表了问答
{{o.pubDate | formatDate}}

{{formatAllHtml(o.title)}}

{{parseInt(o.replyCount) | bigNumberTransform}}
{{parseInt(o.viewCount) | bigNumberTransform}}
没有更多内容
暂无内容
0 评论
0 收藏
分享
OSCHINA
登录后可查看更多优质内容
返回顶部
顶部
返回顶部
顶部