dazn-lambda-powertools 正在参加 2021 年度 OSC 中国开源项目评选,请投票支持!
dazn-lambda-powertools 在 2021 年度 OSC 中国开源项目评选 中已获得 {{ projectVoteCount }} 票,请投票支持!
2021 年度 OSC 中国开源项目评选 正在火热进行中,快来投票支持你喜欢的开源项目!
2021 年度 OSC 中国开源项目评选 >>> 中场回顾
dazn-lambda-powertools 获得 2021 年度 OSC 中国开源项目评选「最佳人气项目」 !
授权协议 MIT License
开发语言 JavaScript
操作系统 跨平台
软件类型 开源软件
所属分类 云计算Serverless 系统
开源组织
地区 不详
投 递 者 首席测试
适用人群 未知
收录时间 2021-12-02

软件简介

CircleCI

DAZN Lambda Powertools

dazn-lambda-powertools is a collection of middlewares, AWS clients and helper libraries that make working with lambda easier.

Motivation

Writing Lambdas often involves the bootstrapping of specific tooling, like reading and forwarding on correlation-id's, emitting logs on a lambda timeout, and more.

Re-writing and maintaining this bootstrapping logic into every individual lambda can be a pain, so to prevent this re-work we created dazn-lambda-powertools.

Usage

The quickest way to get setup is to use the opinionated pattern basic package.

npm install @dazn/lambda-powertools-pattern-basic

const wrap = require('@dazn/lambda-powertools-pattern-basic')

module.exports.handler = wrap(async (event, context) => {
  return 42
})

For more control, you can pick and choose from the individual packages.

Powertools and Middy

All of the powertool middlewares use the middy library (v2.x), and therefore adhere to the middy API.

However, the other tools such as the clients are generic.

What's in Powertools

An integrated suite of powertools for Lambda functions that reduces the effort to implement common lamdba tasks, such as dealing with correlation-ids.

  • support correlation IDs

  • debug logs are turned off in production, and are instead sampled for 1% of invocations

  • debug logging decisions are respected by all the functions on a call chain

  • HTTP requests always report both latency as well as response count metrics

Overview of available tools

  • logger: structured logging with JSON, configurable log levels, and integrates with other tools to support correlation IDs and sampling (only enable debug logs on 1% of invocations)

  • correlation IDs: create and store correlation IDs that follow the DAZN naming convention

  • correlation IDs middleware: automatically extract correlation IDs from the invocation event

  • sample logging middleware: enable debug logging for 1% of invocations, or when upstream caller has made the decision to enable debug logging

  • obfuscater middleware: allows you to obfuscate the invocation event so that sensitive data (e.g. PII) is not logged accidentally

  • log timeout middleware: logs an error message when a function invocation times out

  • stop infinite loop middleware: stops infinite loops

Client libraries

  • http client: HTTP client that automatically forwards any correlation IDs you have captured or created, and records both latency as well as response count metrics

  • CloudWatchEvents client: CloudWatchEvents client that automatically forwards any correlation IDs you have captured or created when you put events to an event bus

  • EventBridge client: EventBridge client that automatically forwards any correlation IDs you have captured or created when you put events to an event bus

  • SNS client: SNS client that automatically forwards any correlation IDs you have captured or created when you publish a message to SNS

  • SQS client: SQS client that automatically forwards any correlation IDs you have captured or created when you publish a message to SQS

  • Kinesis client: Kinesis client that automatically forwards any correlation IDs you have captured or created when you publish record(s) to a Kinesis stream

  • Firehose client: Firehose client that automatically forwards any correlation IDs you have captured or created when you publish record(s) to a Firehose delivery stream

  • Step Functions client: Step Functions client that automatically forwards any correlation IDs you have captured or created when you start an execution

  • Lambda client: Lambda client that automatically forwards any correlation IDs you have captured or created when you invokes a Lambda function directly

  • DynamoDB client: DynamoDB client that automatically forwards any correlation IDs you have captured or created when you perform put or update operations against DynamoDB. These correlation IDs are then available to functions processing these events from the table's DynamoDB Stream.

Patterns

  • basic template for a function: wrapper for your function that applies and configures the function to work well with datadog metrics and sample logging

  • obfuscate template: basic template (above) + obfuscate the invocation event so sensitive data is obfuscated in the after and onError handlers.

Installing the powertools

via NPM

Package Install command
cloudwatchevents-client npm install @dazn/lambda-powertools-cloudwatchevents-client
correlation-ids npm install @dazn/lambda-powertools-correlation-ids
dynamodb-client npm install @dazn/lambda-powertools-dynamodb-client
eventbridge-client npm install @dazn/lambda-powertools-eventbridge-client
firehose-client npm install @dazn/lambda-powertools-firehose-client
http-client npm install @dazn/lambda-powertools-http-client
kinesis-client npm install @dazn/lambda-powertools-kinesis-client
lambda-client npm install @dazn/lambda-powertools-lambda-client
logger npm install @dazn/lambda-powertools-logger
middleware-correlation-ids npm install @dazn/lambda-powertools-middleware-correlation-ids
middleware-log-timeout npm install @dazn/lambda-powertools-middleware-log-timeout
middleware-obfuscater npm install @dazn/lambda-powertools-middleware-obfuscater
middleware-sample-logging npm install @dazn/lambda-powertools-middleware-sample-logging
middleware-stop-infinite-loop npm install @dazn/lambda-powertools-middleware-stop-infinite-loop
pattern-basic npm install @dazn/lambda-powertools-pattern-basic
pattern-obfuscate npm install @dazn/lambda-powertools-pattern-obfuscate
sns-client npm install @dazn/lambda-powertools-sns-client
sqs-client npm install @dazn/lambda-powertools-sqs-client
step-functions-client npm install @dazn/lambda-powertools-step-functions-client

via Lambda layer

You can also deploy the layer via our SAR app, which you can deploy either via this page (click Deploy and follow the instructions) or using CloudFormation/Serverless framework/AWS SAM:

DaznLambdaPowertoolsLayer:
  Type: AWS::Serverless::Application
  Properties:
    Location:
      ApplicationId: arn:aws:serverlessrepo:us-east-1:570995107280:applications/dazn-lambda-powertools
      SemanticVersion: <enter latest version>

and reference the output Outputs.LayerVersion to get the ARN of the layer to reference in your function. e.g. Fn::GetAtt: [DaznLambdaPowertoolsLayer, Outputs.LayerVersion].

You can find the latest version of the SAR app in the lerna.json file here, in the version property.

Design goal

Compliance with best practices around logging and monitoring should be the default behaviour. These tools make it simple for you to do the right thing and gets out of your way as much as possible.

Individually they are useful in their own right, but together they're so much more useful!

The middlewares capture incoming correlation IDs, and the logger automatically includes them in every log message, and the other clients (HTTP, Kinesis, SNS, etc.) would also automatically forward them on to external systems.

Even if your function doesn't do anything with correlation IDs, the tools make sure that it behaves correctly as these correlation IDs flow through it.

Did you consider monkey-patching the clients instead?

Instead of forcing you to use dazn-powertools AWS clients, we could have monkey patched the AWS SDK clients (which we already do in the tests). We could also monkey patch Node's http module (like what Nock does) to intercept HTTP requests and inject correlation IDs as HTTP headers.

We could apply the monkey patching when you apply the correlation IDs middleware, and your function would "automagically" forward correlation IDs without having to use our own client libraries. That way, as a user of the tools, you could use whatever HTTP client you wish, and can use the standard SDK clients as well.

We did entertain this idea, but I wanted to leave at least one decision for you to make. The rationale is that when things go wrong (e.g. unhandled error, or bug in our wrapper code) or when they don't work as expected (e.g. you're using an AWS SDK client that we don't support yet), at least you have that one decision to start debugging (change the require statement to use the official library instead of our own to see if things things still work).

Useful commands

bootstrapping locally

Because of the inter-dependencies between packages, it can be tricky to test your changes haven't broken another package.

You can use Lerna CLI to bootstrap all the dependencies with the current local version:

lerna bootstrap

run all tests

npm test

run tests for a specific package

PKG=correlation-ids npm run test-package

create a new package

lerna create <name of package>

and follow the instruction to bootstrap the new project.

Contributing

Please read our contribution guide to see how you can contribute towards this project.

展开阅读全文

代码

评论

点击引领话题📣
暂无内容
发表了博客
{{o.pubDate | formatDate}}

{{formatAllHtml(o.title)}}

{{parseInt(o.replyCount) | bigNumberTransform}}
{{parseInt(o.viewCount) | bigNumberTransform}}
没有更多内容
暂无内容
发表了问答
{{o.pubDate | formatDate}}

{{formatAllHtml(o.title)}}

{{parseInt(o.replyCount) | bigNumberTransform}}
{{parseInt(o.viewCount) | bigNumberTransform}}
没有更多内容
暂无内容
tar 存在拒绝服务漏洞
拒绝服务
tar 是用于 Node.js 的全功能 Tar。此软件包的受影响版本容易受到正则表达式拒绝服务 (ReDoS) 的攻击。
MPS-2022-14081
2022-08-08 18:23
Ajv 输入验证错误漏洞
输入验证不恰当
Ajv 6.12.2版本中的ajv.validate()函数中存在输入验证错误漏洞。攻击者可利用该漏洞执行代码或造成拒绝服务。
CVE-2020-15366 MPS-2020-10525
2022-08-08 18:23
property-expr 输入验证错误漏洞
输入验证不恰当
property-expr 2.0.3之前版本中存在原型污染漏洞。目前尚无此漏洞的相关信息,请随时关注CNNVD或厂商公告。
CVE-2020-7707 MPS-2020-11771
2022-08-08 18:23
stevemao trim-off-newlines 处理逻辑错误漏洞
trim-off-newlines是NPM的用于删除换行符。 stevemao trim-off-newlines存在处理逻辑错误漏洞,该漏洞源于所有版本的 trim-off-newlines 都容易通过字符串处理受到正则表达式拒绝服务 (ReDoS) 的攻击。
CVE-2021-23425 MPS-2021-17627
2022-08-08 18:23
jsonpointer类型混淆漏洞
使用不兼容类型访问资源(类型混淆)
jsonpointer是开源的一个简单的JSON寻址的软件包。jsonpointer存在安全漏洞,该漏洞源于网络系统或产品的代码开发过程中存在设计或实现不当的问题。 目前没有详细的漏洞细节提供。
CVE-2021-23807 MPS-2021-19846
2022-08-08 18:23
lodash 命令注入漏洞
代码注入
lodash是一个提供模块化、性能和附加功能的现代 JavaScript 实用程序库。 4.17.21 之前的 Lodash 版本容易通过模板函数进行命令注入。
CVE-2021-23337 MPS-2021-2638
2022-08-08 18:23
node-tar 路径遍历漏洞
路径遍历
node-tar是一款用于文件压缩/解压缩的软件包。 npm node-tar 存在路径遍历漏洞,该漏洞源于4.4.18、5.0.10和6.1.9之前的npm包“tar”(又名node-tar)存在任意文件创建覆盖和任意代码执行漏洞。攻击者可利用该漏洞访问受限目录之外的位置。
CVE-2021-37713 MPS-2021-28489
2022-08-08 18:23
istanbul-reports 存在通过 window.opener 访问使用指向不受信任目标的 Web 链接漏洞
通过 window.opener 访问使用指向不受信任目标的 Web 链接
由于指向 https://istanbul 的链接中没有 rel 属性,因此该软件包的受影响版本容易受到反向 Tabnabbing 的攻击。
MPS-2022-13797
2022-08-08 18:23
yup 存在原型污染漏洞
原型污染
是的,是一个死简单对象模式验证。此软件包的受影响版本容易通过 .
MPS-2022-14161
2022-08-08 18:23
simple-git-hooks存在未明漏洞
命令注入
simple-git-hooks是一个应用软件。一个简单的git钩子经理小型项目。simple-git-hooks 3.5.0之前版本存在安全漏洞,攻击者利用该漏洞进行命令注入。
CVE-2022-24066 MPS-2022-5073
2022-08-08 18:23
npm dot-prop 安全漏洞
原型污染
4.2.1 之前的 dot-prop npm 包版本和 5.1.1 之前的 5.x 版本中的原型污染漏洞允许攻击者向 JavaScript 语言构造(例如对象)添加任意属性。
CVE-2020-8116 MPS-2020-1734
2022-08-08 18:23
minimist 输入验证错误漏洞
原型污染
minimist是一款命令行参数解析工具。 minimist 1.2.2之前版本存在输入验证错误漏洞。攻击者可借助‘constructor’和‘__proto__’ payload利用该漏洞添加或修改Object.prototype的属性。
CVE-2020-7598 MPS-2020-3516
2022-08-08 18:23
Npm Node-tar 后置链接漏洞
node-tar是一款用于文件压缩/解压缩的软件包。 Npm Node-tar 中存在后置链接漏洞,该漏洞源于产品未对特殊字符做有效验证。攻击者可通过该漏洞在其他路径创建恶意文件。
CVE-2021-37712 MPS-2021-28488
2022-08-08 18:23
trim-newlines 安全漏洞
拒绝服务
trim-newlines是一个修改换行符的npm包。 trim-newlines 存在安全漏洞,该漏洞源于应用于Node.js在3.0.1与4.0.1版本及之前版本中.end()方法存在相关问题。
CVE-2021-33623 MPS-2021-7398
2022-08-08 18:23
nodejs 资源管理错误漏洞
拒绝服务
nodejs是是一个基于ChromeV8引擎的JavaScript运行环境通过对Chromev8引擎进行了封装以及使用事件驱动和非阻塞IO的应用让Javascript开发高性能的后台应用成为了可能。 nodejs-glob-parent 存在安全漏洞,该漏洞源于正则表达式拒绝服务。
CVE-2020-28469 MPS-2021-7827
2022-08-08 18:23
is-my-json-valid 存在拒绝服务漏洞
拒绝服务
is-my-json-valid 是一个 JSONSchema / 有序验证器,它使用代码生成非常快。此软件包的受影响版本容易通过样式格式受到正则表达式拒绝服务 (ReDoS) 的攻击。
MPS-2022-13795
2022-08-08 18:23
lodash 存在拒绝服务漏洞
拒绝服务
lodash 是一个现代 JavaScript 实用程序库,提供模块化、性能和附加功能。此软件包的受影响版本容易通过 setWith 和 set 函数受到原型污染。
MPS-2022-13842
2022-08-08 18:23
lodash 存在拒绝服务漏洞
拒绝服务
lodash 是一个现代 JavaScript 实用程序库,提供模块化、性能和附加功能。由于对 CVE-2020-8203 的修复不完整,此软件包的受影响版本容易受到 zipObjectDeep 中的原型污染。
MPS-2022-13841
2022-08-08 18:23
jsonpointer 存在拒绝服务漏洞
拒绝服务
jsonpointer 是一个简单的 JSON 寻址。此软件包的受影响版本容易通过 set 函数受到原型污染。
MPS-2022-13815
2022-08-08 18:23
npm node-fetch 安全漏洞
不加限制或调节的资源分配
node-fetch 2.6.1和3.0.0-beta版本中存在安全漏洞。该漏洞源于内容大小超过限制时,将永远不会抛出FetchError。
CVE-2020-15168 MPS-2020-12719
2022-08-08 18:23
没有更多内容
加载失败,请刷新页面
点击加载更多
加载中
下一页
0 评论
0 收藏
分享
OSCHINA
登录后可查看更多优质内容
返回顶部
顶部