Welcome to cfn-kafka-admin’s documentation! ¶
Manage Apache Kafka topics, schemas and ACLs from AWS CloudFormation ¶
This project was built to
-
Control ACLs, Topics and Schemas via GitOps and controlled “as code” in AWS CloudFormation
-
Allow non-kafka power users to have a simple declarative way to define what they need in Kafka
Install ¶
To install the CLI, simply run
pip install cfn-kafka-admin
To install the lambda functions, see functions_installation
Usage ¶
This project is made of two parts that will allow you to manage ACLs, Topics and their associated schemas, using AWS CloudFormation.
If you are using AWS MSK, which we highly recommend, a majority of these functionalities can be done with native AWS implementations. However, this has been used and tested with MSK too and works (schemas excluded, see AWS Glue Schema Registry )
1. The Lambda Functions ¶
First of all, you need to deploy the AWS Lambda Functions into your AWS Account in order to use them as CustomResources in AWS CloudFormation. The reason we went for Lambda and Custom Resource vs new AWS Resources, is because you need access to brokers and Schema registry, which might be private.
2. The CLI ¶
The files in which you will define your ACLs, Topics and Schemas follow a strict JSON definition, to ensure that input is conform and help ensure successful deployment. When the CLI runs, it will generate a new CloudFormation template that will be what’s used to define the resources.
Contents: