74 lines
2.4 KiB
Plaintext
74 lines
2.4 KiB
Plaintext
= Modify the Wasm Transform in the Quickstart
|
|
|
|
This directory contains the Go source code (`transform.go`) for the data transform that is used in the Redpanda Self-Managed quickstart.
|
|
If you're following the quickstart, you *do not* need to modify or rebuild this code. The Docker Compose configuration automatically deploys a pre-built transform called `regex.wasm`.
|
|
|
|
However, if you want to customize the data transform logic, continue reading.
|
|
|
|
== Why customize the transform?
|
|
|
|
- **Custom filtering**: Filter by a different regex or apply multiple conditions.
|
|
- **Data manipulation**: Transform records before writing them out. For example, redacting sensitive data or combining fields.
|
|
- **Extended functionality**: Add advanced logging, error handling, or multi-topic routing.
|
|
|
|
== Prerequisites
|
|
|
|
You need the following:
|
|
|
|
- At least Go 1.20 installed.
|
|
+
|
|
[source,bash]
|
|
----
|
|
go version
|
|
----
|
|
|
|
- The Redpanda CLI (`rpk`) installed.
|
|
|
|
- A running Redpanda cluster. If you're using the local quickstart with Docker Compose, ensure the cluster is up and running. Or, point `rpk` to another Redpanda environment.
|
|
|
|
== Modify and deploy your transform
|
|
|
|
. Open link:transform.go[transform.go] and make your changes. For example:
|
|
+
|
|
--
|
|
- Change the regex logic to handle different use cases.
|
|
- Add environment variables to control new features.
|
|
- Extend the `doRegexFilter()` function to manipulate records.
|
|
--
|
|
|
|
. Compile your Go code into a `.wasm` file:
|
|
+
|
|
[source,bash]
|
|
----
|
|
rpk transform build
|
|
----
|
|
+
|
|
This command compiles your Go source and produces a `.wasm` file that you can deploy to Redpanda.
|
|
|
|
. Deploy the new transform.
|
|
+
|
|
If your Docker Compose setup already has a service to deploy the transform, you can restart that service.
|
|
+
|
|
Otherwise, you can deploy your updated `.wasm` manually using `rpk transform deploy`.
|
|
|
|
. Produce messages into the input topic. For example:
|
|
+
|
|
[source,bash]
|
|
----
|
|
echo '{"key":"alice@university.edu","value":"test message"}' | rpk topic produce logins
|
|
----
|
|
|
|
. Consume from the output topic. For example:
|
|
+
|
|
[source,bash]
|
|
----
|
|
rpk topic consume edu-filtered-domains --num 1
|
|
----
|
|
|
|
== Suggested reading
|
|
|
|
- link:https://docs.redpanda.com/current/reference/rpk/[Redpanda `rpk` CLI Reference^].
|
|
- link:https://docs.redpanda.com/current/develop/data-transforms/build/[Develop Data Transforms^].
|
|
- https://golang.org/ref/mod[Go Modules^] for managing dependencies and builds in Go.
|
|
- https://docs.docker.com/compose/[Docker Compose^] for customizing your environment.
|