Files
MLOPS/docker-compose/transform
2025-10-06 19:52:00 +00:00
..
2025-10-06 19:52:00 +00:00
2025-10-06 19:52:00 +00:00
2025-10-06 19:52:00 +00:00
2025-10-06 19:52:00 +00:00
2025-10-06 19:52:00 +00:00
2025-10-06 19:52:00 +00:00

= Modify the Wasm Transform in the Quickstart

This directory contains the Go source code (`transform.go`) for the data transform that is used in the Redpanda Self-Managed quickstart.
If you're following the quickstart, you *do not* need to modify or rebuild this code. The Docker Compose configuration automatically deploys a pre-built transform called `regex.wasm`.

However, if you want to customize the data transform logic, continue reading.

== Why customize the transform?

- **Custom filtering**: Filter by a different regex or apply multiple conditions.
- **Data manipulation**: Transform records before writing them out. For example, redacting sensitive data or combining fields.
- **Extended functionality**: Add advanced logging, error handling, or multi-topic routing.

== Prerequisites

You need the following:

- At least Go 1.20 installed.
+
[source,bash]
----
go version
----

- The Redpanda CLI (`rpk`) installed.

- A running Redpanda cluster. If you're using the local quickstart with Docker Compose, ensure the cluster is up and running. Or, point `rpk` to another Redpanda environment.

== Modify and deploy your transform

. Open link:transform.go[transform.go] and make your changes. For example:
+
--
- Change the regex logic to handle different use cases.
- Add environment variables to control new features.
- Extend the `doRegexFilter()` function to manipulate records.
--

. Compile your Go code into a `.wasm` file:
+
[source,bash]
----
rpk transform build
----
+
This command compiles your Go source and produces a `.wasm` file that you can deploy to Redpanda.

. Deploy the new transform.
+
If your Docker Compose setup already has a service to deploy the transform, you can restart that service.
+
Otherwise, you can deploy your updated `.wasm` manually using `rpk transform deploy`.

. Produce messages into the input topic. For example:
+
[source,bash]
----
echo '{"key":"alice@university.edu","value":"test message"}' | rpk topic produce logins
----

. Consume from the output topic. For example:
+
[source,bash]
----
rpk topic consume edu-filtered-domains --num 1
----

== Suggested reading

- link:https://docs.redpanda.com/current/reference/rpk/[Redpanda `rpk` CLI Reference^].
- link:https://docs.redpanda.com/current/develop/data-transforms/build/[Develop Data Transforms^].
- https://golang.org/ref/mod[Go Modules^] for managing dependencies and builds in Go.
- https://docs.docker.com/compose/[Docker Compose^] for customizing your environment.