Skip to main content

Creating and managing Transformations within Brick Management

PurposeThis documentation provides a guide on the process of creating and incorporating a Transformation using the Brick Manager.
CreatedNovember 21, 2023

Understanding Transformations​

Transformations are used for modifying the outcomes of a Table using the VRL language (Vector Remap Language). With the implementation of VRL files, users can define the desired transformations.

For more information on VRL follow link below:

Learn more about VRL

Steps to Create and Add Transforms to Your Table​

To initiate the creation of a Transform Brick, navigate to the Settings and access the Brick Management section. Click on the "CREATE BRICK" button, fill in the necessary fields, and include VRL code detailing the intended transformation.

After successfully creating the Transform Brick, navigate to the Tables section within Settings. Here you will be able to start configuring your Table, Start by pressing the "CREATE TABLE" button and fill in the necessary fields where you will find an option to add your previously created Transformation Brick. Select the desired Transformation to proceed.

Note that while transformations alter the results displayed in the table, the data within the table itself remains unaltered.

Example of Transformation Brick in VRL​

Given the data representing an HTTP request:

{
    "message": "Hello VRL",
    "foo": "delete me",
    "http_status": "200"
}

Transformation example:


# Step 1: Remove the 'foo' field from the data
del(.foo)

# Step 2: Add the current time to a 'timestamp' field
.timestamp = now()

# Step 3: Parse HTTP status code into a local variable
http_status_code = parse_int!(.http_status)

# Step 4: Add a field that indicates whether the HTTP request has succeeded
if http_status_code >= 200 && http_status_code <= 299 {
    .status = "success"
} else {
    .status = "error"
}

Output after Transformation:

{
    "http_status": "200",
    "message": "Hello VRL",
    "status": "success",
    "timestamp": "2023-12-11T14:43:57.530Z"
}

Transformations typically involve:

  • Converting values, such as timestamps.
  • Altering the data structure, like flattening a nested arrangement.
  • Mapping values to different representations, for instance, mapping 1, 2, 3 to "LOW," 4, 5, 6 to "MEDIUM," and so on.
  • Parsing data, as seen in the parsing of a log line.
  • Filtering specific elements or criteria.