What is a Brick?

A Raven Brick is the basic building block that get chained into a pipeline structure. We use bricks in Playbooks and Flows and as usecases. Each Brick has specific purpose. You can consider these the building blocks of your data architecture.

Raven Bricks:

  • Do one task and only that task
  • Are completely isolated by design
  • Raven orchestrates and scales bricks
  • Take a serializable in- and output format,

Bricks come in three basic types, namely Source, Transform and Action.

Source#

Used in Raven Flows | Usecases#

A source is a brick that represents an endpoint from where you can retrieve data. It can be used in multiple flows to fetch data from various sources. There are two ways to retrieve data from a source-brick:

  • Push: You can send data to a source-brick using its block-id. This is useful when you have data that you want to send to the source for processing or storage.
  • Pull: Alternatively, the source-brick can pull data from the endpoint by itself. This can be useful when you want to periodically fetch data from the endpoint at specific intervals.

To create a source-brick, you need to specify the endpoint from where you want to fetch data. The endpoint can be a URL, a database, or any other data source that can be accessed through an API or some other data retrieval mechanism.

Once you have created a source-brick, you can add it to any flow where you want to access data from that endpoint.

Transform#

Used in Raven Playbooks | Flows#

Transforms are used to modify data that is passed between actions. They can be used to filter data, add metadata, or perform calculations. Transforms are especially useful when working with data from multiple sources or when the data needs to be formatted in a specific way.

Action#

Used in Raven Playbooks | Flows#

Actions are tasks that are performed by the playbook. They can be anything from sending an email to making an API call. Each action has specific inputs and outputs, and can be customized to suit the needs of the playbook.

Trigger#

Used in Raven Playbooks#

Triggers are events that initiate a playbook. They can be internal events, such as a scheduled time, or external events, such as a webhook. When a trigger is activated, it starts the playbook. The trigger is a simple variation on the action-brick.

Scaling#

Our bricks are build to easily scale.

All data processing steps are done in bricks. If a backlog of messages starts to build up in a brick-queue, our bricks can easily be scaled out to balance the workload.

Besides the bricks, due to the multi-layer architecture of our queuing mechanism, this part also easily scales.

With Raven's architecture build with scalability in mind, we can process huge amounts of streaming data simultaneously.