[Understanding Software Architecture Spec] Pipeline Architecture

Pipeline Architecture Topology

The basic principle behind Unix terminal shell languages such as Bash and Zsh is familiarity with this architecture.
And Pipeline Architecture is used in functional programing languages like MapReduce Programing model.

Specification

Pipe

• Pipes form a one-way communication between filters, which is typically P2P connection.
• In this architecture, pipes are generally unidirectional (not broadcast) for reasons of communication channel ministrations between filters.

Filter

Filters are self-contained and independent of other filters.
And filter performs only one task.

• Producer Filter
  • Starting point of the process.
• Transformer Filter
  • Receives input, optionally transforms some or all of the data, takes it, and sends it to the output pipe.
• Tester Filter
  • Receives input, examines it for one or more criteria, and optionally generates output based on the examination
• Consumer Filter
  • The end point of the pipeline flow.
  • The consumer persists the end result of the pipeline process in a database or displays it in the user interface.

Example Usage with Kafka

Architecture Evaluation