Building a functional TypeScript pipe library for arrays

Check out the arr-pipe GitHub repo for more information and details on how to get started.

One of the core capabilities of a functional programming is piping output from one operator/function/etc. to another. In many languages this looks using the pipe operator |> to accomplish something like:

let square x = x * x
let addOne x = x + 1
5 |> square |> addOne  // Output: 26

This paradigm of declaratively building up a chain of operations is a powerful model for creating flexible and reusable code. I’ve found one of the most practical use cases for this programming style in web applications is building APIs to serve a recommended feed of items or search results. Let’s look at a theoretical, but practical use case that we are all familiar with: a vacation property rental app (think Airbnb, VRBO, Booking.com, etc.)

This type of recommendation + search results system usually has a few common requirements and characteristics:

1. There is usually a cached list of recommendations to show on the main “feed” when a user opens the app. In our example this is a list of properties based on your home location, or a set of interesting properties to grab your attention.
2. A separate search results system that filters results based on some criteria. For example, filter hotels based on location, date, beds, etc. This is usually based on a denormalized search index of the properties.
3. A requirement to only show available properties. This is important to consider when you deal with cached or indexed data.
4. Up-rank or inject sponsored listings that may or may not exactly match search criteria. This logic may not be related to the recommendations.
5. Apply a promotional price or discount based on user or location. This is usually implemented as some type of post processing step that is disjoint from the recommendation or search results retrieval.
6. Send recommendations via marketing emails. Often there is more than one medium for presenting recommendations that may or may not contain the up-ranking or promotional behavior.

Using a programming style that employs a pipeline of reusable operations helps to manage the complexity of building a system that meets these requirements.

In psuedocode this is what our “pipe” could look like for the recommended feed of listings:

fetch cached recommendations 
|> filter unavailable listings 
|> apply promotional pricing 

And we can build a similar, but slightly different pipe for returning search results:

fetch search results 
|> filter unavailable listings 
|> inject sponsored posts 
|> apply promotional pricing

While building this type of pipeline is possible to do with imperative programming, I believe a more declarative approach results in code that is easier to read, simpler to understand, and safer to modify. However, unlike more traditional functional programming languages, TypeScript does not have native support for a pipe operator. And while there are other libraries that have tackled this problem for general purpose functional programming, there are specific array concerns that I believe deserve their own attention. So, I built a small library called arr-pipe that helps to support this programming style with arrays.

Some goals I had while building:

1. Prioritize type safety. For example, you should not be able to construct a pipe where the output of one operation is not compatible with the input of the next.
2. Array specific utility functions. This includes common higher order operators like intersection and union which can be used to compose array results together
3. High test coverage
4. Support a variety of data types (numbers, objects, booleans, strings, undefined, nulls)

arr-pipe has one core function: pipe . It takes in a set of functions that each take an array as input and produce an array as output. The output of the first is passed as input to the next and so forth. With this main function we can construct powerful building blocks that can be composed together to create array processing pipelines.

Let’s take a look at what constructing a pipe looks like with arr-pipe:

import { 
    pipe, 
    union, 
    intersection
} from 'arr-pipe';

const lessThan = (num: number) => 
    (arr: (number[]) => arr.filter((item) => item < num));
const greaterThan = (num: number) => 
    (arr: (number[]) => arr.filter((item) => item > num));

const pipeline = pipe(
    union(greaterThan(8), lessThan(5)),
    intersection(lessThan(9), greaterThan(3))
)

const result = pipeline([
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10
])
// Input is [0...10]
// union of > 8 and < 5 = [0, 1, 2, 3, 4, 9, 10]
// intersection of < 9 and > 3 = [4]

Even though this is just a toy example for illustrative purposes you can see how you could extend this model to build pipelines for our hotel booking search results example.

In my next post I’ll go into detail about how the type signature for pipe is implemented to ensure type safety between operators. It’s an interesting bit of recursive typing that uses some more advanced TypeScript techniques.

Check out the arr-pipe GitHub repo for more information and details on how to get started.