Main game loop

I want to start with something basic. We need a class that will contain our game state and draw the screen. Let’s create a game.rb with the following content:

class Game
  SLEEP_INTERVAL = 0.2

  def run
    loop do
      draw_screen
      sleep SLEEP_INTERVAL
    end
  end

  private

  def draw_screen
    system "clear"

    puts Time.now
  end
end

Game.new.run

Source is here

Run it (using ruby ./game.rb) and you see that it updates the current time every SLEEP_INTERVAL seconds. Nothing fancy yet!

Rendering the map

Our next goal is to render a level map. Let’s keep the map in the file called map.txt, which will be in the same directory as a game.rb. We need five types of objects:

• an empty place (s);
• a start point—a place where player appears (p);
• a final point—an exit from the level (d);
• a tree—the cell that could not be entered (t);
• an enemy—something that kills the player when they appear on the same place (e).

Feel free to design you own map, or use mine:

tsssstsdst
tsssssesst
tstssssset
tstsssssst
tsstssssst
tssssstsst
tstssstsst
tpstsstsst

Here is a class that parses the file and renders trees and empty spaces:

TREE, SPACE = '🌲', "・"

Level = Struct.new(:map, :enemies, :player, :door)

class LevelBuilder
  def initialize(filepath)
    @filepath = filepath
  end

  MAPPING = { 't' => TREE, 's' => SPACE }

  def build
    Level.new.tap do |level|
      level.enemies = []

      level.map = File.readlines(@filepath).map do |line|
        line.sub("\n", "").chars.map do |c|
          MAPPING[c] || SPACE
        end
      end
    end
  end
end

Note that we do not handle start location, final location and enemies yet. Instead, we treat them as empty spaces: since we will have a specific logic tied to them, we will add it later.

Now we need to update our game class to draw (and redraw) the field:

class Game
  SLEEP_INTERVAL = 0.2

  def run
    @level = LevelBuilder.new("./map.txt").build

    loop do
      draw_screen
      sleep SLEEP_INTERVAL
    end
  end

  private

  def draw_screen
    system "clear"

    @level.each do |row|
      row.each do |cell|
        print cell
      end
      puts
    end
  end
end

The only thing it does is just prints whatever we have in in the map field. The only thing that might be unfamiliar to you is system "clear", which just cleans up the terminal.

Source is here

Adding complex objects

Now we need to render player, enemies and door. Let’s extend our Level class definition to support them and decide how we will draw them on the screen:

PLAYER, ENEMY, DOOR, TREE, SPACE = '🧙', '👻', '🚪', '🌲', "・"

Level = Struct.new(:map, :enemies, :player, :door)

After that, let’s define another struct called DynamicObject to represent them:

DynamicObject = Struct.new(:row_idx, :col_idx, :kind)

Now we need to adjust our LevelBuilder to handle new objects. Note that when we find one of those we assign them to separate fields and treat them as empty spaces on the map, because they can change their location:

class LevelBuilder
  def initialize(filepath)
    @filepath = filepath
  end

  MAPPING = { 't' => TREE, 's' => SPACE }

  def build
    Level.new.tap do |level|
      level.enemies = []

      level.map = File.readlines(@filepath).map.with_index do |line, row_idx|
        line.strip.chars.map.with_index do |c, col_idx|
          case c
          when 'e'
            level.enemies << DynamicObject.new(row_idx, col_idx, :enemy)
            SPACE
          when 'p'
            level.player = DynamicObject.new(row_idx, col_idx, :player)
            SPACE
          when 'd'
            level.door = DynamicObject.new(row_idx, col_idx, :door)
            SPACE
          else
            MAPPING[c]
          end
        end
      end
    end
  end
end

Finally, we need to update Game#draw_screen to support our new primitives:

def draw_screen
  system "clear"

  @level.map.each_with_index do |row, row_idx|
    row.each_with_index do |cell, col_idx|
      if @level.player.row_idx == row_idx && @level.player.col_idx == col_idx
        print PLAYER
      elsif @level.door.row_idx == row_idx && @level.door.col_idx == col_idx
        print DOOR
      elsif @level.enemies.find { |enemy| enemy.row_idx == row_idx && enemy.col_idx == col_idx }
        print ENEMY
      else
        print cell
      end
    end
    puts "\n"
  end
end

You can find the whole snippet here

How to read user input

Our next goal is to find a way to control the character. We want to be able to move it up, down, left and right. Let’s start with the code that checks if any key is pressed right now.

class Game
  # ...

  private

  def get_pressed_key
    begin
      system('stty raw -echo')
      (STDIN.read_nonblock(4).ord rescue nil)
    ensure
      system('stty -raw echo')
    end
  end
end

This looks like a bit cryptic, so we will examine it line by line:

• stty raw -echo turns off echoing of input keystrokes;
• STDIN.read_nonblock(4) reads 4 bytes from user input, and .ord returns the integer representation of the entered character (e.g., 'w'.ord == 19);
• stty -raw echo turns echoing back on.

Let’s call this method inside the game loop and use the value to update player’s position:

class Game
  # ...

  def run
    @level = LevelBuilder.new("./map.txt").build

    loop do
      draw_screen

      new_player_position = DynamicObject.new(@level.player.row_idx, @level.player.col_idx, :player)
      new_player_position.move(get_pressed_key)

      @level.player = new_player_position

      sleep SLEEP_INTERVAL
    end
  end

  # ...
end

Finally, we have to implement DynamicObject#move. Let’s use WASD keys to control movements:

UP, DOWN, RIGHT, LEFT = 119, 115, 100, 97

DynamicObject = Struct.new(:row_idx, :col_idx, :kind) do
  def move(dir)
    case dir
    when RIGHT then self.col_idx += 1
    when LEFT then self.col_idx -= 1
    when UP then self.row_idx -= 1
    when DOWN then self.row_idx += 1
    end
  end
end

Source is here

Try it out! You can see that player position is updated on the screen.

If game feels a bit slow to you—try to play with SLEEP_INTERVAL value.

Collision handling

If you ran the previous example, you might have noticed that player can step on trees, ghosts, door and even leave the screen. The reason is that we update the position regardless of what is placed under it. Let’s fix that!

First of all, we need to add a method that will check new position and tell us what will happen if we move object there. Note that we will make this method generic: it will be able to check if passed object collides with any of objects we pass. We will use it to handle other collisions later.

There can be multiple outcomes:

• :out_of_border is returned when object attempts to leave the border of the level;
• :tree, :ghost, :door or :player is returned when object bumps into the corresponding object.

This is how Game#run looks like:

class Game
  # ...

  def run
    @level = LevelBuilder.new("./map.txt").build

    loop do
      draw_screen

      new_player_position = DynamicObject.new(@level.player.row_idx, @level.player.col_idx, :player)
      new_player_position.move(get_pressed_key)

      case check_collision(new_player_position.row_idx, new_player_position.col_idx, @level.enemies + [@level.door])
      when :door
        puts "🎉 Level passed 🎉"
        break
      when :enemy
        puts "☠️ You died ☠️"
        break
      when nil
        @level.player = new_player_position
      end

      sleep SLEEP_INTERVAL
    end
  end

  # ...
end

Now we should implement Game#check_collision:

class Game
  # ...

  private

  def check_collision(row_idx, col_idx, objects)
    return :out_of_border if row_idx < 0 || row_idx >= @level.map.length || col_idx < 0 || col_idx >= @level.map[0].length
    return :tree if @level.map[row_idx][col_idx] == TREE
    objects.find { _1.row_idx == row_idx && _1.col_idx == col_idx }&.kind
  end
  # ...
end

Source is here

Try to run the game and see that now we handle collisions properly: there’s no way to cross the border, step on the tree and game ends when you bump into the ghost or door.

Adding fancy AI to our ghosts

Our ghosts are not dangerous at all, because they can’t move, let’s address that! First of all, we need to change our main loop to trigger movement. Moreover, we should check the collisions after that to make sure that ghost did not bump into the player:

class Game
  # ...

  def run
    @level = LevelBuilder.new("./map.txt").build

    loop do
      move_enemies
      draw_screen

      case check_collision(@level.player.row_idx, @level.player.col_idx, @level.enemies)
      when :enemy
        puts "☠️ You died ☠️"
        break
      end

      new_player_position = DynamicObject.new(@level.player.row_idx, @level.player.col_idx, :player)
      new_player_position.move(get_pressed_key)

      case check_collision(new_player_position.row_idx, new_player_position.col_idx, @level.enemies + [@level.door])
      when :door
        puts "🎉 Level passed 🎉"
        break
      when :enemy
        puts "☠️ You died ☠️"
        break
      when nil
        @level.player = new_player_position
      end

      sleep SLEEP_INTERVAL
    end
  end

  # ...
end

Now we can implement the smart AI–powered algorithm that makes decision on enemy movement:

class Game
  # ...

  def move_enemies
    @level.enemies.each_with_index do |enemy, idx|
      next if rand(1) > 0.8

      new_enemy = DynamicObject.new(enemy.row_idx, enemy.col_idx, :enemy)
      new_enemy.move([RIGHT, LEFT, UP, DOWN].sample)
      @level.enemies[idx] = new_enemy if check_collision(new_enemy.row_idx, new_enemy.col_idx, [@level.door]).nil?
    end
  end

  # ...

Source is here

Extracting the rendering logic

I have a feeling that our Game class becomes some kind of a God object: it handles user input, screen rendering and movement logic. Let’s try to extract all rendering–related code to the separate class:

class Screen
  def render_level(level)
    system "clear"

    level.map.each_with_index do |row, row_idx|
      row.each_with_index do |cell, col_idx|
        if level.player.row_idx == row_idx && level.player.col_idx == col_idx
          print PLAYER
        elsif level.door.row_idx == row_idx && level.door.col_idx == col_idx
          print DOOR
        elsif level.enemies.find { |enemy| enemy.row_idx == row_idx && enemy.col_idx == col_idx }
          print ENEMY
        else
          print cell
        end
      end
      puts "\n"
    end
  end

  def render_death_message = puts "☠️ You died ☠️"
  def render_level_passed_message = puts "🎉 Level passed 🎉"
end

With this change we can easily replace terminal with a web page or even print game state to the paper!

Now we can change the main loop Game class and drop #draw_screen method:

class Game
  # ...

  def run
    screen = Screen.new

    @level = LevelBuilder.new("./map.txt").build

    loop do
      move_enemies
      screen.render_level(@level)

      case check_collision(@level.player.row_idx, @level.player.col_idx, @level.enemies)
      when :enemy
        screen.render_death_message
        break
      end

      new_player_position = DynamicObject.new(@level.player.row_idx, @level.player.col_idx, :player)
      new_player_position.move(get_pressed_key)

      case check_collision(new_player_position.row_idx, new_player_position.col_idx, @level.enemies + [@level.door])
      when :door
        screen.render_level_passed_message
        break
      when :enemy
        screen.render_death_message
        break
      when nil
        @level.player = new_player_position
      end

      sleep SLEEP_INTERVAL
    end
  end

  # ...
end

Final code is here

Let’s take a break here: we built more or less playable game in 138 lines. I have some more ideas for you to practice:

• try to add fireballs that are created when player presses SPACE and which should eliminate ghosts;
• try to add more static objects (like houses 🏠) to the level—you will probably need to refactor LevelBuilder#build to some kind of DSL;
• add a way to have more than one level (probably you will need to turn Game class into the state machine);
• build up level generator to make the game endless.

I will use RSpec, but you can easily port these approaches to any other testing framework you prefer.

Queries and types

There are two approaches of testing data fetch: you can check either requests or types. In case of requests, you can have a single spec file that fetches a lot of data. If you decide to go with types—you should have a separate spec per type.

I personally prefer the second approach: it’s more verbose (and you end up with more files) but also gives you more granular control over the spec and makes it harder to miss something.

You can even set up Rubocop to enforce every type have a corresponding spec file

What is the responsibility of the GraphQL type? Technically speaking, it’s a serializer: you give it some data, it transforms the data and returns the result. You can even extract the transformation to the separate PORO class and test it separately.

Here is an example of the spec I usually create:

describe Types::PostType do
  # you can configure RSpec to add this to all specs
  # inside spec/graphql folder
  include GraphqlHelpers

  let(:query) do
    <<~GRAPHQL
      query Post($postId: ID!) {
        post(postId: $postId) {
          id
          title
          content
          author {
            id
          }
        }
      }
    GRAPHQL
  end

  let(:post) { create :post }
  let(:variables) { { postId: post.id } }

  specify do
    perform_request

    expect(gql_errors).to eq(nil)

    resolved_object = gql_data.dig("post")

    # Ideally you should also make sure that you fetch all fields that are
    # resolved by this type
    expect(resolved_object).to match(
      "id" => post.id.to_s,
      "title" => post.title,
      "content" => post.content,
      "author" => {
        "id" => post.author.id.to_s
        # note that we do not check contents of the author field, because
        # it's more straightforward to test Author separately
      }
    )
  end
end

There are three things that I care about:

• HTTP request is performed;
• there are no errors (or there are expected errors, in rare cases);
• there is some data returned and contents make sense.

Do not confuse top–level errors with data–level errors—check out my huge post on that topic

If you do have some complex logic inside the type—you can also add some checks to this spec.

Now let’s inspect the GraphqlHelpers module I used:

module GraphqlHelpers
  extend ActiveSupport::Concern

  included do
    let(:variables) { { input: input } }
    let(:input) { {} }
  end

  def perform_request
    # you will probably handle auth here
    post "/graphql", # replace with your endpoint
      params: JSON.dump(request_params),
      xhr: true
  end

  def json_response_body
    JSON.parse(response.body)
  end

  def gql_data
    json_response_body["data"]
  end

  def gql_errors
    json_response_body["errors"]
  end

  private

  def request_params
    { query: query }.tap do |params|
      params[:variables] = variables if defined?(variables)
    end
  end
end

This module encapsulates the boilerplate logic of performing HTTP request and parsing the response, which should be common for all your type specs.

Mutations

Imagine that we have all types covered with specs, and now we want to test mutations. What is mutation in GraphQL? It’s the same thing as query, but we also expect some changes in the state.

As a result, we only care about three facts:

• data was updated;
• mutation returns some data (but we don’t need to test the whole type!);
• no errors were returned.

For example:

describe Mutations::CreatePost do
  # you can configure RSpec to add this to all specs
  # inside spec/graphql folder
  include GraphqlHelpers

  let(:query) do
    <<~GRAPHQL
      mutation CreatePost($title: String!, $content: String!) {
        createPost(title: $title, content: $content) {
          id
        }
      }
    GRAPHQL
  end

  let(:title) { "First post" }
  let(:content) { "Bla bla bla" }
  let(:variables) { { title:, content: } }

  specify do
    expect { perform_request }.to change(Post, :count).by(1)

    Post.last.tap do |created_post|
      expect(created_post).to have_attributes(title:, content:, author: current_user)

      expect(gql_errors).to eq(nil)

      resolved_object = gql_data.dig("post")

      expect(resolved_object).to match(
        "id" => created_post.id.to_s
      )
    end
  end
end

Subscriptions

Subscriptions are a bit more tricky: the problem is that they can return data two times: first time when client is subscribed (via def subscribe) and second time when subscription is triggered. The first case is simple: you just need to perform the query and check the response, like we did for types.

However, how can we test the payload that comes when subscription is triggered?

Let’s start with the spec file:

describe Api::Types::DailyGoal::EarnTicketEventType do
  include GraphqlSubscriptionHelpers

  let(:query) { <<~GQL }
    subscription {
      postCreated {
        post {
          id
        }
      }
    }
  GQL

  let(:post) { create :post }
  let(:current_user_id) { 42 }
  let(:context) { { channel: mock_channel, current_user_id: current_user_id } }

  specify do
    subscribe(query, context:)
    trigger_subscription(:post_created, payload: { post: }, scope: current_user_id)

    event = mock_channel.mock_broadcasted_messages.first.dig("data", "post")
    expect(event).to match("id" => post.id.to_s)
  end
end

We expect the event that was sent to the user to be inside the message queue of the mock_channel. In order to get it we need to subscribe to the subscription and emit the event.

All the magic sits in the GraphqlSubscriptionHelpers:

module GraphqlSubscriptionHelpers
  extend ActiveSupport::Concern

  included do
    let(:mock_channel) { MockSubscriptionCable.fetch_mock_channel }

    # we want queue to be empty when individual example is run
    before { MockSubscriptionCable.clear_mocks }
  end

  def subscribe(query, context:)
    MockSubscriptionCable::GraphqlSchema.execute(query, context:)
  end

  def trigger_subscription(subscription, payload:, scope:)
    MockSubscriptionCable::GraphqlSchema.subscriptions.trigger(:post_created, {}, , scope:)
  end
end

We have an accessor to get the mocked channel, and we have two methods—subscribe and trigger_subscription that imitate the real subscription flow.

Now we need to implement that MockSubscriptionCable module. Turns out, that graphql-ruby gem has the code we need right in the repo—they need to test subscription as well. Now we can build our own version inspired by their implementation of the mock subscription queue:

class MockSubscriptionCable
  class MockChannel
    attr_reader :mock_broadcasted_messages

    def initialize
      @mock_broadcasted_messages = []
    end

    def stream_from(stream_name, coder: nil, &block)
      block ||= ->(msg) { @mock_broadcasted_messages << msg[:result] }
      MockSubscriptionCable.mock_stream_for(stream_name).add_mock_channel(self, block)
    end
  end

  class MockStream
    def initialize
      @mock_channels = {}
    end

    def add_mock_channel(channel, handler)
      @mock_channels[channel] = handler
    end

    def mock_broadcast(message)
      @mock_channels.each_value { |handler| handler&.call(message) }
    end
  end

  class << self
    def clear_mocks
      @mock_streams = {}
    end

    def server
      self
    end

    def broadcast(stream_name, message)
      stream = @mock_streams[stream_name]
      stream&.mock_broadcast(message)
    end

    def mock_stream_for(stream_name)
      @mock_streams[stream_name] ||= MockStream.new
    end

    def fetch_mock_channel
      MockChannel.new
    end

    def mock_stream_names
      @mock_streams.keys
    end
  end

  # you can inherit from your app's schema or just build a new one
  class GraphqlSchema < ::GraphqlSchema
    use GraphQL::Subscriptions::ActionCableSubscriptions,
      action_cable: MockSubscriptionCable,
      action_cable_coder: JSON
  end
end

In this post we learned how to test various responses from your Ruby backend powered by GraphQL Ruby.

Building a GraphQL API in Rails and want it done right? I offer GraphQL design and architecture consulting.

In this post we will discuss two things that are often missing: contracts and composability. After that I will share a list of anti–patterns I found while working on dozens of Rails applications. We will examine more and less popular gems to see if they help us to avoid these bad practices. The final part is all about diagnosing your code for these problems and fixing them quickly.

Bear in mind that we will talk about pretty standard Rails monoliths with the relational database, key–value storage, background jobs, mailers etc. If you have something complex (e.g., message queues or search engines)—you will be probably able to extend these cases to handle these interactions as well. Also, sometimes service objects are also called interactors or (business) actions—all issues are applicable to anything in this list.

Contracts

Ruby does not have explicit type annotations out of the box. In simple cases it’s easy to guess: what class will be used if the instance called user?

However, what if you want to compose your services? In that case you will have to dig up some code or tests to understand types and make sure you handled all possible cases.

You might be using rbs or Sorbet to get some types. If you use them—let me know if missing contracts are still an issue in your project.

Ideally service objects should somehow specify their input and output types explicitly. Also, it would be really helpful to have the validation of the returned value to make sure that there are no hidden scenarios that return something different.

There is a number of ways to add validation for input parameters, for instance using Rails Attributes API:

class SignUp
  include ActiveModel::Model
  include ActiveModel::Attributes

  attribute :birthday, :date
  validates :birthday, presence: true
end

You can also take a look at dry-initializer.

Output validation is harder, and it’s more dangerous. Take a look at this code:

result = PlaceOrder.call(user:)

What is result? Can this method raise exceptions? We can open up the source code and check, but what if it calls other classes? We will have to reach the maximum depth to get the list of possible results and exceptions. And we will do it every time for every class, because this knowledge is not written anywhere.

As a homework exercise, try to estimate how much time you team could waste while figuring out these missing contracts.

Service composition

In functional programming composition is an act or mechanism to combine simple functions to build more complicated ones. In other words, you can compose two functions into a new one that runs the first function on the input and passes the value to the second one. You get the same thing as you would get if you call two functions manually.

In object–oriented programming composition means the ability to use one object from another. They say that there is a has–a assoctiation between them.

Do not confuse it with has_one from ActiveRecord.

You can call one service object from another, i.e., compose them like this:

class AddItemToCart
  def initialize(user:, item:)
    @user = user
    @item = item
  end

  def call
    user.with_lock do
      # this is composition of services:
      cart = FindOrCreateCart.call(user: @user)

      cart_item = cart.cart_items
        .create_with(quantity: 0)
        .find_or_initialize_by(item: @item)
      cart_item.quantity += 1
      cart_item.save!
    end
  end
end

Object–relational impedance mismatch

Object–relational impedance mismatch is a set of concepts that are similar between object–oriented languages and relational databases, but they work in completely different ways. For instance, objects and tuples both can hold data, but databases do not have encapsulation.

The most important thing for us is the difference in error handling. In the program you can write some logic, raise exceptions and handle them, but in the database there is an additional level—transactions. When you start transaction, you can make some changes in the database, and commit or rollback them altogether. This is called atomicity, which is represented by A in ACID).

Depending on the isolation level, database can behave differently when you make queries. For instance, default isolation level READ COMMITTED can return different data when you make the same query two times if something had changed in between, but REPEATABLE READ will make queries return the same data until the commit.

Non–atomic actions

Database is not the only thing we use when we write logic: we can also change other data stores (e.g., Redis or Elasticsearch), perform HTTP requests, work with file system and so on. These actions are not managed by transaction—after the rollback, these changes will stay unless we handle it in some way.

What’s wrong with composition

Take a look again at the example above. Do you think it works fine? Maybe, depends on the implementation of the FindOrCreateCart. For instance, this is how it could look like:

class FindOrCreateCart
  def initialize(user:)
    @user = user
  end

  def call
    user.with_lock do
      cart = user.orders.find_or_create_by(status: :cart)
    end

    Redis.instance.incr("carts", 1)
  end
end

Imagine that we use this service directly as well. We need the lock and transaction to make sure that user has only one cart, and we need to increment counter (just for the demonstration purposes).

This service object looks fine as well, this is how it works:

• transaction opens;
• cart item is looked up:
  • if it’s found—transaction commits—Redis counter is incremented;
  • if it’s not found—insert is attempted:
    • if insert succeeds then transaction commits—Redis counter is incremented;
    • otherwise—rollback error is thrown, transaction is rolled back and exception prevents counter from the incrementation.

However, when it’s called as a part of AddItemToCart, two bad things can happen.

First one appears when FindOrCreateCart goes to one of scenarios when counter is incremented, but FindOrCreateCart raises a Rollback error after: in this case cart creation will be rolled back, but counter increment will still happen.

The second one happens when something causes user.orders.find_or_create_by(status: :cart) to raise Rollback: in this case the nested with_lock block will catch the exception, but won’t rollback the transaction because it’s not the place where it was open. As a result, everything will be commited!

Read more about nested transactions issue here

How could we avoid these issues? Well, for the second one we could add requires_new: true to make sure a nested transaction will use the SAVEPOINT. The first issue is not trivial: nested service has no way to know that parent transaction was rolled back; parent service has no way to know if nested action needs a specific rollback.

This is the perfect example of services that cannot be composed. One way to fix this issue is to not reuse the code at all: for instance, you can keep everything in controller actions, which will make transaction usage clearer, but I believe that it will lead us to the unmaintainable mess.

Another way is to split service objects into groups: first is used only on top level (opens up transactions, sets locks etc.) and cannot be called from other services; second contains only database actions; third contains only non–atomic actions.

As a summary of this section we can conclude that services should either be fully composable or composition should be prohibited at all. In the next section we will discuss more service object anti–patterns, some of them cause actions to be non—composable.

Service implementation anti–patterns

Nested transactions

Nested transactions were already illustrated above: this problem happens when one service object opens up the transaction and calls another service that tries to open up transaction as well. This can lead to the situation when nested rollback is just thrown away.

Behavior is different depending on how service was called

This one was also illustrated above. The symptom of the problem is that action behaves differently depending on whether it was called directly or from another service object.

Incompatible transaction levels

This is kinda a sub–problem of nested transactions: if some service needs a higher isolation level that a default one and it’s called from another action that did not request this isolation level, it will either behave in a wrong way or database error will happen.

I might be wrong, but all modern databases I know does not allow it. However, even if it was possible, imagine the following situation:

class ParentService
  def call
    ApplicationRecord.transaction do
      ChildService.call
    end
  end
end

class ChildService
  def call
    ApplicationRecord.transaction(isolation: :repeatable_read) do
      # logic
    end
  end
end

ChildService works fine when called directly, but when it’s called inside READ COMMITTED—it will behave in a wrong way.

Non–atomic action inside transactions

Non–atomic action is literally anything that changes a state of anything except the database that runs the transaction. When something fails, transaction will be rolled back, but change made by non–atomic action will stay.

For example, take a look at the following service:

class RegisterUser
  def initialize(email:)
    @email = email
  end

  def call
    ApplicationRecord.transaction do
      user = User.create!(email: @email)
      UserMailer.welcome(user).deliver_now
      IssueDiscount.perform_later(user:)

      CreateCart.call(user:)
    end
  rescue ActiveRecord::RecordNotUnique
    # user already exists
  end
end

This looks kinda fine: if user already exists than we won’t sent mail and issue discount. However, what if CreateCart fails? In this case user record will not be committed, but email will be sent and job will be enqueued.

After that, later job will raise an error because it won’t be able to load (or deserialize) user from the database. Moreover, even if transaction will commit, there is a huge chance that background job processor will pick up the job before the commit, leading to the same error. Of course, it will restart and load the user, so job will succeed but there will be a reported error. Pretty nasty bug to investigate later, right?

It worth mentioning that this issue can be obscure when transaction is opened on the upper level. In this case you won’t notice the issue unless you examine the whole call stack. One possible sign is that you have atomic and non–atomic action in the same class: unless there’s an implicit transaction—there is something going wrong here.

IO actions inside transaction

This section is dedicated to another similar pattern, but causing different sympthoms is IO actions inside transactions. Imagine a following service:

class CheckPayment
  def initialize(order:)
    @order = order
  end

  def call
    @order.with_lock do # lock will force DB snapshot to be taken
      response = PaymentProviderClient.check_payment(order_id: order.external_id)

      order.update!(payment_status: :processed) if response[:status] == :processed
    end
  end
end

From the business logic perspective it looks fine: we fetch the status from the external API and update our database based on the response. However, what if that API is down?

Make sure to not make ANY IO actions in the main (i.e., web server) thread cause it can consume all workers and application will be down. This is the anti–pattern itself, not related to the place where you keep the business logic. Prefer background jobs for such things instead.

That will make our transaction longer. Moreover, locks will be held while application waits from the response, preventing other transactions from finishing.

Two transactions for single controller action

I know there might be exceptions, but most of the time when you have two transactions (not nested) inside the single action it means that something is wrong. What if the second one fails? Should we revert a first one? How?

For instance, imagine the situation, when you need to do 3 things:

1. change database state (transaction 1);
2. fetch some data from HTTP (we already know it should be outside the transaction);
3. change database state one more time depending on the response (transaction 2).

The best way to do that is to run background job after the first transaction and keep steps 2 and 3 there—it will either complete or we will get the exception to our error tracker, fix the problem and re–run the job.

I’m using a gem for service objects!

In this section we will discuss ways how to implement service objects using existing popular gems and approaches. As you might have noticed, all these anti–patterns can be more or less easily refactored, but can we prevent them by design? Does gem design help to achieve that?

Linters seem to be helpless here because usually they can check only a single file in isolation, there is no way to see if transaction was opened around. As mentioned earlier, we can mitigate these patterns if we do not have any composition, but that might be a bad solution in terms of maintenance.

interactor

One of most popular solutions is interactor.

Do not confuse it with Interactor pattern coming from the Clean architecture. By definition from the internet it means “little, reusable chunks of code that abstract logic from presenters while simplifying your app and making future changes effortless”. Not related to our topic at all.

A very first thing that’s mentioned in the README at the moment of writing is context. Effectively it’s just a hash that contains all passed arguments, and you can add more if you want. When service runs successfully you get the whole context back, otherwise you get Interactor::Failure. You can also use context.fail! to halt and cause service to return a failure object.

class AuthenticateUser
  include Interactor

  def call
    if user = User.authenticate(context.email, context.password)
      # adding two more variables to the context
      context.user = user
      context.token = user.secret_token
    else
      context.fail!(message: "authenticate_user.failure")
    end
  end
end

# args will go to context
AuthenticateUser.call(email:, password:)

The context itself feels a bit dangerous, because it can be used as a replacement of instance variables, which sounds like a breach of encapsulation. Are we sure someone even wants to read these new variables?

Frankly speaking, it took me a second to understand that we need to pass email and password to make this call. However, just to make things even more hard, let’s take a look at organizers:

class PlaceOrder
  include Interactor::Organizer

  organize CreateOrder, ChargeCard, SendThankYou
end

What does it accept and return? You will have to go and read specs or code of all three services. Understanding the contract becomes a really hard job.

Let’s see what we got for transactions and non–atomic actions. There is an around hook you can use:

class PlaceOrder
  include Interactor::Organizer

  organize CreateOrder, ChargeCard, SendThankYou

  around do |interactor|
    ApplicationRecord.transaction { interactor.call }
  end
end

I bet SendThankYou contains some non–atomic logic. Can we pull it away from the transaction?

class PlaceOrder
  include Interactor::Organizer

  organize CreateOrder, ChargeCard

  around do |interactor|
    ApplicationRecord.transaction { interactor.call }

    SendThankYou.call(interactor.context)
  end
end

That kinda works, but what if we need to do something non–atomic inside the service in the middle of the chain? That would be really hard because you will have to not forget to pull it away as well. Moreover, in this case CreateOrder and ChargeCard cannot be used directly without wrapping them into the transaction.

It worth mentioning, that there is a rollback method that can help us to do some cleanup at least.

As a result, composition does not play well here too: we have to prohibit a direct usage of services that not open the transaction for themselves. Also, we should always remember to not add anything non–transactional to these services and have separate services for that.

active_interaction

The next stop is active_interaction. Let’s rewrite our previous example:

class AuthenticateUser < ActiveInteraction::Base
  string :email
  string :password

  def execute
    if user = User.authenticate(email, password)
      { user:, token: user.secret_token }
    else
      fail AuthenticationFailed
    end
  end
end

This looks a bit better in terms of contract: we can see that there are two strings expected. The returned value ({ user:, token: user.secret_token }) will be placed to the result object:

outcome = AuthenticateUser.run(email:, password:)
outcome.valid? # => true
outcome.result # => { user:, token: user.secret_token }

We still need to read the code to understand, what is returned. Note that there is no validation to make sure that result is always the same. For instance, in some cases we might return only user, and the code that uses this service should be aware of it.

What about the composition? According to the README, we can call another service using compose and pass arguments explicitly:

class PlaceOrder < ActiveInteraction::Base
  object :user

  def execute
    ApplicationRecord.transaction do
      order = compose(CreateOrder, user:)
      compose(ChargeCard, user:, order:)
    end

    compose(SendThankYou, user:)
    order
  end
end

It’s better than implicit context and organizer we saw in interactor! However, all other problems are still here: we have to split services into ones that can be directly used and manage transactions by ourselves.

mutations

Let’s take a quick look at mutations:

class AuthenticateUser < Mutations::Command
  required do
    string :email, matches: EMAIL_REGEX
    string :password
  end

  def execute
    if user = User.authenticate(email, password)
      { user:, token: user.secret_token }
    else
      :auth_failed
    end
  end
end

There is an input validation. The same as active_interaction there is an implicit result, but the difference is that there are no other validations except data types, README suggests to do everything manually in the execute method.

Composition is done using plain old method calls:

class PlaceOrder < Mutations::Command
  required do
    object :user
  end

  def execute
    ApplicationRecord.transaction do
      order = CreateOrder.execute(user:)
      ChargeCard.execute(user:, order:)
    end

    SendThankYou.execute(user:)

    order
  end
end

This approach has same issues as previous solutions we inspected.

LightService

Another popular gem I found is LightService:

class Transactional
  def self.call(context)
    ApplicationRecord.transaction { yield }
  end
end

class CalculatesTax
  extend LightService::Organizer

  def self.call(order)
    with(order:).with(Transactional).reduce(
        LooksUpTaxPercentageAction,
        CalculatesOrderTaxAction,
        ProvidesFreeShippingAction
      )
  end
end

class LooksUpTaxPercentageAction
  extend LightService::Action
  expects :order
  promises :tax_percentage

  executed do |context|
    tax_ranges = TaxRange.for_region(context.order.region)
    context.tax_percentage = 0

    if object_is_nil?(
      tax_ranges,
      context,
      'The tax ranges were not found'
    )
      next context
    end

    context.tax_percentage = tax_ranges.for_total(context.order.total)

    if object_is_nil?(
      context.tax_percentage,
      context,
      'The tax percentage was not found'
    )
      next context
    end
  end

  def self.object_is_nil?(object, context, message)
    if object.nil?
      context.fail!(message)
      return true
    end

    false
  end
end

What we can notice here:

• interactor–like organizers;
• interactor–like around hooks for transactions (or run them manually);
• promises to specify the output.

Contract is defined better for separate services, but it’s still a challenge to read it for the organizer. Composition has all the issues we saw before.

Granite

You might be curious how gem with <200 stars appeared here. Take a look:

class PlaceOrder < Granite::Action
  subject :user

  private def execute_perform!(*)
    # no need to open transaction — gem will do that for us when needed and use same
    # transaction for other services
    order = create_order_service.perform!
    charge_card_service.perform! # not sure how to pass order here, see below
  end

  after_commit do
    UserMailer.thank_you(user: subject).perform_later
  end

  memoize def create_order_service
    CreateOrder.new(subject)
  end

  memoize def charge_card_service
    ChargeCard.new(subject)
  end
end

The thing that’s done in a different way is that Granite supports service composition out of the box. As you see, we instantiate other services and call them, the gem handles transactions.

Here comes a problem: gem requires all services to be defined as memoized instances, making it impossible to pass anything that was created by another service. I guess it’s possible to implement anything one might need using some code jiggling.

There are many more things in the gem: various hooks, validations, data representers, policies, context, associations, exceptions, I18n and many more. What’s missing is the documentation—it’s fairly short so you will have to dig into gem source to learn it better. I like this gem way more than others because it has a lot of cool ideas, but writing services this way feels hard: you need to keep in mind too many rules.

dry-*

Often times people mention dry stack when they are asked about the way they implement service objects. This is a set of gems that can do different things, so the way you cook them really matters. There are a couple of thins to remember if you decide to go that way.

dry-validation can be used for the contract of input data, output is not covered by anything if I’m not mistaken.

dry-transaction is about business transactions and does not help with database transactions at all, so you have to manage that manually as discussed in previous solutions.

dry-monads are often used as result objects, but there are some common pitfalls. One of the most popular features is do notation that implements a popular railway pattern:

class CreateAccount
  include Dry::Monads[:result]
  include Dry::Monads::Do.for(:call)

  def call(params)
    values = yield validate(params)
    account = yield create_account(values[:account])
    owner = yield create_owner(account, values[:owner])

    Success([account, owner])
  end

  def validate(params)
    # returns Success(values) or Failure(:invalid_data)
  end

  def create_account(account_values)
    # returns Success(account) or Failure(:account_not_created)
  end

  def create_owner(account, owner_values)
    # returns Success(owner) or Failure(:owner_not_created)
  end
end

This looks good, but be aware that when yield Failure happens the database transaction will be rolled back. This brings us to the problem of “all or nothing”. Imagine that you want to create an order for user from his cart, but also cleanup the cart from items that are out of stock. As a result we have 3 possible scenarios:

• Success when order is created;
• Failure when cart is empty;
• something when cart became empty.

I’d really like to use Failure(:empty_cart) but I cannot, because it will roll back my transaction! As a result we have to do a following thing:

def call
  ApplicationRecord.transaction do
    yield clear_cart
    yield create_order
  end
end

def clear_cart
  cart.cart_items.filter(&:out_of_stock?).delete_all
end

def create_order
  if cart.cart_items.any?
    order = user.orders.create!(cart)
    Success(:order_created, order:)
  else
    Success(:empty_cart)
  end
end

Alternatively, we can stop using do notation or yield, but that’s what we came for.

One last thing to mention that Failure rolls back transaction only when used with yield, this won’t work:

def call
  ApplicationRecord.transaction do
    yield clear_cart
    create_order
  end
end

Anyway, are my services fine or not?

After reading this bunch of text you might be asking yourself if your services are fine or they need some changes. I prepared a checklist for you.

First of all, if you are not reusing the code at all or have separate classes for actions that happen before and after transactions and never mix them—probably you can stop thinking about composition issues. Otherwise—your services might be affected by some of the related anti–patterns.

The fastest way to check this is to install isolator. It will report cases when someone performs non–atomic action, schedules job or makes HTTP request from inside the transaction. It might find some issues caused by composition. As soon as you get some offences, you can quickly fix some of them by after_commit_everywhere:

class SomeService
  include AfterCommitEverywhere

  def call
    ApplicationRecord.transaction do
      # db operations

      after_commit do
        # jobs, mails and etc go here
      end

      # you can continue db operations if you want
    end
  end
end

This is not ideal and you might still need to redesign your service object, but at least it will help to fix some nasty bugs right away by separating flows.

HTTP queries can be moved away from transaction by putting it before the transaction block:

class CheckPayment
  def initialize(order:)
    @order = order
  end

  def call
    response = PaymentProviderClient.check_payment(order_id: order.external_id)

    ApplicationRecord.transaction do
      order.update!(payment_status: :processed) if response[:status] == :processed
    end
  end
end

However, it’s not always easy, because this service might be called from another one which opened the transaction already. As you might have guessed, the hardest part is nested transactions. I did not found an existing solution, so I prepared a quick and dirty script for you:

ApplicationRecord.singleton_class.prepend(Module.new do
  def transaction(**kwargs)
    Thread.current[:transaction_stack] ||= []

    unless kwargs[:requires_new]
      location = caller_locations(1, 1).first

      if Thread.current[:transaction_stack].any?
        puts <<~TEXT
          Found nested transaction in #{location}.

          Opened up in:
          #{Thread.current[:transaction_stack].join("\n")}
        TEXT
      end
    end

    if Thread.current[:transaction_stack].empty? || !kwargs[:requires_new]
      Thread.current[:transaction_stack] << location
    end

    super(**kwargs).tap do
      Thread.current[:transaction_stack].pop
    end
  rescue e
    Thread.current[:transaction_stack] = []
    raise e
  end
end)

Put it to the initializer and run your specs—maybe there will be something.

As mentioned, input validation can be handled by Rails Attributes API, dry-initializer or something similar.

You might be wondering how to to completely avoid nested transactions, write fully reusable and composable service objects without any way to make it wrong. I’m curious too, stay tuned and I’ll share something in the next post!

If you want to take a look at the thing I’m experimenting with—here it is.

At the meantime, take a look at your services, see if you can spot some anti–patterns using your eyes and tools. Check if that causes some bugs or performance issues in your system. Think if you are using composition, try to remember bugs caused by unhandled scenarios after the service call.

Maybe you will be able to solve it by redesigning the class layout. Probably making some new team agreements will help. It might turn out that your services are fine and you don’t face any issues at all.

If your service layer looks like any of the patterns above, a focused audit can help you untangle it. Let’s talk.

Refactoring as a process

Refactoring is a technique of transforming the code without making changes in its functionality. Why people do that?

Sometimes the way that code written makes it harder for to engineers to improve the performance or add a new feature. Another example is a scenario when application is big and there are many ways to do the same thing or same responsibility is implemented in different layers of application.

This is how refactoring looks like in books:

1. the problem is spotted and a better solution is found;
2. add some tests if coverage is not enough;
3. code is rewritten.

Please, please, do not start refactoring before covering the code with specs!

Eventually it will be done, but a number of issues that might cause issues:

1. there are too many places to fix, so refactoring will take a lot of time to complete;
2. most of the time there are more than one problem that’s being refactored at the given moment;
3. people might accidentally write code in the “old style”, so amount of work will keep growing.

Clearly, we need a tool to somehow track all occurrences of the given issue with the code, build a list of them to track what should be done, and another tool to prevent writing the code in the old style.

Unexpected tool to solve our problems

Fortunately, Ruby ecosystem already has such tool. Rubocop is a linter that has a lot of plugins to enforce various best practices for your codebase. When particular rule is offended—Rubocop can highlight it in the code editor. Most of Ruby applications have it executed on the CI as well.

If you cannot fix the problem right now, but want to do it later—you can add the exception to the .rubocop_todo.yml:

Style/BracesAroundHashParameters:
  Exclude:
    - 'app/models/user.rb'

But how can it help us to refactor our application to our best practices? Easy, you just need to write your own cops! That might sound complex, but you can get used to it right after you write one or two of them.

Writing custom cops for Rubocop

For instance, imagine that you want to restrict opening DB transactions anywhere except app/business_actions/. Here’s a cop:

module Isolation
  class IllegalTransactionBlock < Cop
    MSG = "Transaction shoud only be opened from business actions".freeze

    def_node_matcher :transaction?, "(send ... :transaction ...)"

    def on_send(node)
      add_offense(node, message: MSG) if transaction?(node)
    end
  end
end

Now you can configure Rubocop using .rubocop.yml to run it for a specific folder:

Isolation/IllegalTransactionBlock:
  Exclude:
    - app/business_actions/**/*.rb

Looks like we reached our goals:

1. now we can regenerate a TODO list for Rubocop to find all problems to fix;
2. with new TODO we can see where we are in terms of overall refactoring process;
3. no one can add new transaction in the illegal place without breaking the CI.

Read more about writing cops in my post, old but gold!

Dealing with larger codebases

If you ever worked on a huge application, you can imagine what your .rubocop_todo.yml will look like, especially after installing new plugin, upgrading old ones and writing a number of custom cops. You’re looking at this N–thousand–lines file and asking yourself where to start. You clearly understand that you cannot just fix everything right away, it will take weeks of human–hours. Where to start?

I’d suggest to take a look at two things.

Firstly, some offences are a bit better than others. For instance, if you have nested transactions (why is it bad? read this)—you might want to fix that before, say, replacing => with : in hash literals. Also, some issues might block others, so they should go first. Looks like we need to add some weights to our cops.

Secondly, files are updated with a different frequency. I’d prefer “hot” files to satisfy all team best practices, cause people will learn them faster just by looking at this code. Also, files that were not updated for years likely gonna have no issues or can eventually be sunset, so we can postpone the refactoring. Sounds like we need to consider a frequency of updates as well. Where do we have it? In the git history!

Now we need to combine these two things—meet rubocop_director!

Setting up rubocop-director 🎬

To get the ball rolling, install the gem and generate the default config:

$ bundle add rubocop_director
$ bundle exec rubocop-director --generate-config

You will get a file with weights of all cops that have offences, default cop weight and file frequency update weight:

update_weight: 1
default_cop_weight: 1
weights:
  Isolation/IllegalTransactionBlock: 1
  Layout/HashAlignment: 1

Now you can grab some pop corn and let rubocop_director to make a refactoring plan for you:

bundle exec rubocop-director

When it’s done (it might take a while)—you’ll see the report:

💡 Checking git history since 1995-01-01 to find hot files...
💡🎥 Running rubocop to get the list of offences to fix...
💡🎥🎬 Calculating a list of files to refactor...

Path: app/controllers/user_controller.rb
Updated 99 times since 1995-01-01
Offenses:
  🚓 Isolation/IllegalTransactionBlock - 2
Refactoring value: 1.5431217598108933 (54.79575%)

Path: app/models/user.rb
Updated 136 times since 1995-01-01
Offenses:
  🚓 Isolation/IllegalTransactionBlock - 1
  🚓 Layout/HashAlignment - 1
Refactoring value: 1.2730122208719792 (45.20425%)

Files are sorted by the refactoring value, which is calculated using a formula: sum of value from each cop (<count of offences> * <cop weight> * (<count of file updates> / <total count of updates>) ** <update weight>). As you see, file update frequency is here too!

What can you tune here?

1. change cop weights to increase the value of ones you want to go first;
2. change the update frequency weight (you might want to increase or reduce it);
3. change the date, since when updates are counted (e.g., --since 2023-01-01).

When you’re done—rebuild the report and you’ll get a new prioritized list. Now you can pick up some files from the top and start working on them!

In this post we discussed a method of an observable refactoring process of Ruby applications. Here are key takeaways:

1. start with detecting issues in the code and finding solutions for them;
2. if you can describe an issue in terms of Rubocop—do that and get all the benefits (CI, linters etc.);
3. otherwise—describe it in docs and track the progress manually;
4. if you cannot fix issues all at once—build a list of files to fix, prioritize them and work on them from the top;
5. if you were able to do everything using Rubocop—you can grab rubocop_director for this job.

Large-scale refactoring is easier with a plan. If you’re facing a similar situation, I can help you map it out.

Errors as a part of the GraphQL response

According to the GraphQL specification, errors are the part of the response (along with the data and extensions), for instance:

{
  "data": {
    "viewer": nil
  },
  "errors": ["access denied"],
  "extensions": {
    "complexity": 12
  }
}

The specification does not define a type for errors, like it does for the data. The array element inside the errors key should be a valid JSON structure (e.g., strings or objects).

Expected versus unexpected errors

Instead of putting everything under the errors key, let’s try to divide errors into groups:

1. errors that are completely unexpected and cannot be fixed by user in any way (e.g., database is down);
2. errors that affect a part of the user workflow (e.g., user is not authorized to access a certain part of the graph, and the error will look the same for any field);
3. errors in the submitted data (i.e., validation errors).

We can spot a couple of interesting patterns here.

First and second type of errors can happen both in queries and mutations. They are not depending on the submitted data at all. Morevover, user cannot fix this error by submitting different arguments with the same query string.

Third group can only happen in mutations, because this kind of errors can represent either invalid state of the data in the database (e.g., processed order cannot be cancelled) or invalid mutation arguments (e.g., percent cannot be greater than 100).

Taming top–level errors

GraphQL errors are not typed, so we have to add some types on our own. Let’s make errors objects (not just strings) with code and details fields:

{
  "data": {
    "processOrder": nil
  },
  "errors": [
    {
      "code": "permission_denied",
      "details": { "message": "you cannot perform this action" }
    },
    {
      "code": "server_error",
      "details": { "message": "DB is down" }
    }
  ]
}

Codes should go to the special list and existing codes should never change. It will help client applications to handle such errors in a common manner. Details might contain anything you want, but it might be helpful to have the message field everywhere.

Permission errors

If application has a permission/role system and it is possible for user to try to do some action without success guarantee — we might want to have this as a specific error. Permission errors can happen both in mutations and queries, so it makes sense to return them inside the top–level errors field.

For instance, imagine that there is a mutation for order processing:

mutation ProcessOrder($id: ID!) {
  processOrder(id: $id) {
    status
    user {
      email
    }
  }
}

Where permission error can happen? It can be on the mutation level:

{
  "data": {
    "processOrder": nil
  },
  "errors": [
    {
      "code": "permission_error",
      "details": {
        "field": ["processOrder"],
        "message": "you cannot process this order"
      }
    }
  ]
}

…or processing might work fine, but user cannot be accessed:

{
  "data": {
    "processOrder": {
      "status": "processed",
      "user": nil
    }
  },
  "errors": [
    {
      "code": "permission_error",
      "details": {
        "field": ["processOrder", "user"],
        "message": "you cannot access this data"
      }
    }
  ]
}

One more thing: one more approach to permissions is to have a special place to ask server if the particular action is possible. It could be a special type in the type or root. If it says you can do it — do it, otherwise you’ll get an error in the errors key.

Application errors

When something goes really bad on the server side (e.g., database is not responding or query format is wrong) we should put it to the errors. For instance:

{
  "data": {
    "processOrder": nil
  },
  "errors": [
    {
      "code": "client_error",
      "details": { "message": "request format is wrong" }
    }
  ]
}

Note that usually such errors should not be shown to the client as is. Instead, we should tell them that something wrong is happened and optionally ask to contact the support (optionally, because we should already know about this issue from our error tracker).

Handing validation errors

Validation errors follow a similar pattern as mutation arguments, so it makes sense to keep them as a part of the response, rather than put them to the top–level errors array. In this case we keep the whole power of the GraphQL type system. In this case each mutation will have a union as the return type: Success will be returned when everything is fine, while Failure will represent validation errors.

For instance, imagine the mutation that registers a new user:

mutation SignUp($email: String!, $password: String!) {
  signUp(email: $email, password: $password) {
    __typename

    ...on SignUpSuccess {
      user {
        id
      }
    }

    ...on SignUpFailure {
      email
      password
    }
  }
}

The successful response will look like this:

{
  "data": {
    "signUp": {
      "__typename": "SignUpSuccess",
      "id": 123
    }
  },
  "errors": []
}

In case of failure we can access errors for each fields, which can be just lists of strings to show to user:

{
  "data": {
    "signUp": {
      "__typename": "SignUpFailure",
      "email": [],
      "password": ["should be longer than 8"]
    }
  },
  "errors": []
}

Sometimes mutation fails not because of the input, but because of the database state, which is related to this particular mutation. For instance, imagine the mutation ProcessOrder that checks for order status and prohibits processing orders which are already processed. In this case return type could be ProcessOrderSuccess | OrderCannotBeProcessed | ProcessOrderFailure:

mutation ProcessOrder($id: ID!) {
  processOrder(id: $id) {
    ...on ProcessOrderSuccess {
      order {
        ...OrderFields
      }
    }

    ...on OrderCannotBeProcessed {
      order {
        ...OrderFields
      }
      # no reason cause it's obvious
    }

    ...on ProcessOrderFailure {
      order {
        ...OrderFields
      }
      # we can also grab a reason of failure
      reason
    }
  }
}

The alternative to this approach could be a shared Success type which always points to the root of the graph, allowing clients to fetch whatever they want:

mutation ProcessOrder($id: ID!) {
  processOrder(id: $id) {
    ...on ProcessOrderSuccess {
      # we're in the root so we can fetch whatever we need
      orderById($id: ID!) {
        ...OrderFields
      }
      viewer {
        email
      }
    }
  }
}

This is more flexible but also more verbose.

A word about monitoring

One of the common techniques in monitoring is USE:

• Utlization represents the ratio of resource used (e.g., how many background workers are in use);
• Saturation stands for the lack of the resource (e.g., how many background jobs are waiting while all workers are busy);
• Errors represent the amount of errors in the monitored component.

As you might guess, we care about the last one in this post. In REST it was pretty simple to find errors: you can look at response HTTP codes and count 5xx. It’s note so easy with GraphQL, where most of responses are 200 OK, so we should not forget to configure our monitoring tool to count responses with errors filled. Moreover, we should differ real errors from, say, permission errors. This is the moment when error codes come really handy.

Let’s sum up what we discussed in the post:

• generic errors that user should not see should go to errors;
• permissions errors are likely gonna go to errors too;
• when if we put something to errors — it’s better to have a structure (e.g., code and details)
• validations and mutation specific errors should go to the mutation payload.

Building a GraphQL API in Rails and want it done right from the start? I offer GraphQL design consulting.

Overall

During the year I published 12 posts: 8 about Rails, 2 about Haskell, 1 about databaseas and 1 essay about architecture. Overall there were 26 300 unique visitors. Thanks for the attention 🙂

Top articles

🥇 Why Ruby has symbols

A post about Ruby internals, rolled out in April, which got most attention: about 11 000 unique visitors spending 5 minutes 30 seconds on the page! Moreover, it was translated to Japaneese, recognized as one of most popular posts in RubyWeekly and reached #4 on Hackernews.

🥈 Understading why attr_accessor in Ruby is faster than a regular method

Another post on how Ruby works, which was rolled out in June. This one got 3 500 unique visitors with 4 minutes 45 seconds on the page.

🥉 How to make Ruby interpreter run program written in a natural language

A weird post about metaprogramming, that was visited by 2 300 unique users, and it took them 3 minutes 22 seconds in average to dig into it.

Those who didn’t make it to the top

It is quite easy to find a fix memory bloat when it happens (you have a monitoring tool set up, right?), but can we prevent it? In this article I will present a new approach for that, using a new gem io_monitor as a reference implementation.

Understanding how bloat happens

Imagine a pretty regular Rails app that sometimes goes to the database for data. When we load some data from the database we put it to the memory, and, when we do not have enough memory, Ruby VM goes to the operating system and asks for more. All Ruby objects in memory are organized into pages, and Ruby VM can return memory back to the operating system only from last free pages. As a result, if your app loaded a lot of data, the pages it was stored at can be returned to the system only if there is nothing that’s still in use after them.

I recommend this post with lots of pictures to get a better undestanding of this process.

Why bloat is a bad thing?

Your app will need more resources to run; moreover, if you live in the Kubernetes cluster you’ll be able to run less pods on the same machine and can see some restarts when quota is reached.

The memory bloat is easy to find when it happens: most of APMs will show you the place where application performed a lot of allocations, so you just need to go it fix this. Can we do it before it starts being a problem?

Early bloat detection

In most of cases (well, I do not have statistics except my own experience) memory bloat is caused by I/O operations. You fetch data from the database or Redis, read file from the disk, perform the network request, and unexpectedly big amount of data is about to be allocated in the application memory.

Of course there is a chance that your app itself creates a lot of objects from the code, but it’s a pretty rare thing to happen

The fix is usually trivial: you need to either load data in batches (so memory consumption will be the same as the batch size) or do not load it at all and perform calculations somewhere else. For instance, if you need a sum of transactions — do Transaction.sum(:amount) (which is SELECT SUM(amount) FROM transactions) instead of Transaction.all.sum(&:amount) (which is SELECT * FROM transactions and send it over the wire).

Here is an idea: what if we try to measure the size of data that was loaded from the I/O and compare it to the response size? Even in case if you have 100 transactions to get the sum and your APM keeps silence, this ratio will still be quite big!

For instance, for ActiveRecord you can do something like this:

ActiveRecord::ConnectionAdapters::AbstractAdapter.prepend(Module.new do
  def build_result(*args, **kwargs, &block)
    io_bytesize = kwargs[:rows].sum(0) do |row|
      row.sum(0) do |val|
        ((String === val) ? val : val.to_s).bytesize
      end
    end

    Aggregator.instance.increment(io_bytesize)

    super
  end
end)

Now we need to subscribe to notifications from controllers for action processing:

ActiveSupport::Notifications.subscribe("process_action.action_controller") do |*args|
  io_bytesize = Aggregator.instance.io_bytesize
  body_bytesize = args.last[:response].body.bytesize

  ratio = body_bytesize.to_f / io_bytesize

  Rails.logger.info "Loaded from I/O #{io_bytesize}, response bytesize #{body_bytesize}, I/O to response ratio #{ratio}"
end

As a result, we can see messages about actions that have a high ratio in our logs!

You can find the whole example here.

Let’s add a very simple controller with two actions — fast and slow:

class App < Rails::Application
  routes.append do
    get '/slow', to: 'app#slow'
    get '/fast', to: 'app#fast'
  end
end

class AppController < ActionController::Base
  def slow
    render json: {sum: Transaction.all.sum(&:amount)}
  end

  def fast
    render json: {sum: Transaction.sum(:amount)}
  end
end

If you run it — you’ll notice the difference in logs:

Started GET "/slow" for 127.0.0.1 at 2023-02-04 23:01:27 +0300
Processing by AppController#slow as */*
  Transaction Load (4.5ms)  SELECT "transactions".* FROM "transactions"
Completed 200 OK in 43ms (Views: 0.1ms | ActiveRecord: 11.7ms | Allocations: 95836)
Loaded from I/O 69899, response bytesize 15, I/O to response ratio 4659.933333333333

Started GET "/fast" for 127.0.0.1 at 2023-02-04 23:01:35 +0300
Processing by AppController#fast as */*
  Transaction Sum (3.1ms)  SELECT SUM("transactions"."amount") FROM "transactions"
Completed 200 OK in 4ms (Views: 0.1ms | ActiveRecord: 3.1ms | Allocations: 336)
Loaded from I/O 7, response bytesize 15, I/O to response ratio 0.4666666666666667

io_monitor

If you want to try this approach in your application, you don’t need to build this from scratch. Use io_monitor! Right now it supports Redis and HTTP along with ActiveRecord, and can publish to logs, Rails notifications and Prometheus.

Setup is fairly simple. After installation you need to configure what to monitor and where to publish and you’re good to go:

IoMonitor.configure do |config|
  config.publish = [:logs, :notifications, :prometheus] # defaults to :logs
  config.warn_threshold = 0.8 # defaults to 0
  config.adapters = [:active_record, :net_http, :redis] # defaults to [:active_record]
end

After that you need to include it controllers you want to check (or all of them):

class MyController < ApplicationController
  include IoMonitor::Controller
end

If something is not right you’ll see something in logs:

ActiveRecord I/O to response payload ratio is 0.1, while threshold is 0.8

I never tested it in the real app yet, so please let me know if you notice something weird!

In this post we discussed memory bloats: now you can not only notice and fix them, but also try to prevent. The main outcome is a bit different though: if you work with I/O — always consider the amount of data you might get and try to minimize this amount.

Memory issues in production are tricky to reproduce. If you’re dealing with a Rails app that’s eating too much memory, I offer performance audits.

Run the irb and define the method like usual:

def foo = 42

If you try to call it—you’ll obviously get 42. What does it mean? It goes without saying that the method works and we can call it, but the interesting part is not about it. We know that Ruby is purely object–oriented language: everything is class or instance of the class.

All examples in this post can be run in IRB or copied to file and run using the ruby interpreter, the result would be the same

Let’s create a new class and try to call #foo from it:

class Example
  def run_foo
    foo
  end
end

Example.new.run_foo # => 42

It works! Our method can be called even from the method in a different class. It behaves like a global method.

Wait, doesn’t it sound suspicious? We defined the method globally and we can call it but there is no global scope: it should be defined in some class. So where is it defined?

Ruby object model

This section is a brief recap of the Metaprogramming Ruby 2 by Paolo Perrotta and Crafting Interpreters by Robert Nystrom

Every class in Ruby is the instance of the class called Class. In the Ruby VM it’s represented as a structure called RClass, which contains:

• a list of methods;
• a list of instance variable names;
• a list of class–level instance variables;
• a pointer to the class this class is instance of (as we know, it’s Class);
• a pointer to the superclass, which is a parent class for the current one.

According to the encapsulation principle of Object–Oriented Programming, data should be bound to the methods that operate on that data into a single unit. However, implementing it in the VM would require copying the byte code of methods to the each instance of the class. Since methods are shared, it makes sense to keep them in the single place, and Ruby keeps them in the instance of the Class (e.g., the byte code of #run_foo is hold in the Example).

Because of the metaprogramming features of Ruby we can access and modify contents of the class in the runtime. With this knowledge, let’s try to understand, where the code from the “top–level scope” is executed.

Examining the top–level scope

Let’s try to understand where we are when we write the code on the top–level scope outside of any class:

puts self #=> main

It did not help a lot: we learned only that we are inside the object that overrides the #to_s method. Let’s check our class:

puts self.class # => Object

Good, we’re inside some Object, which is a default base class for all classes. If we are inside the Object instance and there is a #foo method, we are supposed to be able to create a new instance and call the method:

Object.new.foo #=> private method `foo' called for #<Object:0x00000001402cec50>

Exception: we cannot call the method because it’s private! Let’s make sure that it’s correct:

private_methods.include? :foo #=> true

Does Object have a base class?

Object.superclass # => BasicObject

According to the docs, BasicObject is a base class for new object hierarchies, and it’s completely empty. Therefore, “top–level” methods won’t be available in your class if you inherit right from it:

BasicObject.new.foo # => undefined method `foo' for #<BasicObject:0x00000001058d5648> (NoMethodError)

Let’s summarize our explorations:

• when some method is defined on the top–level scope, it’s defined in the Object class;
• this method can be called from any other class, except ones inheriting from BasicObject;
• the defined method is private, so it cannot be called directly on the instance of the Object.

Looks like that’s the answer! There is no global scope, we can access the method on the top level (because it’s defined here) or in other classes (because they inherit from Object), but this method is not visible from the outside because it’s private.

What about variables?

Let’s repeat the experiment, but instead of defining foo as a method we are going to make it variable:

foo = 42

class Example
  def get_foo
    foo
  end
end

Example.new.get_foo # => undefined local variable or method `foo' for #<Example:0x0000000126ac0d38>

Exception again! The problem is that foo was not available inside the method, because it’s not local for the instance of the Example class.

Like before, let’s check how foo was defined.

instance_variables # => []

No instance variables! Maybe it’s local?

local_variables #=> [:foo, :_, :version, :str]

Correct! Our variable foo is defined locally for the top–level scope (or, more preciselu, for the instance of the Object we are executing our code) and that’s why it’s not available inside the instance of Example, which inherits from the Object.

How does it work?

When we run Ruby script using ruby or REPL using irb, Ruby creates a new workspace for us. By default it will use TOPLEVEL_BINDING for everything, but it’s possible to pass a different binding to the workspace constructor. Here is a source code that runs a new IRB session.

In the previous post we implemented safe currying on applicative functors. We used a container called Either to create these interfaces. In this post we will go even further and create applicative lists and parsers.

Applicative array

Let’s take a look how functors can be implemented for lists:

class ApplicativeArray < Array
  include Functor
  def fmap(&fn) = map(&fn.curry)

  include Applicative
  class << self
    def pure(x) = ApplicativeArray.new([x])
  end

  def ^(other)
    ApplicativeArray.new(flat_map { |fn| other.fmap(&fn) })
  end
end

Functor#fmap is almost a regular map, but it uses curry call functions inside the list partially. pure takes the value and wraps it into the array. Application (^) takes function from another array and applies it to each element of the current array. Let’s try it:

def plus(x, y) = x + y
def mult(x, y) = x * y

array_with_functions = ApplicativeArray.new([method(:plus), method(:mult)])
array_with_args = ApplicativeArray.new([2, 7])
array_with_args_2 = ApplicativeArray.new([3, 5])

array_with_functions ^ array_with_args ^ array_with_args_2
# => [5, 7, 10, 12, 6, 10, 21, 35]

Why 8 elements?

1. plus is applied to two elements;
2. mult is applied to same elements, getting 4 partially applied functions;
3. these functions are applied to each of two elements getting 8 elements in total.

Can we apply functions to the corresponding element only? Sure, in Haskell this type of the list is called ZipList:

class Applicative::ZipList
  attr_reader :list

  def initialize(list) = @list = list

  include Applicative::Functor
  def fmap(&fn) = @list.map(&fn.curry)

  include Applicative::ApplicativeFunctor

  class << self
    def pure(x) = Applicative::ZipList.new(repeatedly { x })

    private

    def repeatedly(&block)
      Enumerator.new do |y|
        loop { y << block.call }
      end.lazy
    end
  end

  def ^(other)
    eager_list =
      if @list.is_a?(Enumerator::Lazy) && other.list.is_a?(Enumerator::Lazy)
        @list
      else
        @list.take(other.list.length)
      end

    Applicative::ZipList.new(eager_list.zip(other.list).map { |fn, x| fn.curry.(x) })
  end
end

Implementation is slightly more complex, because we have to handle endless lists: we need to do that because pure returns it. Here is an example:

plus = lambda { |x, y| x + y }
mult = lambda { |x, y| x * y }

functions = ZipList.new([plus, mult])
args = ZipList.new([2, 7])
args_2 = ZipList.new([3, 5])

(functions ^ args ^ args_2).list # => [5, 35]

Let’s try the same thing but using pure, that constructs an endless list for us:

functions = ZipList::pure(plus) # endless
args = ZipList::pure(2) # endless
args_2 = ZipList.new([4, 6, 8]) # finite

(functions ^ args ^ args_2).list.eager.to_a # => [6, 8, 10]

In this case we get only 3 elements (2 + 4, 2 + 6 and 2 + 8).

Applicative parser

Let’s take a look at more complex application of applicatives: an applicative parser. We start with the new data structure called Pair, which can hold two values:

class Pair
  attr_reader :fst, :snd

  def initialize(fst, snd)
    @fst = fst
    @snd = snd
  end

  def deconstruct = [@fst, @snd]

  include Applicative::Functor
  def fmap(&fn) = Pair.new(fst, fn.(snd))
end

It will be a functor, where fmap applies function to the second element of the pair, without changing the first one. Now let’s define a class that presents a generic parser:

class Parser
  def initialize(&parse_fn) = @parse_fn = parse_fn

  def parse(s) = @parse_fn.(s)
end

parse_fn function should accept and parses some string. When parsing fails it should return Left with error message, otherwise — Right with the Pair. The first element of the Pair is the remaining string, the second element is the thing we just parsed. For instance, here is a parser that looks for a character 'x' in the beginning of the string:

x_parser = Parser.new do |s|
  if s.empty?
    Left("unexpected end of input") # because there is no 'x' at the beginning of the empty string
  else
    char = s[0]
    char == 'x' ? Right(Pair.new(s[1..], char)) : Left("unexpected #{char}")
  end
end

x_parser.parse("") # => Left("unexpected end of input")
x_parser.parse("ab") # => Left("unexpected a")
x_parser.parse("xab") # => Right("ab", "x")

Let’s implement Functor for the parser. fmap accepts some function that can be applied to the result of the parser and constructs a new parser. Note that we do not parse anything, just build a new parser:

class Parser
  include Applicative::Functor

  def fmap(&fn)
    Parser.new do |s|
      parse(s).fmap { |pair| pair.fmap(&fn) }
    end
  end
end

x_parser.fmap(&:upcase).parse("xab") # => Right("ab", "X")

Let’s add a couple of helpers to build parsers easily:

class Parser
  class << self
    def satisfy(&predicate)
      Parser.new do |s|
        if s.empty?
          Left("unexpected end of input")
        else
          char = s[0]
          predicate.(char) ? Right(Pair.new(s[1..], char)) : Left("unexpected #{char}")
        end
      end
    end

    def char(c)
      satisfy { |current| c == current }
    end

    def string(str)
      Parser.new do |s|
        if s.start_with?(str)
          Right(Pair.new(s[str.length..], str))
        else
          Left("unexpected #{s}, expected #{str}")
        end
      end
    end
  end
end

Now let’s make our parser Applicative to make it possible to combine parsers:

class Parser
  include Applicative::ApplicativeFunctor

  class << self
    def pure(value)
      Parser.new { |s| Right(Pair.new(s, value)) }
    end
  end

  def ^(other)
    Parser.new do |s|
      case parse(s)
      in Applicative::Either::Right(Pair(s1, g))
        case other.parse(s1)
        in Applicative::Either::Right(Pair(s2, a)) then Right(Pair.new(s2, g.(a)))
        in left then left
        end
      in left then left
      end
    end
  end
end

pure creates a new parser that always “parses” any value we pass, returning the remaining string as is. Application creates a new parser using following steps:

• if current parser cannot parse a string — we return Left;
• otherwise it returns us the remaining string s1 and the result a, we tries to parse s1 with the second parser:
  • if second parser fails — we return Left;
  • otherwise second pair returns us a function g (because we’re in the Applicative), so we return Right:
    • the first element is a string that remained from the second parser;
    • the second element is a result of calling g with a.

This is how it works:

parser = (
  Parser.pure(lambda { |a, b| a + b }.curry) ^
    Parser.char('A') ^
    Parser.string("BC")
)

parser.parse("ABC") # => Right ("", "ABC")

As you see, ^ operates as the AND operator: in this example we ask parser to find us character A followed by string BC. What about OR?

module Alternative
  def |(other) = raise NotImplementedError
end

class Parser
  include Alternative

  def |(other)
    Parser.new do |s|
      case parse(s)
      in Applicative::Either::Right(r) then Right(r)
      in _
        case other.parse(s)
        in Applicative::Either::Right(r) then Right(r)
        in left then left
        end
      end
    end
  end
end

The new method | tries to parse with the first parser, if it fails — with the second, and returns the result.

Let’s combine everything together:

parser = (
  Parser.pure(lambda { |a, b, c| a + b + c }.curry) ^
    (Parser.char('A') | Parser.char('B')) ^ # we want A or B ...
    Parser.char('C') ^ # ... then C ...
    Parser.string("42") # ... then 42
).fmap(&:downcase) # parser is functor, so `fmap` can be used to transform the result

parser.parse("AC42D") # => #<Either::Right:… @value=#<Pair:… @fst="D", @snd="ac">>
parser.parse("BC42D") # => #<Either::Right:… @value=#<Pair:… @fst="D", @snd="bc">>
parser.parse("DCB") # => #<Either::Left:… @error="unexpected D">

Impressive, right?

Traversable

Let’s try to build something on top of applicatives.

Imagine that you have some applicative container f which has a function from a to b inside. For instance, this function accepts Either Int and tries to increment it returning Either Int:

increment = lambda { |either_value| either_value.fmap { |value| value + 1 } }

increment.(Right(42)) # => Right(43)
increment.(Left("error")) # => Left("error")

Then, we have a new module called Traversable. It has a single method called traverse:

module Traversable
  def traverse(applicative_class, &_fn) =
    raise NotImplementedError
end

This function accepts some applicative_class (which includes Applicative) and a function we described above. The result of the call should be the instance of the same Applicative that contains the instance of the the same Traversable. Traversable should contain the value, that was returned by fn applied to the data inside the initial traversable.

That sounds smart, so let’s just write it and run. Here is the implementation for array:

class ApplicativeArray < Array
  include Traversable

  def traverse(applicative_class, &fn)
    return applicative_class::pure([]) if empty?

    x, *xs = self

    applicative_class::pure(lambda { |ta, rest| [ta] + rest }) ^
      fn.(x) ^
      ApplicativeArray.new(xs).traverse(applicative_class, &fn)
  end
end

And the example of usage:

increment = lambda { |maybe_value| maybe_value.fmap { |value| value + 1 } }

ApplicativeArray.new([Right(1), Right(3), Right(5)])
  .traverse(Either, &increment) # => #<Either::Right:... @value=[2, 4, 6]>

ApplicativeArray.new([Right(1), Left("error")])
  .traverse(Either, &increment) # => #<Either::Left:... @error=“error">

Now we can describe the behavior in the easier way: traverse goes through some data structure containing values in containers, applies the function to the value in container and extracts container to the top. In our example, list of Either Int becomes Either list of Int:

• if initial list has only Right then Right will be outside;
• if there are any lefts — the first one will be extracted.

Outro

In this post we implemented two different applicative functors for lists, created a simple (but yet powerful!) applicative parser and made our lists traversable. As you see, applicative functor is a powerful abstraction that helps to describe complex behavior and hide it inside the container.

Hopefully one day we will make something out of it!

In this post we will see how applicative programming can be used for implementing code in the Railway style using a gem applicative-rb.

Railway programming is a common patten for scenarios when you have a sequence of steps, which are executed in a given order. When everything is good—you get a successful result, if one step fails—you don’t perform the remaining ones and return the failure. In other words—it’s a fancy way of error–handling.

Imagine a service that process an order:

• we deduct money from user account;
• we check that the required item is in the stock;
• we update order status.

This is how we can do it with a DSL similar to well–known dry-monads:

class ProcessOrder
  def initialize(order) = @order = order

  def perform
    result = ApplicationRecord.transaction do # start the transaction
      deduct_from_user_account.bind {
        prepare_shipment.bind { # we get here only when previous step returned success
          update_order_status
        }
      }.tap do |result|
        # rollback in case of failure
        raise ActiveRecord::Rollback.new(result.failure) if result.failure?
      end
    end
  end

  private

  def deduct_from_user_account
    if @order.user.balance > @order.amount
      @order.user.deduct_amount(@order.amount)
      # we return success without explanation
      Right()
    else
      # we return failure with the error message inside
      Left("cannot deduct #{@order.amount}, user has #{@order.user.balance}")
    end
  end

  def prepare_shipment
    @order.item_id == 42 ? Right() : Left("not enough items in warehouse")
  end

  def update_order_status
    @order.processed!
    Right()
  end
end

To make things work each step should return either success (Right) or failure (Left). The behavior itself depends on the container: the decision what to do is made inside the #bind method.

Container Either

Let’s step away from the service object for now and focus on the container. The container is called Either, and it can be implemented in the following way:

class Either
  class Left < Either
    attr_reader :error

    def initialize(error) = @error = error
    def deconstruct = [@error]
  end

  class Right < Either
    attr_reader :value

    def initialize(value) = @value = value
    def deconstruct = [@value]
  end
end

Right means a successful result and might hold some value, Left means some failure with the error message inside. As you see, it’s very similar to well–known Maybe, but can hold a value inside.

Let’s see it in action, fetch_email is a method to get the email by user ID:

def fetch_email(user_id)
  if user_id == 42
    Either::Right.new("john.doe@example.com")
  else
    Either::Left.new("User #{user_id} not found")
  end
end

def format_email(either_email)
  case either_email
  in Either::Right(email) # here we unpacked email string from the container
    Either::Right.new("Email: #{email}") # here we pack new string back to the container
  in left
    left # return container as is
  end
end

format_email(fetch_email(42)) # => #<Either::Right:… @value="Email: john.doe@example.com">
format_email(fetch_email(1)) # => #<Either::Left:… @error="User 1 not found">

How we work with the value inside the container? You have to unpack it if possible (there’s no need to change the error value), work with value and pack back.

Looks like there will be a lot of code where we pack and unpack things! Can we avoid it?

Functors

There is a nice abstraction for it called functor:

module Functor
  # (a -> b) -> f a -> f b
  def fmap(&_fn) = raise NotImplementedError
end

Functor interface has one function, we will call it fmap, like it’s called in Haskell. Ruby does not have types, so we cannot add a signature to the code. Let’s use a Haskell notation, since it’s pretty readable (a -> b) -> f a -> f b.

In other words, it should pass the function that transforms a value of type a to a value of type b. #fmap will call this function on the data in the container, which has a type f a (f is “functor”). As a result, a value of type f b will be returned.

Read more about Functors in my Haskell post

Here is the example implementation for Either:

class Either
  # ...

  include Functor
  def fmap(&fn)
    case self
    in Functor::Right(value) then Functor::Right(fn.curry.(value))
    in left then left
    end
  end
end

With this function format_email can be simplified:

def fetch_email(user_id)
  if user_id == 42
    Either::Right.new("john.doe@example.com")
  else
    Either::Left.new("User #{user_id} not found")
  end
end

def format_email(either_email) = either_email.fmap { |email| "Email: #{email}" }

Proper functor should follow some rules:

1. If identity (def identity(value) = value) is passed, than fmap should return value without changes;
2. fmap (f . g) == fmap f . fmap g, where . is a composition of functions.

Applicative functors

This approach works great for functions with only one argument. But what if we two arguments or more? Thanks to curry, functions can take less arguments and return another function:

def sum(x, y) = x + y

Either::Right.new(42).fmap(&method(:sum))
# => #<Either::Right:... @value=#<Proc:... (lambda)>>

We can call this function with another argument and get the result:

Either::Right.new(42).fmap(&method(:sum)).value.(1) # => 43

How to make it more readable?

Applicative functor interface is the extension of Functor and contains two more methods:

• pure that returns most simple container with value;
• ^ takes the function from the first container and applies it to the value stored in the second container.

Applicative functors also have some laws, but they are a bit more complex so we will not discuss them here. Let’s assume that all the implementations in the post are valid.

This is how interface looks like:

module Applicative
  include Functor # applicative functor should have same methods as functor

  def self.included(klass)
    # a -> f a
    klass.extend(Module.new do
      def pure(_value) = raise NotImplementedError
    end)
  end

  # a -> f a
  def pure(value) = self.class.pure(value)

  # f (a -> b) -> f a -> f b
  def ^(_other) = raise NotImplementedError
end

Let’s try to use it for Either:

class Either
  # ...
  include Applicative

  def self.pure(value) = Right.new(value)

  def ^(other)
    case self
    in Right(fn) then other.fmap(&fn)
    in left then left
    end
  end
end

Note that things will happen only if both containers are Right, otherwise it will just keep the error.

Let’s rewrite format_email one more time:

def format_email(either_email)
  add_label = lambda { |label, email| "#{label}: #{email}" }
  Either.pure(add_label) ^ Either::Right.new("Email") ^ either_email
end

Why is it useful? It makes curry “safe”, because we can describe a “golden path”. Error will be propagated because of the container semantics: we agreed that Left should be kept as is without changes. If you have some steps connected with ^ and one of them returns Left, all steps to the right won’t even happen and the Left will be returned.

If it’s stilly blurry—check out this post with pictures

However, there is a small downside: each argument for the function we want to curry safely (Either.pure(add_label)) should be calculated independently, because these calculations cannot see each other. This is what monads were invented for.

Monads

We discussed monads briefly in the beginning, and I could not stop myself from implementing monads from scratch. We are not going to dig dip into the theory, and jump right to the practice.

Unlike applicative functors, monads can access the data form the previous steps. In order to make something monad you need to implement only two methods return and bind:

module Monad
  include Applicative # monad should have same methods as applicative functor

  def self.included(klass)
    klass.extend(Module.new do
      # a -> m a
      def returnM(value) = pure(value)
    end)
  end

  # m a -> (a -> m b) -> m b
  def bind(&fn) = raise NotImplementedError
end

Check out the source here. As you see, returnM does the same thing as we did in Applicative#pure, while bind is more interesting: it accepts the block, calls it with the current value in the container (if it makes sense, as usual), and returns the result.

We will go with returnM cause return is a reserved word in Ruby

Compare signatures of Applicative#^ and Monad#bind:

• Applicative#^: f (a -> b) -> f a -> f b
• Monad#bind: m a -> (a -> m b) -> m b.

Note the difference: in the Applicative#^ we applied a function inside the container to the value inside the container, while in Monad we pass the function to transform the unpacked value of type a to the value of type b packed to the container m.

Let’s see how to do it for Either:

class Either
  include Monad

  def bind(&fn)
    case self
    in Right(value) then fn.(value)
    in left then left
    end
  end
end

Let’s change fetch_email: now it can also return the invalid email, so we need to validate it before formatting:

def fetch_email(user_id)
  case user_id
  when 42 then Right("john.doe@example.com")
  when 666 then Right("invalid")
  else Left("User #{user_id} not found")
  end
end

def validate(email)
  email.include?("@") ? Either::returnM(email) : Left("invalid email")
end

def format_email(email) = Right("Email: #{email}")

We can write a function that fetches and validates the email using the monad interface of Either:

def fetch_validate_and_format(user_id)
  fetch_email(user_id).bind { |email|
    # we get here only if `fetch_email` returned Right,
    # but email is a String, not Either!
    validate(email).bind { |validated_email|
      format_email(validated_email)
    }
  }
end

fetch_validate_and_format(42) # => #<Either::Right:… @value="Email: john.doe@example.com">
fetch_validate_and_format(666) # => #<Either::Left:… @error="invalid email">
fetch_validate_and_format(1) # => #<Either::Left:… @error="User 1 not found">

Monads are kind of extension for Applicatives, and give us more features. Can we use them always? Not really, because it’s possible to create more applicatives than monads (check out this article to learn more on that).

Service object in applicative style

Let’s get back to the service objects. This is what we had in the beginning of the post:

class ProcessOrder
  include Dry::Monads[:result]

  def initialize(order) = @order = order

  def perform
    ApplicationRecord.transaction do
      deduct_from_user_account.bind {
        prepare_shipment.bind {
          update_order_status
        }
      }.tap { |result|
        raise ActiveRecord::Rollback.new(result.failure) if result.failure?
      }
    end
  end

  private

  def deduct_from_user_account
    # ...
  end

  def prepare_shipment
    # ...
  end

  def update_order_status
    # ...
  end
end

Note that all three actions are completely independent, which makes it the ideal candidate for the applicative approach! Let’s update the service object itself:

class ProcessOrder < MultiStepService
  def initialize(order) = @order = order

  add_step :deduct_from_user_account
  add_step :prepare_shipment
  add_step :update_order_status

  def deduct_from_user_account
    # ...
  end

  def prepare_shipment
    # ...
  end

  def update_order_status
    # ...
  end
end

Now we need to add the base class:

def identity(value) = value # the most helpful function ever
def Right(value = method(:identity)) = Either::Right.new(value) # but we need it to make application work
def Left(error = method(:identity)) = Either::Left.new(error)

class MultiStepService
  class << self
    def add_step(step) = steps << step

    def steps = @steps ||= []
  end

  def perform
    ApplicationRecord.transaction do
      self.class.steps
        .reduce(Right()) { |result, step| result ^ send(step) }
        .on_error { |error| raise ActiveRecord::Rollback.new(error) }
    end
  end
end

#add_step collects a list of methods to be called. #perform reduces steps using the Right() as the initial value. Right() holds the function that just returns a value passed to it (identity), which makes the application work: ^ will run steps until we execute them all or get the first error.

You can find the full example here.

So the only use of applicatives is error handling?

Nope! In this post we explored a single container called Either (and Maybe, because it’s almost the same thing), but there are many more data structures that can implement Functor and Applicative interfaces! Also, the implementation of the applicative functor that follows the laws can be either used as the argument of other functions or have some extensions (like Monad). These way of combining functions can lead us to more complex and interesting calculations.

Outro

Railway programming gives us a clearer way to handle errors: we describe a golden path and errors are handled by the external code in the predictable way. It is usually implemented using Either/Maybe monads, but we saw how to replace them with applicative functors in cases when steps are completely independent.

There is a common mistake to think that monads and applicative functors can be used only for that. In the next post we will see how different data structures can implement the Applicative interface and what interesting behavior they can provide to us.