Making PR reviewing work across projects

A crucial part of our development process at Firmhouse is that we review each other’s code and leave comments via Pull Requests on GitHub. Code only gets into the master branch of our projects after someone submits a Pull Request and reviews the code. I think we’re no different from any other proper development team out there.

But in our case, we always work on several projects at the same time. Either two of us pair up on a project, someone is soloing a project, or someone has the “support” role, which means they will be freewheeling between projects that have support issues coming up. It is not uncommon that you are the sole developer for the duration of a sprint.

So how do we at Firmhouse make sure your code gets reviewed without much delay? We went through some steps the past few months and eventually ended up creating a very nifty tool: Review Please.

1. Using a #review-please channel in Slack

We use Slack at Firmhouse for all our ad-hoc communication throughout the day. A few years ago we introduced the #review-please channel. All developers would sit in that channel, and when someone would finish a Pull Request, they would post the link to the Pull Request in that channel. Someone else would then review that Pull Request.

Posting PRs to a Slack channel manually works great when you only have a few Pull Requests per day. But as the team grew more and more Pull Requests came in throughout the day that flooded the #review-please channel. The consequence: no more overview of which Pull Requests had been reviewed or not, and some Pull Requests would sit for hours or days before they would get reviewed.

2. Introducing "Review Please"

During Firmhouse “adventure time”, Jeroen and Bram started an awesome little project to solve several problems around reviewing Pull Requests within the team. They came up with an awesome service for reviewing and managing Pull Requests within our team. They dubbed it (drumroll) review-please.com.

Review Please has three main elements that help us throughout a day of software development. Here they are:

The web app and GitHub integration

A Rails application is at the heart of Review Please. In that app, everyone on our team can log in with GitHub accounts. When they add a repository, Review Please creates a webhook in GitHub. That webhook pushes Pull Request changes back to Review Please, ready to be processed in the proper way.

Slack integration with #review-please

In our old way, we would manually post a “review please” request to our #review-please channel with a link to the pull request on GitHub. Not anymore. Review Please now takes care of that through informative Slack messages:

Review Please new Pull Request

It even reminds us if a Pull Request stays open for too long. In our development process we’d like to keep sustainable pace, so we want to prevent Pull Requests from staying open too long. Here’s such a reminder in our Slack channel:

Review Please Slack Reminder

Open PRs in Rails app and Menubar app

The Slack notifications in #review-please are very nice, but you still need to be in the Slack channel to follow them. That’s why Bram started working on a little Menubar Mac app. It’s a tiny icon that lives in your Menubar, and when you click it, it shows outstanding Pull Requests over all projects in your development team:

Review Please Mac Menubar app

The menubar app is great to keep up with open PRs that need a review, without having to be on Slack all the time.

The web application also features such an overview:

Review Please open PRs in web app

Extra nifty feature for Pull Requests in progress

As mentioned above, Review Please will remind us in the Slack channel when a Pull Request has been open for too long. So that’s already a nice thing to have. It also reminds us via MacOS notifications if you’ve got the Menubar app installed.

We also took care of the situation where you want to post “work in progress” Pull Requests and don’t want to notify everyone yet. Posting a work in progress Pull Request is useful if you want to get feedback on your work from someone 1–1 or if you don’t want to have a formal code review yet, but just discuss something about design or architecture. So Bram and Jeroen came up with two solutions:

  • Prefix your PR with [NR] (no review) and Review Please will ignore it altogether
  • Prefix your PR with [WIP] (work in progress) and it will wait for you to remove that tag. Once you remove the tag via GitHub, it will continue to notify the rest of the team that your pull request is ready for formal review.

Want some Review Please too?

We’re happy to help you set up your own on-premise Review Please installation too. The Mac menubar app and Slack integration will just work because you can configure your own endpoint. If you’re interested in a reviewplease.yourcompany.com, let us know! You can email bram@firmhouse.com or michiel@firmhouse.com.