The RealWorld project is an ambitious effort to gather practical examples of a single web application, written in a wide variety of languages and frameworks. The application is called Conduit, which is essentially a clone of some of the core features of this very application (Medium!). The idea is this: RealWorld provides an interface spec and an API spec (i.e. frontend and backend), developers implement one spec without worrying about the implementation of the other, and then we find ourselves with several hundred unique “stacks” by pairing any frontend example with any backend example. Ever curious to see a Vue frontend talk to a Rust backend? Aurelia and Rails? Dojo and Go? You get the picture: there are a ton of rich examples to explore.

We’ve recently added a new such example to the list by implementing the RealWorld API with hapi pal: a combination of nodejs, the hapi web framework, Objection ORM, and SQLite, all under hapi pal’s methodology/toolset for building APIs. This article takes an in-depth look at the stack and the codebase used to implement the RealWorld API spec, which includes features such as:

• Users can sign-up and authenticate with JSON Web Tokens (JWTs).
• Articles can be created by users, and later edited or deleted only by their owning user.
• Articles may be favorited by users.
• Tags may be attributed to articles, which can later be used in paginated article searches. These searches can additionally filter by articles favorited by or written by a given user.
• Users may follow other users, and each user has a paginated feed of articles by their follows.
• Users may comment on articles.
• Articles must have unique slugs generated from their titles, while titles need not be unique.
• There are particular requirements around response payload formatting and errors.

And a closer look at the stack:

• hapi, our nodejs web framework of choice for its stability, high-quality ecosystem and architecture, bright community, and security-mindedness.
• Objection, our favorite ORM for its SQL-savviness, emphasis on queries over records, flexibility, top-notch maintenance, and helpful community.
• SQLite, an easy-to-install (via npm!), transactional, and relational SQL database—perfect for a widely-shareable project like this.
• And of course hapi pal, an ecosystem of tools and best practices for the hapi web framework. Pal primarily provides us our project structure and application architecture while letting hapi, Objection, and SQLite do what they do best. Beyond that it also gives us some great scaffolding and debugging tools.

Let’s begin!

With all that said, let’s take a look under the hood. We suggest following along with the codebase, which you can find right about… here. You can also find it deployed as a tweakable, hackable live demo hosted on Glitch. If you’re looking for a lighter, more guided introduction to hapi and hapi pal, please feel free to hop on over to our Getting Started tutorial, which utilizes the exact same stack as this project.

Directory structure

The codebase is based upon the pal boilerplate, which splits-up projects into two top-level directories: lib/ and server/.

The lib/ directory contains all the core functionality of the application. Strictly speaking, it constitutes a hapi plugin, which is a portable and well-encapsulated way to articulate a web service. The sub-directories under lib/ each define some pieces of the application: routes, models, services, other hapi plugins, etc. Most of the contents of lib/are picked-up automatically by pal's file-based hapi plugin composer haute-couture, and were scaffolded using the hpal CLI. Without haute-couture we would instead make many imperative calls to the hapi server interface; for example, we would call server.route() rather than creating a file in lib/routes/, server.auth.strategy() rather than a file in lib/auth/strategies/, server.register() rather than a file in lib/plugins/, etc.

The server/ directory contains all configuration and code required to deploy the application. Given that lib/ exports a hapi plugin, server/ is primarily responsible to create a hapi server and register the app's plugin with some configuration provided by server/.env.

The reasoning behind this separation of lib/ and server/ is explained in detail in an article: The joys of server / plugin separation.

The model layer

This application’s model is based upon Objection ORM, which is integrated into hapi using the schwifty plugin. Each model lives in lib/models/ and corresponds to a particular table in the SQLite database: Users, Articles, Comments, and Tags.

Our model layer is very light. It represents a thin mapping between the application and the database, and enforces some basic rules related to data integrity: setting createdAt and updatedAt fields, computing an article's slug from its title, and validating column values when they are persisted to the database. The models are used to interface with the database via Objection's wonderfully expressive SQL query builder which extends knexjs.

You will find that models are used exclusively within the service layer, which is detailed below.

The service layer

The service layer represents a sort of “headless” interface to all the actions and means of fetching data within the application. You’ll find a service method that causes one user to follow another, another to fetch a user’s articles feed, etc. In this way our route handlers/controllers have a means of sharing common logic (“how do I get an article by its id?”) while hiding away the implementation details (e.g. details of the model) in a common library. The service layer is actually generic enough that it could also be re-used to write a different interface to the exact same data and actions, such as a CLI.

We endow our application with a service layer using the schmervice hapi plugin. Alongside the plugin, schmervice also ships with a base service class that provides some useful and convenient functionality, such as access to the hapi server and application configuration (plugin options), integration with the server’s start/stop lifecycle, and the ability to leverage hapi’s robust system for persistent caching.

This application has three services: the ArticleService, the UserService, and the DisplayService, all in the lib/services/ directory. Each service is a class that extends schmervice's base class. The ArticleService comes with methods such as create(), findBySlug(), and addComment(); it provides an interface to articles, comments, tags, and favorites. The UserService comes with methods such as signup(), findByUsername(), and login(); it provides an interface to users, following, and authentication.

Lastly, the DisplayService is responsible for enriching and transforming user, article, comment, and tag models into objects transferred by the API endpoints. This allows us to defer to the ArticleService and UserService to worry about the details of fetching/searching articles and users in various ways—which are complex in their own right—without having to also be concerned with composing the data in these equally complex API responses. For example, the articles list (GET /articles) must be able to paginate while filtering by tag, author, or favorited status; then the API response must additionally include specially-formatted information about whether the author of each article is followed by the current user (if there's a logged-in user), and whether each article is favorited by the current user. In the RealWorld specification there are also multiple representations of some models; for example, a user presents differently when the current user is acting on or asking for information about themselves, versus a separate user (a.k.a. a "profile"). That's a lot of responsibility, so we decided to decouple fetching from enriching/formatting! Luckily, as you will see in the DisplayService, Objection's loadRelated() feature is especially well-suited to this approach.

The final point of interest in the service layer is its convention for transactions. Objection’s handling of SQL transactions is very ergonomic. We take advantage of the fact that you may optionally specify a knex transaction object at query-time to any Objection query. By convention, each of our database-backed service methods take a transaction object as an optional final argument. That transaction object is simply passed down to any queries inside the method, and commits/rollbacks of a transaction are handled by the caller of the service method. In this way any database-backed service method may be composed into arbitrary transactions with other service methods without the caller having to understand the underlying queries being made. More on this in the next section on routes!

Routes

At the end of the day, we do all this work so that we can create some routes, or API endpoints. Each route consists of a hapi route configuration placed as a file in lib/routes/. These configurations provide information about the matching HTTP method and path; validation of incoming query, path, and payload parameters; authentication; and a handler or controller implementing the logic behind the route.

Validation is specified using hapi’s robust joi validation library, which is the same means of validation used by our model layer. Since the routes and models use the same means of validation, routes are able to refer to the model’s validation. For example, when a user logs-in the payload contains an email parameter that must be a valid email; in the route configuration we defer to the User model's definition of a valid email and mark it as a required field: User.field('email').required().

The route handlers themselves are relatively light. They generally compose payload, query, and path parameters, and the user’s authentication status into one or many calls into the service layer, then return a response. Handlers are also responsible for the transactional integrity of their calls into the service layer. For example, if a user makes requests in quick succession to favorite then unfavorite an article, each of those requests must come back reflecting the proper state: there should be no way for the request to unfavorite the article sneak its way in so that the request to favorite the article responds with favorited: false, or vice-versa. So, handlers will often generate a transaction object using a thin helper around Objection.transaction() (defined in lib/bind.js), then pass that transaction to the various service methods that it uses. As mentioned in the previous section, handlers typically end with a call to the DisplayService, whose sole purpose is to format and enrich information about the model (users, articles, comments, and tags) for API responses.

Authentication

Per the RealWorld API spec, authentication occurs via signed JSON Web Tokens (JWTs). There are essentially two sides to this form of authentication:

• The application must hand-out a JWT to a user when that user provides a matching email and password.
• The application must verify the authenticity and contents of the JWT when it is passed with future requests.

In order to hand-out a JWT, we have a login endpoint that performs the process described above by calling into the service layer. In particular, the UserService has a login() method to lookup a user by their email and password, and a createToken() method to create a JWT containing the user's id. Aside from the user id, createToken() also needs a "secret key" in order to sign the token. In our case, we obtain the secret from our application's plugin options (this.options.jwtKey), which the UserService has access to because it extends the schmervice base class. The jwtKey plugin option is set using the APP_SECRET environment variable inside our app's deployment, configured within server/.

In order to verify the authenticity and contents of the JWTs passed with future requests, we utilize the hapi-auth-jwt2 (registered via haute-couture in lib/plugins). This plugin creates an "auth scheme" for JWTs which we configure into an auth strategy in lib/auth/strategies/jwt.js. The auth strategy determines the details underlying our JWT auth: tokens should be signed using a certain hashing algorithm, with a certain secret key (as described above); the token is further validated by looking-up the user whose id is stored on the token; etc. One the auth strategy is created in this way, it's trivial to protect an API endpoint with JWT authentication using hapi's auth route configuration, as can be seen on the route for article deletion.

Error handling

The RealWorld API Spec is particular about the format and HTTP codes that our application responds with. In order to meet those requirements we wrote a centralized hapi request extension, which can be found in lib/extensions/error.js. This request extension is a hook into hapi's request lifecycle to process all responses right before the server responds. In hapi parlance this request extension point is called "onPreResponse".

There are a few different types of errors that are encountered in the app and pass through this request extension. Whenever a route needs to express a standard HTTP error, its handler will throw a boom error, which is standard in the hapi ecosystem. Other errors also may come from within the model layer (e.g. when a record is not found) or from a route’s request validation (these are already considered “400 Bad Request” boom errors). We interpret errors from the model with help from Objection’s objection-db-errors plugin — which normalizes database errors across the various flavors of SQL — and avocat which further transforms them into hapi’s preferred boom HTTP error objects; for example, a uniqueness violation may be transformed into a “409 Conflict” boom error. Once the error is interpreted as an HTTP error, the final step is to simply format them into the shape preferred by the RealWorld specification.

Conclusion

Well, that’s just about all we have for you for now! We hope that the deep-dive proved useful, and that you have some idea what a hapi pal codebase might look like out in the wild. We work on APIs like this every day, which is why we insist that pal is for the practicing, working developer looking for practical tooling. If you’re looking for more info, there’s a community of us who would love to chat with you! Please join us in the #hapipal channel in the hapi hour slack and say hello 👋

Your pals,

Devin (@devinivy) and the pal team

On the homepage of hapipal.com we try to make a memorable first impression when we actively broach the topic we know you’re wondering about: “is this tool going to make things easy for me?”

Writing scalable web services is hard work, and we don’t think there’s any way around that.

“Easy” isn’t exactly what we’re going for.

Now, that’s not to say that we think pal is hard to learn—quite the opposite! (As far as we can tell, folks find it delightful.) We just acknowledge that we have probably not solved all the hardest problems that you have in front of you as you embark on authoring your next web service. You’re going to have to deal with countless issues—sometimes massive cross-cutting concerns of your entire application—that the JS framework/tooling scene has a habit of casually grouping into broad terms: “performance” “security” “realtime” “business requirements”. And we take issue with calling any of this stuff easy.

In fact, we designed hapi pal in such a way that acknowledges the difficulties that lay ahead for you: basically, pal is prepared to get out of your way when it comes time for heavy-lifting. Let us explain!

“Build a production-ready realtime backend in days”

We’re used to seeing claims like this (👆) as theses for blog posts and tutorials, as hooks on framework homepages, in Twitter posts, in contentious Hacker News comments, etc. Whether intentionally or not, it is designed to play into our anxieties about choosing the right tool for the job, as we swirl around in a sometimes dizzying sea of JS tooling and frameworks.

If I try this framework out and it’s taking me over a week to build my application, is it because I’m doing something wrong? 😓

Rest assured, if your application is taking more than days to complete then it doesn’t necessarily mean there’s anything wrong with your technology choices or your approach. Depending on the inherent complexities of the problem you’re servicing, data consistency guarantees, security requirements of your business/country/industry, service-level agreements your API must uphold in the way of availability and responsiveness, your Ops budget, the number of anticipated concurrent users and magnitude of traffic influxes, the size of your team, etc. it would not be at all weird for a project to take months or longer under any framework. And if the term “production-ready” doesn’t encapsulate the laundry list of concerns above, what are we even talking about?

Now, if we all agree that your production-ready application oughtn’t be assumed to be easy to build, what are the technical ramifications for framework authors and tool builders? It all comes down to the abstractions that these libraries choose to introduce their users.

Abstraction leaks

The only thing worse than memory leaks are abstraction leaks. You know that feeling when you feel cornered by a “missing feature” of one of your project’s dependencies? When some library or reusable module feels like it carries existential weight with it wherever it shows-up? When the code that you write using a library seems so distant from its performance characteristics? That’s what a bad leaky abstraction feels like.

A common definition for this you’ll find on wikipedia: “a leaky abstraction is an abstraction that exposes details and limitations of its underlying implementation to its users that should ideally be hidden away.” I would opt to amend that, because I think these leaks go both directions. If a library requires changes to its implementation in order for application code to circumvent its limitations, that’s a leak too. For example, how many times have you tried to use some particular MySQL or PostgreSQL feature only to find that you were completely barred from it until it’s implemented in your ORM of choice? That’s not an ORM with missing features—it’s an abstraction leak.

Yes, abstractions are beautiful. They’re also bold and sometimes fraught with hubris. After all, creating an abstraction is an assertion that you can solve multiple different problems at the same time. In a world of development where we all share problems that are similar but not quite identical, what does a healthy abstraction look like? Just how far ought framework and library authors go in attempting to solve others people’s problems?

There’s a balance to be struck here between having a tight, minimally-leaky abstraction that doesn’t offer much value, and a massive-but-leaky abstraction that provides great value only within its limits. Libraries that claim to make your life “easy” or “quick” have a tendency to end in one of these “massive-but-leaky” abstractions. And the reasoning there isn’t so controversial: optimizing for easiness implies feature richness over the ability to customize and fine-tune. Needless to say, we’ve sort of obsessed over this balancing act in authoring hapi pal, and we’ve learned a thing or two along the way.

How pal takes a stand

We designed hapi pal with a hyper-sensitivity to all that we acknowledge as “hard.” Sensitivity to all the problems that we could never anticipate, that you are inevitably going to encounter. As such, we have spent a massive amount of effort to ensure pal can get out of your way when the going gets tough. In the end it’s a bunch of small design decisions that we hope add-up to a flexible, minimally leaky experience. Here are a few examples of the many “escape hatches” built into pal,

• haute-couture isn’t a hapi plugin itself. We want you to be able to use haute-couture to compose your hapi plugins from files, but also want you to feel free to write and run any other plugin code you may need—for example, if hapi introduces a new feature that haute-couture doesn’t immediately support. So we made sure that haute-couture can be used as a sub-routine of any plugin, and you can write all the custom code you need!
• Objection ORM is really just a glorious SQL query builder. When deciding on a data access layer for ourselves, we wanted to break away from the massive abstractions ORMs traditionally introduce. Objection strikes a wonderful balance, where it focuses on top notch support for features of SQL data stores specifically (composing transparently with the knex query builder) rather than attempting to create a generic interface for working with “records.” This allows us to drop down to the metal and work closely with all the hard stuff that we need like transactions, column types with specialized indexes (e.g. tsvectors, JSONB, and geospatial types), aggregations, join semantics, etc. Objection makes working with model/table relationships and these critical features of our RDBMS feel organic.
• schwifty hooks-up models to db connections for you… unless you do it first. Schwifty is a hapi plugin integrating Objection ORM as a model layer for your application. It respects hapi plugin boundaries by letting you configure different db connections for your models on a per-plugin basis, e.g. to handle the case that you’re composing multiple pluginized hapi apps into a single deployment. We find this to be very flexible, but we also acknowledge that there may be more complex use-cases out there: what if someone has a single legacy users table that lives in a different db from all their other models? In that case, no sweat—schwifty is designed to get out of your way if it sees you manually hooking-up db connections to a given model.
• the contents of scaffolding files made by the hpal CLI is totally customizable. The hpal CLI has a command hpal make that allows you to scaffold files for new routes, models, services, auth strategies, etc. We have reasonable default contents of those files, but perhaps you’d like to add a little flair or make it match your coding style more closely. The file contents can be customized by simply creating a haute-couture amendment in your .hc.js file with an example property, which means you’re free to make hpal make behave however you like! In a similar vein, you can use haute-couture amendments to change the meanings of directories in your project: perhaps you want models/ to contain Mongoose models rather than Objection models, for example. We’re happy to get out of your way so that you can work within your own requirements.

There are a bunch more examples of this in the pal universe, but these are a few important ones that jump to mind. Several modules/libraries are mentioned above. All these libraries are also designed to function and “make sense” entirely on their own, but if you’re looking for a more integrated experience, we highly suggest checking out the pal boilerplate, which comes setup to author hapi applications with top notch hapi debugging tools, a test suite and linter, a configuration and deployment strategy, and a bunch of “flavors” e.g. to integrate schwifty and Objection ORM, Swagger UI, templated views, and more. Not to mention a very welcoming community on the hapi hour slack (find us in #hapipal!). We’d love to have you.

Your pals,

Devin (@devinivy) and the pal team

Important P.S.!

We want to make it clear that hapi pal is not here to point fingers! No content of this post should be used to shame open-source maintainers for their “leaky abstractions” or shameless promotion of “the quick and easy.” It’s always your responsibility to understand the abstractions, approaches, limitations of the dependencies you introduce into your application. The greater JavaScript community has a long tradition of touting libraries as quick/easy solutions to complex problems, and we are simultaneously subject to hype, tooling anxieties, “JS fatigue” as much as we are instruments of it. We’re just hoping to acknowledge these issues, counter them systemically, and write some delightful hapijs tooling while we’re at it. 💞

Below you’ll find,

• We spoke at the CBBHacks hackathon. The students were very smart.
• We launched a “getting started” tutorial. Finally, right? Also, a simple pal deploy via Glitch.
• We’re releasing a tool that generates knex migrations for you.
• In progress: three novel CLI debugging tools for hapijs. One of them just might be a server- and route-aware cURL command.
• We’re designing “overridable” hapi plugins, paving the way for some big plans.
• A new, concise way to write hapi route pre-handlers.
• The greater pal community has been busy too! Over the past couple weeks we’ve seen the community write pal-oriented tools for schwifty and Objection ORM, and produce a very curious example project.

We spoke at the CBBHacks hackathon

CBBHacks was a 36 hour hackathon hosted by Colby College open to university students across Maine (USA), and attended by nearly 90 students. We visited and had an awesome time. It was heartening to be reminded how much awareness has increased in academia surrounding web technologies, even over the past five years. At 11:30pm on the first night, about 50–60 students and organizers gathered to hear @devinivy give his spiel on building web services; how nodejs relates to the web; why he reached for hapijs; the impetus for starting pal; and finally a demo of hapi pal tooling to build an API that serves a random riddle. One team actually utilized pal for their project (so cool!), and we hope that one of them will join Big Room Studios for an internship this summer.

We launched a “getting started” tutorial

One thing that, in retrospect, we wished we had completed prior to the pal release was a proper “getting started” tutorial. And now, with huge thanks to @zemccartney, anyone should be able to get started in short order! By its nature, hapi pal defers to other packages and ecosystems that are great at what they do — for example, Objection ORM, knex, and hapi itself. Having a high-level overview of how all of these interface with pal (and each other) is really critical context that this tutorial provides. Plus, the tutorial is fun: you help your pal, Paldo, share riddles with their friends by incrementally building a database-backed API. The completed example can be found in the new hapipal/examples repo. Relatedly, we have a new vanilla pal deployment all setup to be remixed on Glitch—just look up and to your left!

We’re releasing a tool that generates knex migrations for you

# Soon!
$ hpal run schwifty:migrate:diff migration-name

As we work on web APIs, we find ourselves spending a fair amount of time combing through new knex migration files to ensure they match-up with our model definitions. It’s tedious, often repetitive, and prone to error. In short, it sounded like a nice thing to automate for ourselves, so we did! (With tons of effort especially from @wswoodruff!) This plays nicely with the hpal CLI’s ability to run commands defined directly on your hapi server: we’re adding a command to schwifty, our model layer for hapi, that performs a diff between your model definitions and the current state of the database, then outputs a base migration file that you can use as a safe starting point to add indexes, foreign keys and other constraints, specifics about column nullability, etc. We’re not trying to cover every possible case or make any assumptions about your application—we just want want to remove the most repetitive tasks associated with writing migration files. We’re aiming to release schwifty:migrate:diff this week!

In progress: three novel CLI debugging tools for hapijs

We have begun development on some new debugging tools for hapijs development, provided as an extension to the hpal CLI, which already allows you to search documentation, create new projects, and scaffold existing projects. We’re working on a REPL for interacting with your server, a panel describing your deployment (routes, services, models, etc.), and a cURL-like command that exposes your routes and their validated parameters over the command line. We’re especially excited about this last one because it potentially removes the common development flow of restarting the server then making requests to it repetitively. Instead, requests can be made easily as one-offs, even when a server isn’t running. Looks for these debug tools released as hpal-debug some time in the next few weeks! Below is a demo of what we currently do have working.

# Just a demo for now, but planning to release soon!

$ hpal run debug:curl post /users --name Paldo -v

post /users (41ms)

payload
───────────────────────────────────────
 name   Paldo

request headers
───────────────────────────────────────
 user-agent        shot
 host              my-computer.local:0
 content-type      application/json
 content-length    16

response headers
───────────────────────────────────────
 content-type      application/json; charset=utf-8
 cache-control     no-cache
 content-length    24

result
───────────────────────────────────────
{ id: 10, name: 'Paldo' }

We’re designing “overridable” hapi plugins

We’ve dreamt of highly reusable application plugins for almost as long as we’ve been deploying hapi servers. Imagine the ability to pull-in a community- (or internally-) maintained “users” plugin into your project as a simple dependency (users CRUD, login and logout, password reset, etc.), without losing the ability to customize all of its internals for the purpose of your application. It’s a tricky problem that we’ve been chewing on for a while, and over time we’ve gradually built-up the machinery that we think is necessary to make this a reality. In particular,

• We now have a class-based service layer via schmervice—including the concept of “plugin ownership” of services—that can be used to implement and extend the business logic used in route handlers or elsewhere.
• A nice result of composing hapi plugins from files via haute-couture is that we have a natural identifier for each call your plugin makes to hapi’s API—say, to add a route with server.route() or a model with server.schwifty(). The identifier is just the filename itself, e.g. "routes/user-create.js” or "models/comment.js”! So if we want to allow skipping or altering any calls like this (which we do!), we have a natural way to reference them.

@devinivy threw together a proof of concept of one plugin overriding another plugin’s routes with relatively minimal changes to haute-couture, and @mattboutet is refining it to more gracefully handle other hard problems that come with this new ability. We don’t know precisely when we’ll be releasing this haute-couture feature or if it necessitates any breaking changes, but be on the lookout!

// Just an experiment... for now

const HauteCouture = require('haute-couture');

// my-user-plugin overrides a base-user-plugin
module.exports = {
    name: 'my-user-plugin',
    register: HauteCouture.using({
        overrides: require('base-user-plugin')
    })
};

A new, concise way to write hapi route pre-handlers

We love route pre-handlers. They’re a great place to write route-level authorization rules, fetch data to use in the handler, or any routines that are otherwise incidental to the “guts” of your handler. We love them, but have always wished that they were just a little pithier—mostly because writing an object that looks like { assign: 'user', method: getUser } just feels like it should be able to be written { user: getUser }. And that’s all we’ve done! The new Toys.pre() method transforms concise pre-handler configs into hapi’s longer configs. The toys package contains all sorts of other goodies like this, such as concise request extensions, and helpers to work with events and streams inside async handlers.

The greater pal community has been busy too!

Over the past couple weeks we’ve seen some really sweet work by community members.

Avocat is a much-needed tool by @optii to help transform errors from Objection ORM into relevant boom HTTP errors. Its API is inspired by hapi’s own bounce module, which is pretty neat.

The schwifty-i18n module, also by @optii, makes schwifty models translatable. It’s labeled as a work in progress, but seems to be coming along really nicely. We can see that it’s designed to work especially well with haute-couture, which warms our little developer hearts.

Finally, @jeff-kilbride shared a really interesting example of a minimal haute-couture project organized using multiple plugins. Jeff decided to drop some dependencies of the pal boilerplate so that it could run lighter on AWS Lambda. Slick stuff!

That’s all!

Well, that’s all for now. We clearly have a lot of gratifying work ahead of us, and better yet, many new conveniences on the horizon. Again, big thanks to those mentioned in this update for their contributions. If you’re looking to get involved, feel free to pick up any issue labeled help wanted or simply join us in the #hapipal channel of the official hapi hour Slack.

Your pals,

Devin (@devinivy) and the pal team

Today we are overjoyed to announce hapi pal, a suite of tools and best practices for the working hapijs developer. We have invested years of energy into what we have found to be a delightful, sustainable approach to developing web services using nodejs, and it’s finally time to open up.

What is hapi?

If you’re not familiar with hapijs, it’s an open-source web framework for nodejs that got off the ground with a 2+ million dollar investment from Walmart Labs and now operates entirely through an open governance model. (Sometimes we joke that we’re driving the Rolls Royce of web frameworks.) The framework is extensive, offering a uniquely rich plugin architecture and ecosystem, integrated concepts of caching, authentication and authorization, a strong community, and an emphasis on stability and security. It touts itself as “enabling developers to focus on writing reusable application logic instead of spending time building infrastructure.” And it really does — it’s a beautiful thing. We’re familiar with the alternative frameworks, those both well-established and new, and we find ourselves regularly renewing our vows with hapi.

Creating efficiency

But it’s important to note that the core hapi organization has intentionally remained unopinionated about how to author applications using hapi. The framework only offers a programming interface: there’s no directory structure, concrete deployment strategy, model or service/business layer, and generally no suggested architecture whatsoever. To be honest, we think that this is totally fair on hapi’s part. For one, there are only so many resources to spend on hapi development, so not focusing on additional tooling means that the rest of the framework is more stable and fully-featured. And secondly, we’re glad they’ve made room for the community to experiment.

For this reason, we found it invaluable to create for ourselves common conventions, tooling, and patterns that jive with hapi. Big Room Studios takes on a lot of projects, and there are incentives — for our business and those of our customers — to avoid solving the same problem or making the same decision too many times, which our development team understands acutely. It’s important for us to share a common knowledge base, to be able to onboard new developers easily, to have consistency across projects that enables us to effectively perform long-term maintenance. These issues are probably familiar to you too. Our solutions to these issues, refined over the course of nearly four years, constitute hapi pal.

Building a set of tools

You can think of hapi pal as a methodology for building hapi apps, backed both by software and written articles to guide you. While consulting on others’ hapi deployments, we find certain hapi features to be underutilized, often simply because they are covered without much ceremony in the hapi docs, or for other superficial reasons: sometimes it can be hard enough to decide where to put some code that you fall back to familiar techniques. We think that this is a very real, valid conceptual gap, and that it’s worth closing for the betterment of the community. We want to make it simpler to learn the framework, and to bring the most powerful features of hapi into reach. No single aspect of pal is required (we’ve incrementally adopted these tools ourselves) but here are some things pal has to offer:

• hpal — A CLI tool for searching hapi-related documentation, starting new projects, scaffolding existing projects, and running custom commands in the context of your project.
• the pal boilerplate — A generic boilerplate setup for testing, linting, development, and deployment while enforcing 12factor practices and pluginization of your app. The boilerplate can be customized using “flavors,” which add common functionality, e.g. database integration, templated views, Swagger API documentation, and more.
• haute-couture — A plugin composer that wires-up your plugin for you based upon a (customizable) directory structure. Rather than calling server.route(), place route configurations in the routes/ folder; implement your auth strategies in auth/strategies/; register additional plugins by configuring them in plugins/; define models in models/; services in services/, etc.
• schwifty — A model layer for dialects of SQL based on the inimitable Objection ORM.
• schmervice — A service layer that integrates with hapi caching.
• hecks — Interoperability with express applications.
• underdog — A plugin adding support for HTTP/2 server-push.
• tandy — A plugin that helps auto-generate CRUD routes for your models.
• toys — A collection of hapi-oriented helpers.
• best practices — A growing list of articles on idiomatic hapijs app development.
• More, including a package that helps resolve hairy inter-plugin dependencies (hodgepodge).

A hapi pal when you need it

Now, I’m specifically not here to claim that hapijs or hapi pal are going to allow to you to ship complex features with tight SLAs to millions of users in a matter of minutes, hours, or days: one of our founding design principles is acknowledging that building scalable web services is hard work. We know this because we’ve built tons of them, and in plenty of different languages and frameworks: there’s no silver bullet. So we’re not interested in any platform that gets in our way when it comes time for heavy lifting. When that time comes, wouldn’t you rather just have a pal to cheer you on, offer you a hand and moral support, and leave you to your own devices if you don’t really need the extra help? That’s what hapi pal is all about.

Sponsored and maintained

And did we mention hapi pal is sponsored?! Big Room Studios has demonstrated a commitment to open-source software through development and design of hapipal.com, development of schwifty and tandy, lots of dogfooding, and the ongoing maintenance of numerous pal and hapijs modules. The hapi pal organization is openly governed, but Big Room has helped make this initial release a reality, and plans to support several of us in our maintainership of hapi pal going forward. We’re thankful for that because we love using and writing well-maintained software. Now, don’t get us wrong — there’s still a lot of work to do and we could use your help too.

The community

So, come check it out. Join us — we’d love to have you! If you’re already a part of the hapi community, we hope you’ll be pleased to learn that hapi pal is designed to be entirely compatible with the hapi organization. We have the same code style, identical issue labels, the same standard of 100% code coverage, and a consistent code of conduct, all of which we are quite familiar with, as some of us are also core and community hapi collaborators. We will continue to move in step with the hapi organization, and we earnestly look forward to seeing where we end up. Also, be on the lookout for some big new pal capabilities on the horizon. We have some fun plans — come chat with us about them in our #hapipal channel in the hapi hour Slack. Hope to see you there.

Your pals,

Devin (@devinivy) and the pal team