One year ago I had a problem and I decided to solve it. I was teaching Node.js in a workshop with Hacker Paradise and I got to the point where I normally say:

This code is too complex to explain at this point, so let’s just copy/paste it and continue.

That part was when adding all of the middleware to make Express have bodyparser, sessions, cookies, etc. I thought it’d be really nice to have this set of basic functionality working for everyone by default.

Now with server I’d do this and have a fully working website:

const server = require('server');
const { get, post } = server.router;

server(
  get(ctx => 'Hello world'),
  post(ctx => console.log(ctx.data))
);

Getting a great npm name

Modern javascript

Another thing that I didn’t like about Node.js programming is Callback Hell. With Await/Async this is easily solved. Let’s say a simple scrapper that saves the links of the passed url into a database and replies to it:

const server = require('server');
const { get, post } = server.router;
const { render, json } = server.reply;
const fetch = require('node-fetch');

// Up to you how to implement these (Cheerio + MongoDB?)
const scrapeLinks = require('./fetch-links');
const saveLinks = require('./save-links');

server(
  get(ctx => render('index.jade')),
  post(async ctx => {
    const saved = await Links.find({ url: ctx.data }).exec();
    if (saved.length) return json(saved);
    const body = await fetch(ctx.data).then(res => res.text());
    const links = scrapeLinks(body);
    await saveLinks(ctx.data.url, links);
    return json(links);
  })
);

That’s the main logic for it. Quite intuitive and direct compared to the callback way. Async/await are truly awesome.

Achievements

These are the things I am most proud of, from a personal point of view and for the project itself.

I could make the library that I enjoy and use for my personal projects. It is easy and intuitive. I found many bugs and edge cases from daily use and would love that you report any bug that you might find.

While neither of them officially released, it has great socket.io integration and a powerful plugin system. That is on top of being backwards-compatible with any Express middleware that you can find, a big win IMO.

The Documentation has seen a lot of care and hard work. I will do my best to separate it into a different module in the future to be able to redistribute it and use on other projects. Want some nice docs like those? Hire me.

Things I’d change

Make it sooner! I didn’t expect it to take a full year. I knew it was a longer project, but I expected to release it 3–6 months ago. The two main reasons are life, which got in the way, and me trying to make it future-proof.

Get more people using it sooner. When I teach web programming to friends I still follow express route. However I know that if I am spending my time teaching friends and acquaintances they won’t care or they’d even be happy to give me a hand on this.

Next steps

First to take a small break. If it becomes popular it’s probably going to be short, but I’d like to explore some other areas for a bit.

Then, the main priority is expanding the documentation and tutorials. This is a lot of work but IMO one of the best return of investment for any library. If they cannot find it, it doesn’t exist kind of thing.

Finally, make a plan and look forward for the 1.1. This will include official socket.io and the full plugin system. Also, some pre-made plugins such as an auth one, sass, react, etc.

Looking for Sponsors

I’m also looking for sponsors for the project. It doesn’t get many visits but surprisingly it gets a decent and steadily growing number of installs. So this looks like a small but passionate userbase, feel free to sponsor the project: Sponsor website.

Please let me know what what you create:

Server.js website, Documentation & Tutorials

~happy hacking ♥

Server.js 1.0 released — for Node.js 8 LTS was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.

When I was traveling with Hacker Paradise through Asia and we got to the same point I decided to do something about it. First I created a file upload middleware and then started the ground work for what now is server.

The final trigger was getting the package server in npm. After that and based on the previous experiments, I decided to set out for real and make something worthwhile. So I dug into express.js, middleware, routers, etc and now I'm proud that server is something I use to make websites faster and easier.

Join me in Github to get the best out of server and help Node.js achieve its full potential. Next up are websockets routes:

const server = require('server');
const { get, socket } = server.router;
server(
  get('/', ctx => ctx.res.render('home')),
  socket('connect', ctx => ctx.io.emit('connected', ctx.socket.id)),
  socket('message', ctx => ctx.io.emit('message', ctx.data))
);

Goals

These are the main things that I wasn’t happy with the state-of-the-art, so I decided to launch server to build upon the great work of express:

1. Make things work by default (by domain):

• Parsers: json, urlencoded, file uploads, etc.
• Persistence: session, cookies, csrf, etc

2. Make things simpler to use:

• Many low-level things work out of the box
• Sensible, secure defaults => no need to change anything
• Easily customizable options

3. Make some important services available where possible, or some hooks to make it easier:

• Passport
• Database (MongoDB, etc)
• Websockets

4. Better error handling (if possible). No more ALLCAPS error messages with no information at all.

This will in turn make it much easier to get started, for both people who are new to Node or for experienced people who don’t want to set-up everything again and again. The main frustration that I’ve seen from people coming from:

• Different web backgrounds (Ruby on Rails, PHP), where now they have to hunt down and compare dozens of libraries to do simple tasks.
• Different programming backgrounds (Arduino, C++) where I have to explain not only how to get a server ready with these new tools, but also how the current state of the art and the fun of it is to build your own stack.
• Starting from scratch. I pity those people starting with no programming experience coming to Node.js. This shouldn’t be so difficult, should it?

So that’s it, those are the reasons for me to start server.js. Star the package in Github if you like it and check all the info in the official website.

About server.js was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.

You might be surprised to know that I am developing server.js under the npm name “server”, so I can direct users to just do npm install server. Wait, wasn’t this package taken already?

In fact it was; there was a 0.0.3 version published a few years back. But npm’s “Package Name Disputes” is pretty clear about it:

“Some things are not allowed, and will be removed without discussion if they are brought to the attention of the npm registry admins, including but not limited to: […] “Squatting” on a package name that you plan to use, but aren’t actually using. […] Putting empty packages in the registry.”

From the lack of use from the author and devs, npm decided to hand it over when the original author didn’t answer my emails. Let’s see how to do it.

A package has a name

First of all we have to identify a package name that we might want to use. My rule of thumb is to try to find a package that follows most of these:

• Unstable: latest version in the low 0.x or even in the 0.0.x. This means the package is still marked as “in development” as per semver guideliness so the work should still not be used in production by anyone.
• Inactive: The package has not received any significant update in years, meaning that the authors are probably long gone or don’t care anymore.
• Not used: The weekly installations are in the low numbers; they will never be 0 though, since there are many caches and crawlers just installing them (under 10/week is a good number). You don’t want to break packages used by thousands of developers.
• Trivial: there hasn’t been a lot of work put into the package. This is more of a respect thing as someone put a lot of work into it.

I find this info with url hacking writing https://npmjs.com/package/NAME for precise matching, but the search functionality has been improved greatly in the recent years so you can use that as well.

There was something weird going on in the early npm versions so the name is case-sensitive; you’ll have to check the lowercase name as that’s the only supported ones nowadays.

Get the package

Great! You found an amazing package name that follows most of the points mentioned above. The quoted dispute policy is to automatically remove a package, but we might also want to use a name that is not in direct violation of the conditions. It’s time to wear our peopleware hat.

It’s simple and well explained: email the person holding the current package. Most people are actually easy to reason with so just ask nicely and give a valid reason (the name is perfect a package you are finishing). CC npm staff so in any way it will be resolved within 1–2 months.

The other person might not have access to that email or not care anymore, in which case npm will intervene and give their verdict about what will happen after few weeks.

Next steps

You might be asked by npm to publish from the 1.x version to avoid any possible compatibility issue. However you can get around this to not yet launch an official build by using 1.0.0-alpha.1 versions first.

Let me know if you get a great name or have any question! Also check out server.js to make a full Node.js server easily with npm install server and:

// index.js

// Include it and extract some methods for convenience
const server = require('server');
const { get, post } = server.router;

// Launch server with some options and a couple of routes
server({ port: 8080 },
  get('/', () => 'Hello world'),
  post('/', ctx => console.log(ctx.data))
);

Getting a great npm name was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.

After 50 alpha releases — since NPM asked me not to publish 0.x versions — I am proud to release the first public-testing beta release. Feel free to install and use it under the permissive MIT license:

const server = require('server');
const { get, post } = server.router;

server(
  get('/', () => 'Hello 世界'),
  get('/html', () => '<p>I am a <strong>paragraph</strong>!</p>'),
  post('/', ctx => console.log(ctx.data))
);

Test release

The code is in a good state to be used right now and with your help I’ll be able to make it production-ready for the final 1.0.0. Right now it should work with Linux, macOS and Windows.

There are few things missing from the documentation so I do recommend everyone to use server.js only for small projects right now. But please do so, break it and report it so I can fix it!

If you have any questions about how to do something just open an issue and ask, it’ll also help building the documentation.

Next steps

Right now the documentation it’s just a stub with some parts missing or inaccurate. The idea is for the documentation to be in a usable state for the Beta 2 so devs can effectively use it and then polish it all the way to the 1.0.0. Hopefully there will be just 2–3 betas.

It is a long road, but I think I will be able to launch the second beta with usable documentation in 2–4 weeks. For people using it now there’s a specification file to make sure you know what the stable API is.

Finally and to make it truly production-ready many tests and examples must be added. I’ve been adding some — specially with unexpected bugs— but I’ll need to write many more.

Try it out!

npm install server

Then create a hello world in your index.js:

const server = require('server');
server(ctx => {
  return 'Hello world!';
});

Open http://localhost:3000/ and you’ll see it working. To know how to use it all see the official website.

I would love to know what you think, so please leave your feedback on the Hacker News thread and check the code and star it on Github.

Server.js beta 1 released was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.

Good news: from Node.js 7.6 you have native async/await support!

Server middleware

By building Comments Network I learned a couple of tricks. If we were to do it naively we could implement it like this with server:

// Promise-based requests
server(get('/story/:id', ctx => {
  request(`API CALL${ctx.req.params.id}`).then(api => {
    request(`HN${api.url}${ctx.req.params.id}`).then(hn => {
      // Do some work with `api` and `hn` here
      ctx.res.send(workedout);
    });
  });
}));

// Async/Await based requests
server(get('/story/:id', async ctx => {
  const api = await request(`API CALL${ctx.req.params.id}`);
  const hn = await request(`HN${api.url}${ctx.req.params.id}`);
  // Do some work with `api` and `hn` here
  ctx.res.send(workedout);
}));

You can use it now and forget about messy Promises and Callback Hell. Of course you will have to use libraries that are based on Promises as well, since some of the popular Node.js libraries such as fs and request are callback-based (check out fs-promise and request-promises).

Testing

Now let’s see about Jest. By default, Jest and other testing frameworks accept two ways of doing asynchronous tests. Through a function that accepts a done parameter or through a function that returns a Promise. But thanks to async the third option is a no-brainer (see it live):

// Callback based
it('can be loaded', done => {
  drive.load().then(db => {
    expect(db.data).toEqual(jasmine.any(Array));
    expect(db.data.length).toBeGreaterThan(0);
    done();
  }).catch(done);
});

// Promise based
it('[promise] can be loaded', () => {
  return drive.load().then(db => {
    expect(db.data).toEqual(jasmine.any(Array));
    expect(db.data.length).toBeGreaterThan(0);
  });
});

// Async/Await based
it('[async] can be loaded', async () => {
  const db = await drive.load();
  expect(db.data).toEqual(jasmine.any(Array));
  expect(db.data.length).toBeGreaterThan(0);
});

That’s it. You don’t even need a return as functions return when finished so they will resolve then as expected.

HOWEVER not everything is as great as it looks. Testing for a case where an error is thrown is quite more difficult with async:

// Callback based
it('can be loaded', done => {
  drive.load().catch(err => done());
});

// Promise based
it('[promise] can be loaded', () => {
  let passed = false;
  return drive.load().then(db => {
    passed = true;
  }).catch().then(() => {
    if (passed) throw new Error('Did not throw an error');
  });
});

// Async/Await based
it('[async] can be loaded', async () => {
  try {
    const db = await drive.load();
  } catch (err) {
    return;
  }
  throw new Error('Did not throw an error');
});

But the good news is that with a bit of code you can fix this:

throws = cb => async () => {
  try {
    const res = await cb();
  } catch(err) {
    return;
  }
  throw new Error('No error was thrown');
};

Then you can use it like this with async/await; back to the good bits:

it('[async] can be loaded', async () => {
  await throws(() => drive.load());
});

The future for Javascript is looking quite good even though there’s a long, fragmented road to get there. I’m building server for Node.js to get there faster, subscribe or follow me to get updates!

Async/Await are awesome was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.

I decided to step back and change the way middleware works before committing to 1.0.0. It hasn’t been easy, however I think that we can achieve a lot more with ES6+ than with Connect’s middleware and Koa’s middleware is way too complex for Server’s purpose.

Definition

I decided to create a new standard for Server’s middleware:

Synchronous: just a function that gets the context and returns anything.

// Parse the body from the request
const bodyParser = ctx => {
  ctx.req.body = parsebody(ctx.req);
};

Asynchronous: a function that gets the context and returns a promise that will be resolved (or rejected) when the processing finishes.

// Using a Promise-aware library, more and more everyday
const auth = ctx => findUser(ctx.req.session.id).then(user => {
  ctx.req.user = user;
});

// Porting an existing callback-based library
const ported = ctx => new Promise((resolve, reject) => {
  doSomethingAsync({}, (err, cb) => {
    if (err) reject(err);
    resolve(cb);
  });
});

This was a “what would the best way for creating middleware with ES6 from scratch?” kind of idea. I still needed to check if this makes sense at all.

Joining middleware

For testing I made a function that joins them. I got bitten with express when I tried to do the same (100% my fault for not learning how it worked better) but I was surprised at how easy it was when using promises.

The objective of join is that this code works as expected:

// my-custom-middleware.js
const join = require('./join');

const parseBody = ctx => {
  ctx.req.body = bodyparser(ctx.req);
};

const auth = ctx => findUser(ctx.req.session.user).then(user => {
  ctx.req.user = user;
});

// Return a single middleware that chains them
module.exports = join(parseBody, auth);

This join can be defined with just native code:

// join.js
// Pass a group of middleware and return a single one
module.exports = (...mid) => ctx => mid.reduce((prev, next) => {

  // Pass always the original context; not the returned one
  return prev.then(next).then(fake => ctx);

// Get it started with the right context
}, Promise.resolve(ctx));

We should totally do loadware(mid).reduce... with loadware to flatten the array, but I didn't want to add this dependency for the example.

That’s it. Now you get a single middleware from those two by just using native code and Promise-aware middleware.

Compatibility with Express/Connect

I also made a function to make any of the original express middleware work with server (making it promise-aware). It is quite easy to use:

// compress.js
const compress = require('compression')({ /* opts */ });
const modern = require('./modern');

module.exports = modern(compress);

To allow for dynamic options is slightly more complex:

// body-parser.js
const compress = require('compression');
const modern = require('./modern');

module.exports = ctx => {
  const middleware = compress(ctx.options.compress);
  return modern(middleware)(ctx);
};

Problems

The main problem is that a promise chain cannot be stopped in a non-error way so it’s not possible to easily halt the call even when a router has successfully been called.

We are mostly ignoring the return value and I think promises require to be resolved or rejected to avoid memory leaks, so leaving an unsolved promise also doesn’t help.

This leads to the first constrain:

All the routers will be final; only the first router that matches a path will be used and the rest are ignored:

server(
  get('/', ctx => { /* called  */ }),
  get('/', ctx => { /* ignored */ }),
  get('/', ctx => { /* ignored */ }),
);

The second but also important problem is handling errors. All of the middleware accepts a single parameter, ctx, so it cannot be analyzed to see which one is supposed to handle errors.

This leads to the second constrain:

Only throw to the middleware chain when there is a truly catastrophic global issue; try to resolve the local issues within your middleware:

server(
  ctx => {
    // Recoverable error
    // NO: throw new Error('No nnn active. Halt it all!');
    // Yes:
    ctx.log('No nnn active. To use it make sure ...');
  },
);

Conclusion

This is still alpha software, but so far every indication is that we can transition to an awesome Promise-aware world while easily bringing all of the ES5-world code. See more about the project in Github.

Server’s middleware ♥ Promises was originally published in Server for Node.js on Medium, where people are continuing the conversation by highlighting and responding to this story.