LYRID H1 2021 — Aurora v1.1 Release

Hello everyone!

We are dropping some exciting news that we have been brewing for a couple of months now to bring you our next evolution of the Lyrid Platform as we make it even more accessible for everyone to get their codes built, deployed, and running serverless applications.

Right after our first Medium post back in December, we have gained many early users and feedbacks that allow us to focus on what we should build for our next iteration. We are truly grateful to our early believers and we made sure that they are heard. And our goal has always been the same: anything we do is to make it easy and simple for any developers that want to deploy their applications on the cloud as serverless applications.

Some of these updates have been pushed out at the end of March 2021, and we have been ironing out the release, getting more feedback, as well as getting our partners ready to use these new features for their own solutions.

With that, let us dive into the updates!

Lyrid Execution Domain (*.lyr.id)

If I have to pick which of the feature is the most important, I will have to say this one. In this update, we created a new domain (lyr.id) for our end-users to access their applications in the public domain. This shapes how we serve the serverless function HTTP endpoint. We put a wildcard rule and certificate and created a new HTTP Proxy Service that maps a randomly generated string as a subdomain name into the serverless deployment on any public cloud that we support. Here is how the flow looks like:

This is a publicly generated endpoint that is accessible by default, every single submission will generate these URLs. Your applications will be instantly available to the end-users without the need to configure any virtual machine, container instance, or network firewalls. We host this endpoint *.lyr.id with Geo-distributed DNS on multiple clouds.

There's an option to turn this off during submission if you want to keep the endpoint hidden. Check out our documentation for further information about this: https://docs.lyrid.io

Custom Domain DNS Support

A custom domain can be mapped to the execution subdomain name. All the user has to do is to create a DNS record (either CNAME or A record) that maps their application DNS name into the subdomain name.

Once the name is mapped, at the first call, the platform automates the certificate request with LetsEncrypt CertBot APIs:

The issued certificate is stored internally within the platform and we apply account level at rest encryption to save and distribute these certificates within our platform.

In the future, we will support users to be able to upload their own certificates, but if you do require those capabilities, please let us know and we will be happy to support your own certificates.

Additional Support for Node.JS Server-Side Rendered (SSR) Platforms

Another thing that we have been working on over the past few months is adding new support for a Single-page Application (SPA) that is built using Server-Side Rendered (SSR) frameworks.

The rise and success of these frameworks for single-page applications is a natural progression to building web-native applications. Creating a scalable web application that has a high performance and search-engine-optimized is no longer something that requires long development cycles. And we definitely see the value of supporting these frameworks natively in our platform.

Next.JS

We choose first to support Next.JS inside our platform:

Next.js by Vercel - The React Framework

Next.js is an open-source React front-end development web framework created by Vercel that enables functionality such as server-side rendering and generating static websites for React-based web applications.

Supporting SSR frameworks require us to modify our Node.JS workflow to support a chain of commands to builds the SSR assets and saves them to be served inside an Express.JS application.

We did that following update into our own landing page lyrid.io:

Once we knew we can support one SSR framework, we tested the new workflow against all these frameworks and most of them can be naturally supported in our new SSR workflow such as:

Nuxt.JS

The Intuitive Vue Framework

Gastby.JS

The Best React-Based Framework | Gatsby

We will write guides as well as tutorials for these, but in the meantime, check into our updated documentation on how to get your Next, Nuxt, Gatsby project running inside our platform as Lyrid Serverless Application.

Managed Environment Variables Supports

Within this period, we also introduced a way to securely set environment variables in the platform. This allows our users to not rely on a file to set their environment variables. Environment variables can be injected into the platform using our API and Web:

And it will be set at the deployment time of application into the appropriate platform. Users will also able to override values for the environment for each deployment.

AWS Lambda Defaults to Container Deployment

Back in December 2020, Amazon Web Services Lambda announces Container Image is publicly available:

New for AWS Lambda - Container Image Support | Amazon Web Services

We decided to take full advantage of this as soon as we can spend some engineering cycle. We were quite excited about this because building containers not only allows us to build a larger package (we hit the 50MB upload limit in a lot of instances), it also streamlines our build and deployment process as it fits into our workflow pipeline.

UI/UX Improvements

Along with environment and policy management for applications, we also implemented various improvements in the application user interface to improve the serverless application experience.

App Overview

A new application overview that shows the detail of a serverless application. From the size of the submission to the size of the builds for each platform to the details on where it is being deployed.

Execution Domain Management

We also added a user interface for the user to add and attach their custom domain and previews their application:

Console View

We brought our console monitoring into the UI to be able to do monitor the console output of their applications:

Code View

And lastly, in our new code viewer, users will be able to check the running revision of the submission.

Tighter Deployment and Various Infrastructure Improvements

As for updates in our backend infrastructures:

• We updated our infrastructure deployment codes.
• Integrate and repacked our modules into our own Serverless Application.
• Utilize more of the Kubernetes Cluster automatic scaling in our backend platform.

Here’s the anatomy of how Lyrid is managing a Kubernetes cluster with our services installed inside them.

The tighter integrations into the services resulted in a massive performance increase for our

As shown in our latency metrics, we successfully brought back our services with reduced jittery-ness and improved the latency for almost every call into the platform upward of 80%.

These tighter Kubernetes integrations are a sneak peek into our ability to bring Lyrid on-premise solutions a step closer into your data center and harnessing the same capabilities of what we produced in the SaaS that is available at the moment.

Conclusion

The past few months have been quite a busy time for us and we have been expanding our reach and our partner networks to constantly improving the platform. Here are some of the focus for the product that we identified for the near future:

• Improving our onboarding and embedding tutorials
• Support for submission using Git URLs
• Native CRON Scheduling Event

We strive to be the most fluid and agnostic platform to deliver any applications. There’s so much potential of where we can go from here. And with that, you can register a Free Lyrid account to get started with the following link and try these all yourself:

Register Here To Start Lyrid

Lastly, please do not be shy to contact me at hsutanto@lyrid.io or join our Slack channel to say hi. Thank you for reading, stay tuned for more, updates, tutorials, and how-to guides!

Handoyo

LYRID H1 2021 — Aurora v1.1 Release was originally published in Lyrid on Medium, where people are continuing the conversation by highlighting and responding to this story.

In most cases, when building API for your services, you will need a way to verify if the user is properly authenticated to read the data that belongs to the user. The calls will need to be authenticated and the identity of the caller needs to be passed down to the service to make the appropriate database calls that only show relevant data to that user.

In this tutorial, we will continue our Lyrid Serverless Function builds from our previous NodeJS post to build an API endpoint that is protected, able to identify its caller user identity, and only update fields that belong to this user.

For this post, I am crediting these 2 blog posts as I am creating a remix of these 2 posts to show how to create a login, along with the protected routes that do CRUD (Create-Read-Update-Delete) API using standard NodeJS Express and to publish them into Lyrid Function:

1. Tutorial for creating NodeJS protected routes using JWT by Jason Watmore:

https://jasonwatmore.com/post/2018/08/06/nodejs-jwt-authentication-tutorial-with-example-api

NodeJS - JWT Authentication Tutorial with Example API

2. Tutorial for NodeJS CRUD API using PostgesSQL by Geshan Manandhar:

https://geshan.com.np/blog/2021/01/nodejs-postgresql-tutorial/

Node.js Postgresql tutorial: Build a simple REST API with Express step-by-step

These 2 blog posts are an excellent starter to grasp the concepts that are usually required to understand how we build API services that know the “user” and its scope on how to read/write/update his record in the backend database. The full code for this is hosted on this GitHub page:

https://github.com/LyridInc/lyrid-jwt-crud-expressjs.git

What is JWT and Why?

In an effort to make this post shorter, I will not dive too deep into how JWT works. In short, it describes the flow of authentication and authorization in HTTP calls, example are as follows:

Modern Token Authentication in Node with Express

JWT is a 2 step authentication and authorization process that uses standard HTTP Request and Response to grant a user access into restricted API calls. The steps are as follows:

1. Authentication:
   Usually requires the user to pass in username and password (in some systems it can be a verification token from email or SMS) into an authentication endpoint of the service.
   This authentication endpoint will check for the credentials and grant access by creating an authorization token in form of a signed JWT that is generated by a private key in form of a secret string or certificate. Anyone can read the JWT information but only systems that have the secret string or certificate can verify these tokens.
   Internally, the JWT can contain information regarding the scope of the API call (user id, email address, etc), along with token issue time and expiration time. A shorter expiration time is usually better for security, but it does introduce more calls into the authentication endpoint.
2. Authorization:
   During any authenticated API call, the caller will need to pass authorization in the form of an Authorization Bearer token header. This contains the JWT that is generated on the first step.
   The API service will need to share the same secret or certificate that allows the API server to verify the token without the need to communicate with the authentication service. This way, the REST API will actually carry an identity without containing any information about the credential of the user.

By decoupling the Authentication and Authorization step, developers will then have the ability to also decouple their APIs with their user authentication and management that has major advantages like:

https://dzone.com/articles/jwtjson-web-tokens-are-better-than-session-cookies

No Session to Manage (stateless): The JWT is a self contained token which has authetication information, expire time information, and other user defined claims digitally signed.

Portable: A single token can be used with multiple backends.

No Cookies Required, So It’s Very Mobile Friendly

Good Performance: It reduces the network round trip time.

Decoupled/Decentralized: The token can be generated anywhere. Authentication can happen on the resource server, or easily separated into its own server.

JWT is one of the techniques that is commonly used in HTTP authorization and there are other tutorials there that explain JWT on how it is being used. Here are some additional information on the topic:

JSON Web Tokens

We want to show how these standard techniques can also be applied to the serverless function that you build in the Lyrid platform.

Prerequisites

• A laptop or PC (Mac/Windows/Ubuntu don’t really matter)
• NodeJS 12.x or above installed: https://nodejs.org/en/download/
• npm: https://www.npmjs.com/get-npm
• Lyrid account: you can register here for free, no credit card required, along with Lyrid CLI downloaded.
• Git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
• Any of your favorite IDE, I use Visual Studio Code in this example.
• Some HTTP testing tool (I will use Insomnia in this example, but either one works):
  Postman: https://www.postman.com/downloads/ or
  Insomnia: https://insomnia.rest/download/
• PostgreSQL Database: Using a Postgres-as-a-Service database is the easiest and quickest way to get started. As the post from Geshan suggested, I will also create a small free instance over at ElephantSQL: https://www.elephantsql.com/

Setup

First, let's clone our repository and run them:

git clone https://github.com/LyridInc/lyrid-jwt-crud-expressjs.git
cd lyrid-jwt-crud-expressjs
npm install

The next step is that we need to inject seed data into this SQL for testing the application. You can do that by pasting the data inside db/init.sql into the SQL browser of ElephantSQL and click on execute:

For those who can’t read SQL queries, all this script does is create a new table and inserts data into the database.

If you read and follow Geshan’s post, you will notice that the original SQL query from the original post does not have user-id information, and this modified query has an ID of 1 or 2 in the database that represents the user ID.

The next step is to copy the connection string into the environment file. Open your .env file and put your PostgreSQL URL. You will also need to generate a string that will be used to sign the JWT, you can use a randomly generated string of 32–64 characters. The final environment file looks something like this:

Now you can run your REST API to test them by calling:

npm start

Now that it is running, we dive deeper into the additional middlewares and routes from our previous post. We added a few additional configurations in our src/entry/entry.js that does the following:

• Configures JWT middleware to secures the API with rules.
• Create login endpoint.
• Create protected CRUD endpoints.

Let’s dive deeper into this middleware and endpoint.

Authentication Endpoint

Inspired by the blog post from Jason @ https://jasonwatmore.com/, we created the middleware that handles POST request with the path of /login as follows:

The login will take in the username and password as a body of the request, and create a JWT (using a library called jsonwebtoken) that has the user ID that expires in 7 days. This response is then signed by the secret string that is inside our environment.

Configuring Protected/Unprotected Routes

After coding the login endpoint, we need to configure our Express application to be able to handle JWT bearer tokens in the header by adding a middleware library called express-jwt.

This middleware allows every single request to be authenticated, and we are able to configure the endpoints that don’t need to be authenticated as follows:

The login endpoint will not be authenticated and for backward compatibility, we will include the original routes from our first blog post to be unprotected.

Protected Routes and CRUD API

Since these calls are protected by JWT middleware, all the internal APIs have information about the user context, and the internal SQL queries can be generated using that context.

Here are the resulting protected endpoints:

GET /quotes — Get a page that contains quotes for the following user. It employs pagination as a technique to keep response small and faster user experience.

POST /quotes — Inserts a quote for the following user with data from the request body.

PATCH /quotes/:quoteId — Updates an existing quote with id with data from the request body.

DELETE /quotes/:quoteId — Deletes a quote with id.

Internally, the update and delete verify the id and only does the operation if the user id is the same as the id in the database.

Digging deeper into one of the quotes will show the SQL commands that are generated. For example, opening the src/protected/service.js, and looking at the code flow, validating the input, and injects the parameters into a SQL query, and passed the query into the db.query() function:

Where the db.query() code is as follows:

It has a connection to the database and this is where the SQL queries are executed against the PostgreSQL URL that is configured early in the tutorial.

That’s it, we are ready to run and test this out.

Test and Deployment

Let’s test this login with the same username and password that is hardcoded into src/users/list.js,

{

 "username": "user1",
 "password": "password"

}

Your response will be like this:

The JWT token to authorize the subsequent requests will be in the response as shown above.

Pro Tip: Anyone can read the content of your JWT using any tools out there (example: https://jwt.io/) as it is a standard. Only the servers that have the secret strings can verify if this token is actually valid, so do not share your secret key with any system that is not authorized to do verify the token.

Run GET /quotes for the next request, but you will need to copy and paste the token generated into the Bearer token Authorization:

As you can see, the API request will only return the quotes that belong to the user that is associated with the user.

To insert new quotes data for the following user, call the following endpoint with the same token:

POST /quotes —

{

 "quote": "Where is the beef?",
 "author": "Handoyo Sutanto"

}

As you insert them, you can call GET /quotes again to see the quote that you inserted.

Now we tested the new endpoints, let's submit this function into Lyrid platform by calling:

lc code submit

Passing Authorization Header in Lyrid API Gateway

This part of the tutorial is specific to Lyrid and the Universal API Gateway implementation.

Now that your function is up there in our platform, how do you pass the Authorization Bearer token? As you recall from our previous posts, the Authentication header is already used by our platform to identify the execution against the Lyrid account.

For this purpose, we developed an additional header that our users can use to pass the Authorization into their serverless function. X-Lyrid-Authorization is a Lyrid specific header that will be handled by the platform. The platform will pass down the value of this header as an Authorization header.

Set the X-Lyrid-Authorization Header as following (remember to include the Bearer string in there):

We are currently working on creating a unique execution endpoint that works on a top subdomain of Lyrid that doesn’t require you to pass Basic Authentication header.

Conclusion

Obviously, we won’t use a hardcoded username and password in production. Since the authentication service is decoupled, there are options to do a managed authentication services (example: Okta, Auth0, or AWS Cognito). These services can help developers to authenticate their username and password in a separate secured environment and generates JWT without the need to host in their own environment.

This allows developers to offload the user and password management and focus more on the API that they are building. As shown in this tutorial, we support the usage of any external authentication services into your Lyrid Serverless Function using this method.

As for Lyrid, we utilize JWT heavily inside our connected services that drive all the function build, execution, and deployment. We have developed a modification of this open-source GraphQL Authentication :

Django GraphQL Auth

This enables us to do our own self-hosted authentication management that does the following:

• Python 3.8, Django 3.0.5 application on Lyrid Serverless platform.
• Uses GraphQL endpoint and mutations for authentication API.
• Connects to almost any database, e.g.: PostgreSQL, MySQL, or even MongoDB.
• Manages user ID, logins, and tokens.
• Connects user ID to other Single-Sign-On services to enable login with external login services (GitHub, Facebook, Google).
• 2-Factor-Authentication plugin.
• Configuring communication settings (Email or SMS).
• Administrator dashboard to be able to reset any user’s password.

All of them, packed into the Lyrid Serverless application. We will show you how to instantiate, run, and use this service easily inside your Lyrid Account. This is for our future blogs, but please don’t be shy to contact me hsutanto@lyrid.io or join our Slack channel to say hi if you want to learn more about this!

Thank you for reading!🐱‍🏍Stay tuned in for more tutorials!

Handoyo

Securing CRUD API With JSON Web Token (JWT) On Express and Lyrid Serverless Function was originally published in Lyrid on Medium, where people are continuing the conversation by highlighting and responding to this story.

Let’s continue from our previous tutorial by building a front-end web application from what we learned. We went through how to build a serverless NodeJS 12.x Express application backend that utilizes Sharp image processor:

https://medium.com/lyrid/building-and-distributing-nodejs-12-x-express-serverless-application-with-lyrid-5ed18345b122

In this tutorial, we will use GitHub’s awesome feature that helps web application developers to host their static page applications onto a working website for free. Just like our services, no credit card is required to host your static page.

We will show you how to build a simple React SPA that calls into the API that was built in the previous tutorial.

Prerequisites

• Lyrid Account: Register Here To Start Lyrid
• Go through our previous post: https://medium.com/lyrid/building-and-distributing-nodejs-12-x-express-serverless-application-with-lyrid-5ed18345b122
• GitHub Account: https://github.com/join

Create an Execute Only Access

First thing to do is to create an execute only access and secret key. Why execute only access?

The reason is because we will be hosting the access key and secret on our front end, and the credential will be exposed by anyone that are able to open your front-end application. The execute only access keys will only be able to execute function inside your account, and this access key will not be able to do any administrative functionality.

To do this, go to your account settings page, and click on the IAM tabs: https://app.lyrid.io/app/settings/account

Give the key some name, and make sure the restrict access flag is checked, and then click on Generate Keys.

Store the access key and secret someplace, and we will use them on the later part of the tutorial.

Setup GitHub Pages

For this tutorial, we have setup a GitHub repo template that can help you bootstrap the project faster:

https://github.com/LyridInc/react-staticpage

Click on Use this template button. And fill the appropriate form:

When creating a new repository with this template, please ensure that you check that “Include all branches” settings. This will copy the branch that is hosting the static files as well.

Now that you have copied this template, you will need to clone them into your local machine:

git clone https://github.com/{GitHub-Username}/{GitHub-ProjectName}.git
cd {GitHub-ProjectName}

And there’s a few things your will need to change in this template:

• .env — This is where you will be putting your execute only access and secret key.

REACT_APP_LYRID_EXECUTE_KEY=<RestrictedAccessKey>
REACT_APP_LYRID_EXECUTE_SECRET=<RestrictedSecretKey>

• package.json — Change the homepage settings URL at the very top. The format for the URL is as follows:

"homepage": "https://{GitHub-Username}.github.io/{GitHub-ProjectName}/",

In our case, it will be:

https://lyridinc.github.io/react-staticpage/

• src/helpers/api.js — (Optional) This is where you put Lyrid API endpoint URL if you changed any names in the previous tutorial.

Now, lets test and run this locally by running:

npm install
npm start

React App Template

Now, let’s explain a little bit about this react application template that you just copied into your GitHub account. This template is created using create-react-app @ https://create-react-app.dev/, and modified to host our components to drive the API calls.

There are some libraries that we want to mention for this front-end:

• Axios Instance Routing

At src/helpers/api.js, we use this library called axios (https://github.com/axios/axios) to call into our API that we developed on the previous post.

During development, you can change the axios instance to call into your localhost by commenting out the production instance and use the localhost instance instead:

• React Hooks

Most of the components for the UI are built using the newer way of building react functional components with React Hooks (useState and useEffect). Which allows a more flexible and easier to understand code.

We created simple controls that loads an image URL and binds user’s input into the query parameter requests of the backend API. We then show the output of API which is the processed image as the user changes the input.

More about React Hooks here: https://reactjs.org/docs/hooks-intro.html

Modifying CORS for Server Side

Because we will be running API calls from a browser hosted on a cross-site resource, we will be requiring to setup Cross-Origin Resource Sharing (CORS). This allows your front-end to pass authenticated messages using the browser into your serverless function call.

Cross-Origin Resource Sharing (CORS) is an HTTP-header based mechanism that allows a server to indicate any other origins (domain, scheme, or port) than its own from which a browser should permit loading of resources.

CORS is a server side settings, and we will need to make these changes on the backend project. Open the src/entry/entry.js file and add these following code snippet and change the Origin URL appropriately to your hosted GitHub Pages URL:

var cors = require('cors')

var corsOptions = {

origin: [ 'http://localhost:5000', 'https://{GitHub-Username}.github.io' ],

  credentials: true,

}

app.use(cors(corsOptions))

Here is how our CORS settings looks like:

After making the appropriate CORS changes and run the code submit again to update the server side API:

lc code submit

Deploy the Static Page

Now that we setup all the required codes and backend, we can deploy the react static pages using this following command on the react project:

npm run deploy

You may be prompted with GitHub credential login to authenticate if you have access to check into your repository. Just fill in your GitHub username and password and viola! Once the deployment is complete, you can access your web application at:

https://{GitHub-Username}.github.io/{GitHub-ProjectName}/

That’s it! You have just hosted a full web application that has a React front-end and ExpressJS Serverless back-end! Check out the output GitHub page that we deployed for this tutorial:

https://lyridinc.github.io/react-staticpage/

And with that, you can start hacking away and let us know what you are building with our platform!

Conclusion

This tutorial translates to any static page hosting service that you can think of. We like GitHub Pages just because it is free and straight-forward, and there are more features available in GitHub Pages that we have not explored in this tutorial: https://pages.github.com/

We would like to also add that we also have support for our own domain support for static pages with the ability to add your own custom domain into your functions or web servers. We are still working on the release for this feature, but if you like us to help you set up a custom domain, just let us know!

And with that, you can register a Free Lyrid account to get started with this following link and try these all yourself:

Register Here To Start Lyrid

Lastly, please do not be shy to contact me at hsutanto@lyrid.io or join our Slack channel to say hi and let us know if there are any issues for you following this example.

Thank you for reading, stay tuned for more tutorial and how-to guides!

Merry Christmas and Happy New Year! Stay safe!

Handoyo

Driving Lyrid Function Execution From a React Single Page Application (SPA) Hosted in Github Pages was originally published in Lyrid on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

This is a series of tutorials that I will start building mainly focusing on showcasing how things work and how to build an application using Lyrid. This example specifically will show you how to deliver a standard NodeJS 12.x running Express Web Server as a Lyrid application.

Although these blogs are about our own technology, you will soon see that there is nothing in this platform that requires you to conform with a non-standard method of building an application. Everything you do here can still carry forward to writing an application on your own Virtual Machine. For instance, Docker Container, or Kubernetes.

Prerequisites

• A laptop or PC (Mac/Windows/Ubuntu don’t really matter)
• NodeJS 12.x or above installed: https://nodejs.org/en/download/
• npm: https://www.npmjs.com/get-npm
• Lyrid account: you can register here for free, no credit card required, along with Lyrid CLI downloaded.
• Git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
• Any of your favorite IDE, I use Visual Studio Code in this example.
• Some HTTP testing tool (I will use Insomnia in this example, but either one works):
  Postman: https://www.postman.com/downloads/ or
  Insomnia: https://insomnia.rest/download/

Initial Setup

First you will need to register your Lyrid account:

Register Here To Start Lyrid

And go through the 3 sections:

• Registration: https://docs.lyrid.io/docs/en/registration/
• Installation: https://docs.lyrid.io/docs/en/installation/
• Client Initialization: https://docs.lyrid.io/docs/en/initialization/

This whole guide assumes that you already registered with Lyrid, downloaded our command line client, and initialized the client with your access and secret key.

You will then need to clone the base project. For this example, it is hosted at GitHub:

git clone https://github.com/LyridInc/lyrid-expressjs.git

Local Build and Test

To run the sample locally, simply run these commands:

cd lyrid-expressjs
npm install
npm start

Simple…so far so good…lets dig into what we have in the code. This sample REST API is quite simple:

It contains a basic index.js where it will initialize an express web server application instance, and the function entry point (entry.js) will define how this instance is built along the routes. There’s a logging middleware (morgan: http://expressjs.com/en/resources/middleware/morgan.html) that logs every request for this application.

This service has two routes:

/echo/* : This endpoint will just echo all your request url, and method then prints it out as a text response. Test it with Insomnia by passing HTTP POST Method and /echo/hello_world request:

Looks good!

/sharp/ : We will get into this endpoint in the later portion of the tutorial.

Lyrid Definition File and Entry Point

Let’s explain a bit about this GitHub project example. The structure of the code is similar to our initial code when you run:

lc code init --name "$AppName" --module "$ModuleName" --function "$FunctionName" --language "nodejs12.x" --web "express"

Except in this case we already prefilled all those names and description.

A few things to explain in this template is the .lyrid-definition.yml file, and inside it there is an entry point called entry.js

Changing this definition file is not required, but it is crucial to explain how this file is being used by our platform.

The definition file contains descriptors that will be used in our platform to identify how the function will be built and packed. In this scenario, because it is an Express Project, our build will expect an express server instance application to be returned from the entry point (entry.js). The user has the flexibility to modify the entry point to include more routes and middleware inside the project, and our platform will take in the express server instance and wrap the instance for different types of serverless platforms.

The definition file determines how your function will be called inside our API Gateway.

NOTES: This is specific for our NodeJS 12.x Support. Our current support for NodeJS and Express is only for non-asynchronous function execution, and continues until the event loop is empty or the function times out. We will have support for asynchronous Promise based type of wrapper soon.

Public Cloud Deployment

Now your code is working locally, the next question is: how to make this API run on a publicly accessible endpoint?

Run our submission command from the folder that contains our definition file:

lc code submit

In this stage, your function will be zipped, and wrapped by our platform and the platform will perform a build to create a build artifact for all the public cloud serverless platforms that we support.

At the end of the submission, save the endpoint generated by the platform.

In this stage, we went through a standard serverless function build. This runs the pre-built installation using npm install and packages the function as an application.

Run the following command to monitor your build output:

lc monitor -c '*.build.*'

When your build is deployed to your default execution platform, your function is published and ready to use.

Public Cloud Execution

Now that your application is deployed, execute it using our endpoint. Remember the structure of our endpoint is currently:

This turns your endpoint to (if you didn’t change any names in the lyrid definition): https://api.lyrid.io/x/node/express/latest/entry/

When testing with HTTP client like Insomnia, remember to change the authorization to Basic on this tab and fill in your Lyrid access key as username and Lyrid secret key as the password:

Let’s execute this by clicking on the Send button:

Looks good! The app is now hosted inside our platform.

Similarly, you can also test out calling the application using cURL command like this (on Mac or Linux):

curl -u $LYRID_ACCESS:$LYRID_SECRET https://api.lyrid.io/x/node/express/latest/entry/echo/hello_world

Where $LYRID_ACCESS and $LYRID_SECRET corresponds to your access key and secret key.

You can monitor your LYR execution logs using our lc monitor function and filter for all LYR execution in your account:

lc monitor -c '*.execute.LYR'

Express Sharp Endpoint

The other endpoint that we have in this example is our implementation of an image processing that uses a high performance Node.js image processing called Sharp: https://sharp.pixelplumbing.com/

We take inspiration from this GitHub page: https://github.com/pmb0/express-sharp/tree/3.1.1

And made little modifications to the endpoint to be able to support any URL given to the function. The URL format is as follows:

/resize/:width/:height?format=$format&progressive=true&quality=90&crop=false&gravity=center&url=$image_url

Where:

• width: integer, the fixed width of the output image.
• height (optional): integer, the fixed height of the output image. If not provided, it will maintain the aspect ratio of the image.
• format (optional): string enum, valid values are one of the format supported by sharp: https://sharp.pixelplumbing.com/api-output#toformat. Default: webp if supported else, the output format of the requested image.
• progressive (optional): boolean, only for jpeg and png. Use &progressive=true to enable progressive scan.
• quality (optional): integer (1–100). quality is a Number between 1 and 100. Default if 80.
• crop (optional): boolean. Use &crop=true to enable the sharp cropping feature. Default is false. Note: Both width and height params are neccessary for crop to work.
• gravity(optional): string enum. When the crop option is activated you can specify the gravity of the cropping. Possible attributes of the optional gravity are north, northeast, east, southeast, south, southwest, west, northwest, center and center. Default is center;
• url: string. URL to original image.

Lets test this function locally and look for a publicly accessible image URL at Imgur. I will use this one: https://i.imgur.com/3a0qwRe.jpeg

The original image has resolution of 701x701 pixel and 56KB in size.

Lets run though this image with a few different parameters locally first with base URL of http://localhost:3000/sharp and HTTP method of GET in your Insomnia Client:

GET /resize/250/150?crop=true&gravity=north&url=https://i.imgur.com/3a0qwRe.jpeg

GET /resize/500?format=webp&quality=25&url=https://i.imgur.com/3a0qwRe.jpeg

The cropped and resized imaged above is much less than the original image (4.9KB and 6.3KB). Change the endpoint with base URL to http://api.lyrid.io/x/node/express/latest/entry/sharp and add your Access Key and Secret just like the previous section, and test it again. You just hosted an image processing service that you can use to dynamically resize, crop and convert any publicly accessible image.

Sharp is a very complex image processing library, and unfortunately we will not dive further into the what it can/cannot do. We just want to show you in this blog that we are able to integrate complex libraries and package them into a simple API using Lyrid.

Conclusion

In this tutorial, we want to show a basic use-case of our platform. Hopefully, this has given something that can be easily followed and to show how our platform works. More of these upcoming tutorial series will touch on examples that:

• Connects to a Database
• Connects to a Storage
• More Middlewares: CORS, JWT Authentication, etc2.
• Seamlessly moves your application to another public cloud (AWS or GCP) while maintaining the same endpoint.

Notice in this whole process, apart from our command line client, you did not even touch or download any library, framework, or tool to make this work on a different cloud provider. This follows our philosophy of “Only Focus on Your Code — BUT It Still Should Be Able to Run Anywhere”. We are abstracting anything that is not related to your code (the build, distribution and execution).

Along with NodeJS 12.x + ExpressJS, the same flow can be done with our support for:

• Django + Python 3.8
• Flask + Python 3.8
• Gin + Golang 1.x
• ASP.NET + .NET Core 3.1
• And more upcoming support

And with that, you can register a Free Lyrid account to get started with this following link and try these all yourself:

Register Here To Start Lyrid

Lastly, please do not be shy to contact me at hsutanto@lyrid.io or join our Slack channel to say hi and let us know if there are any issues for you following this example.

Thank you for reading and stay safe!

Handoyo

Building and Distributing NodeJS 12.x Express Serverless Application With Lyrid was originally published in Lyrid on Medium, where people are continuing the conversation by highlighting and responding to this story.

Let me start first with who we are and what we do.

Hi!

My name is Handoyo Sutanto and I am the co-founder and creator of Lyrid. Today has only been the 2nd week of December of 2020 and there are already so many announcements from different aspects of our lives. Ranging from COVID-19 vaccine news, to AirBnb, and Doordash IPOs. But on top of all the big news, we have bigger news of our own! This month is where we decided to publicly start allowing access to our platform without any restrictions!

Yes, really! You can register to our platform and deploy your own serverless applications in less than 5 minutes without any credit card requirements!

Register Here To Start Lyrid

Firstly…What is Lyrid?

We are a collection of tools and frameworks that are put together as a SaaS platform. You will be able to use these to help build your cloud-native serverless application that can run on multiple, serverless public cloud platforms (such as AWS Lambda or Google Cloud Run).

Lyrid is first and foremost a platform provider for anyone to run their application servers. We create a seamless experience for anyone to develop their idea as efficient and frictionless as possible without compromising security, inter-operationality, and scalability. Multi-cloud serverless compute is one of the tools to get there.

We perform all the build and deployment process for different cloud platforms for our users. So, they can just focus on what is important for them, which is, their business. We mean it when we say, they should just focus on their business and operations, not figuring out things like:

• Which cloud platform to use?
• Where they need to deploy their services?
• How much resource they will need? CPU? RAM? How many instances? And where?

We achieve this by managing and driving your cloud build, deployment and executions using policies, and building more intelligence into the platform on how/where/when your application should be built, distributed and executed. We also deploy and execute your functions using our Universal API Gateway.

Understanding The Motivation

Prior to Lyrid, my background for the past 15 years was working on enterprise infrastructure and distributed system software. The workload varied from software that distributes media, transcoding jobs to building scalable storage, and, compute and networking subsystem using various technologies.

I love experimenting with new technologies, but the more I wrote and deployed my code in the cloud, the more I started to realize something.

Reducing Repetition and Guesswork

That is…most of the things we (Cloud Engineers and DevOps) do is actually repetition. We like to think that what we do is so revolutionary, but in most cases are just a combination of your muscle memory (repetition) and guesswork. The challenge of knowing exactly what kind of resources and the amount needed at a given point in time is an ongoing battle for us. There are tons of tools/analytics/monitoring solution software that revolves around these problems.

Here’s what you feel like doing when you guess wrong:

We are not condoning property destruction, but boy I can really relate to them!

I am sure we can do better than this…It was about 4 years ago that I got my first exposure to serverless by AWS Lambda, and was immediately drawn into the technology. Especially, on how it can remove almost all of the guesswork (at least on the question of “how many instances do I need?”). The baked-in efficiency that is inherited from pay-per-execution cost structure is just something that I had never seen before!

Clearing Misconceptions about Serverless Technology

However, the benefits that I mentioned above does comes with a cost, almost every managed serverless platform out there (AWS Lambda, Google Cloud Function, Azure Function, etc.) needs to confirm with the cloud’s specifications on how to “run” your function. Case and point, check out these different entry points for your function input/output that the user has to put together in order to make their code work inside the platform:

And because of that, serverless is probably the most misunderstood technology in the past 4 years. The amount of conflicting information floating around the internet is toxic for the growth of the technology itself. Here are some of the information titbits that I found from my personal experience with some of my colleagues that resist deploying a serverless solution:

• How tied it is to the platform dependency. Which means that you can only fully benefit from serverless only if you put all your workload in the same platform.
• Which leads to: investing in making something into serverless will pretty much lock you into the platform further.
• Along with non-upfront and complex pricing calculations. The scale to zero and pay as you scale is actually a double-edged sword to the adoption. There is actually some comfort about knowing what the upfront cost is. Check out this recent blog by written by Sudeep at Milkie Way, Inc. on how it can get out of control: https://medium.com/milkie-way/we-burnt-72k-testing-firebase-cloud-run-and-almost-went-bankrupt-part-1-703bb3052fae
• And lastly, there is still no clear understanding on what can or cannot run as serverless.

And here are some examples of the conflicting information about serverless that confuses most people:

• Why do I need to learn about the platform? Isn’t the point of serverless is to focus on your business logic?
• Why do I really still have to know how big I should scale my machine? Isn’t the platform supposed to take care of that?
• Why do I really still have to know where it is deployed? Isn’t the platform supposed to take care of that as well?

One more misconception about the technology is an association that people make. We also want to create this new way of thinking that AWS Lambda != Serverless. AWS Lambda is a compute platform that runs a serverless workload.

They were the one that coined that term so a lot of people associate them together, and this association is a hinderance to the adoption.

We will get into more detail about what we think to be the common misconceptions about serverless on the next upcoming blogs.

Don’t get us wrong…we are actually strong believer in serverless! And we believe everyone should write more of them! So I had a vision of what serverless should be. With that I start this company, and our views are that serverless is a piece of technology and a philosophy on how a developer should build an application:

• Never have any infrastructure to manage from the end-user side. No infrastructure is needed to build and distribute your application. Not even containers are required. Just build your code on your machine.
• Standardized input/output interfaces regardless where it is run.
• Universally accessible tools, platforms and frameworks that can be used and scaled by anyone from college students, scrappy start-ups, to Fortune 500 companies.
• Focus on your code! Optimize there! Really…and not in how it is being built, distributed, deployed, and executed. Those should be the platform’s job to optimize.
• The platform itself should self-expand, shrink, and auto-load balance with geographical awareness in the background with the ability to scale to zero if there are no requests. Thus, further reducing any guess work.
• Privacy and security needs to be baked into the platform and users will be able to harden their deployment as much as they need to. The platform should always facilitate this without getting in the way.

Building Platform For Everyone

This is pretty much the driving force that makes us wants to create Lyrid.

At the end, what we want to create is a platform for everyone to test and build their ideas rapidly while still keeping all the best cloud practices built in.

We believe in the power of serverless that given the right tools built around it, that we can make your application run on serverless on any given platform.

We see an opportunity here where we can provide the ecosystem of tools necessary for running applications as efficiently as possible (using serverless on any cloud), thus removing all the guesswork and excess as much as possible.

How does it work?

The bulk of what our platform does, runs after the user submits their application into our platform. Our command line client at this point will zip and upload your code to be submitted into our pipeline.

Build

At the code submission, our platform will wrap the users application with a lightweight wrapper that makes your function run on different cloud vendors that we support.

This wrapped code will be stored inside our infrastructure to be built inside our platform. Then we will execute recipes depending on your current programming language, web server framework, and the target cloud infrastructure. The output of these builds are build artifacts that can be directly deployed into the appropriate cloud vendors.

Deployment

We will then use those builds to run our deployment process. During the deployment process, our platform will determine how your function application will be distributed globally based on the policies.

This allows the application or service creator the flexibility to determine what is important for their application. Some users need deployment policies that are able to run their compute as close as possible to the end-users while others need to have something that can utilize all the public cloud resources as much as possible.

Execution

For execution, your request will be routed to our closest managed region (currently we are managing 3 major regions: US West, South-East Asia and Europe Central) and executed based on the policy that you set globally on your account or at the application level.

In addition, for execution, we have an implementation of Universal HTTP API Gateway. In short, it is a translation layer for your pure HTTP calls (Request Path URI, Queries, Headers, and Body) and wrap them into the appropriate serverless native input/output format that is understood by the execution platform.

Read more about it here in our Universal HTTP API Gateway Documentation.

Pricing/Limit

Naturally, your first question would be: how much does this cost?

Free. Really… Free

Well… Free with limits, then there’s the next tier limits of Pro access of $49 a month (or $499 a year). On top of higher limits, Pro access will unlock more features. We also offer 30-day Free Trial to our Pro subscription.

Here is a quick summary of what is the difference between Free, Pro and above:

As shown there, almost all the core features is available in Free tier. In addition, both Free and Pro accounts implement rate limiter functionality for LYR platform execution: https://en.wikipedia.org/wiki/Token_bucket

We calculate your requests and refills based on these parameters:

• Deployed RAM size
• Time taken to execute a function
• Account type (Pro/Free/Enterprise)

We will get into detail about these Lyrid rate limiters and the limit behaviors in the next few topics.

Above Pro, feel free to contact us for the following things for our Enterprise Solutions:

• If you require performance higher than our Pro accounts or requires additional infrastructure to run your workload.
• Managing and supporting dedicated infrastructure clusters for your workload (Compute, Database, Storage, etc2).
• Connecting your own infrastructures on your cloud (on-premise, public or hybrid cloud) to Lyrid to run closer to your data for high IOPS and throughput sensitive operations.
• Building and designing a resilient self-expanding/sustaining application on your own cloud or datacenters.

What is next for us?

The mark of good and successful platform companies are its ability to let their users experience the power of the platform and successfully repeat them with consistency.

And with that in mind, we are building an ecosystem with the platform. An ecosystem of reusable components that is built using our serverless-first mindset.

We are also looking to expand our pricing, and build more cost predictability and consistency using analytic data for all tiers of our users that are looking to burst up and down in more efficient way.

Here are some topics that I will be covering, along with some examples of the components that we have successfully integrated or use inside our platform along with some projects from our early adopters:

• We will dive into more details about Lyrid and its components that we built to create this platform.
• We will walk you through an experience of building a full application using an example of our authentication service endpoint:(https://id.lyrid.io). This service was previously hosted on Auth0 and prior to release but we decided to host this ourselves using our own technology and create a fork from this Django project: Django GraphQL Auth, and put our serverless spin to build the component. We will show you the steps for how we have successfully migrated from Auth0 into this.
• Learn how we package our Docusaurus documentation static files and serve them as a serverless web server function using Golang 1.x and Gin.
• Integrated our build and deployment process using GitLab CI/CD and our tools to continuously update our services.
• Our distributed analytics and metrics collections for all our infrastructure monitoring are based of our own serverless function built with Golang 1.x that extend the metric endpoints and service discovery of Prometheus. This is the backend of how we presented this graph in our dashboard:

• Learn how our friend and mentor at 1800-Battery reduces their AWS dependencies using our .NET Core 3.1 + ASP.NET implementation.
• Learn how we package XgeneCloud NodeJS 12.x + Express application as a Lyrid Serverless and deliver a low-code CRUD HTTP REST API into your SQL-based Database.
• And many more topics…

We will share these processes in future blogs and publish more guides on how to build different things using a mixture of open-source materials hosted on our GitHub page at https://github.com/LyridInc, blog, and video content. It will take us some time to create these contents and if you are interested to know about one of the above topics sooner, just contact us at hello@lyrid.io!

As for what we are looking for:

1. Feedback: So far, all the features that we have and built have been coming from all the feedback from our users. This helps us prioritize what we are working on in order to build this platform that everyone can benefit from. We want to be able to support more languages, and more native serverless platforms. Join our Slack channel to engage with us, tell us what you want to see in the platform and how you want us to build it.
2. Community Supporters and Contributors: Internal contributor to build on the platform or external contributors to build for the ecosystem (or both). We are not quite open-source end-to-end, but we are not shy about sharing our technology to the world.
3. Strategic Partnerships and/or Investors: We are looking to grow and we will need help doing it. I am one of those people that values your time and effort more than anything else in the world.

We want to be more available all across the world and expand our network and ecosystem to include as much support for every cloud vendor, web framework, and programming language!

With that said and done, you can register a Free Lyrid account to get started with this following link:

Register Here To Start Lyrid

Lastly, please don’t be shy to contact me hsutanto@lyrid.io or join our Slack channel to say hi. Let us know what you want to build with our platform!

Thank you for reading and stay safe!

Handoyo

LYRID was originally published in Lyrid on Medium, where people are continuing the conversation by highlighting and responding to this story.