Since the last update on RxJS Ninja there have been a few new operators added, below are some details and links to StackBlitz demos showing them in action.

Numbers and Math operators

In @rxjs-ninja/rxjs-number the missing toFixed operator has been added, alongside a new custom toHex operator and the corresponding parseHex one allowing hex numbers to be worked with ( such as converting colours)

There are also new operators for some basic math - add , sub , div , mul , mod and pow all allowing you to modify source numbers, all accept a number, or an Observable number source.

Working with Browser Streams

Some new operators have been added to @rxjs-ninja/rxjs-utility that allow interoperability between RxJS and the StreamsAPI . These APIs are not in all browsers but there is an available polyfill .

fromReadableStream

This operator accepts a ReadableStream and provides the emitted values as an Observable, allowing you to use RxJS operators to work with the data.

• Demo - Using fetch body with fromReadableStream to show partial images
• Demo - A basic infinite number stream that ticks per second

toWritableStream

This operator accepts a WritableStream . The operator emits the source value and also writes to the stream. The operator takes care of closing the writer when the Observable subscription is closed.

• Demo - Streams an Observable interval value into a WritableStream with optional ability to stop the writer without ending the Observable subscription.

fromFetchWithProgress

This operator uses fetch to do a HTTP request, instead of the body response it emits either a number which is the current progress, or a Uint8Array containing the final response from the body.

• Demo Fetches an image and shows a progress bar with the current percentage and once complete shows the image.

Today I’ve re-released RxJS Primitives as RxJS Ninja . The new libraries have been published as the same last versions under their old name, so it’s easy to migrate to the new version - all now published under @rxjs-ninja instead of @tinynodes (a deprecation notice has also been left the old packages).

This week I released new versions of my RxJS libraries in rxjs-primitives . Since it’s released I’ve added a few new utility operators. Some of these have been out for a while since I originally wrote about the release, so I’ve highlighted them here as they may be useful to some developers.

You can check out the full docs here .

Typescript 4

Upgrading to Typescript 4 has allowed the removal of polymorphic functions in place of Vardic Tuple Types and is why there is a major bump on all packages.

This can be seen in the old and new concat operator in the rxjs-string package.

Most of the tests have also been converted to rxjs-marbles allowing for more robust Observable testing (if you are working with RxJS I highly recommend checking it out, it integrates well with runners like Jest).

it(
    'should return string value of string ending with passed character',
    marbles((m) => {
      const input = m.hot('-a-b-c-|', { a: 'test', b: 'testing', c: 'gone' });
      const subs = '^------!';
      const expected = m.cold('---y---|', { y: 'testing' });
      m.expect(input.pipe(filterEndsWith('g'))).toBeObservable(expected);
      m.expect(input).toHaveSubscriptions(subs);
    }),
  );

rxjs-array

npm install @tinynodes/rxjs-array

In the array module there are some operators to use with finding the difference or intersection between a source and a passed array, for example:

of(['a', 'b', 'd'])
 .pipe(difference(['a', 'c']))
 .subscribe(console.log) // ['b', 'd']

of(['a', 'b', 'd'])
 .pipe(intersects(['a', 'c']))
 .subscribe(console.log) // ['a']

These methods accept an array, or an Observable<Array> of items to compare against.

The module also included a binarySearch operator which returns a custom BinarySearchResult tuple.

rxjs-boolean

npm install @tinynodes/rxjs-boolean

A new Luhn algorithm operator luhnCheck is provided that does validation on numbers such as credit cards, ID cards and other value schemes that use the check.

fromString('4485275742308327')
    .pipe(luhnCheck())
    .subscribe(console.log) // true, this is a valid credit card

rxjs-number

npm install @tinynodes/rxjs-number

inRange / outOfRange and filterInRange / filterOutOfRange both all two numbers, the filter methods return the value from the source observable within the range of those values, while the other methods return a boolean value if in range. An optional third value will include/exclude the range value based on the method

fromNumber([-1, 0, 1, 2, 10, 11])
 .pipe(filterInRange(0, 10))
 .subscribe(console.log) // [0, 1, 2, 10]

// Passing true as the third parameter, the range numbers will also be excluded
fromNumber([-1, 0, 1, 2, 10, 11])
 .pipe(filterInRange(0, 10, true))
 .subscribe(console.log) // [1, 2]

rxjs-string

npm install @tinynodes/rxjs-string

New operators such as titleize , repeat and match add new utility features for strings. Where they can they also support localisation:

fromString('Mary had a little lamb')
 .pipe(titleize())
 .subscribe(console.log) // 'Mary Had A Little Lamb'

fromString('Mary had ä little lamb')
 .pipe(titleize('de-DE'))
 .subscribe(console.log) // 'Mary Had Ä Little Lamb'

rxjs-utility

npm install @tinynodes/rxjs-utility

The utility module contains some specialised tap operators such as tapIf, startWithTap and tapOnSubscribe. These provide a way to do side effects. With startWithTap it can be used with Angular to do a form touch, also tapOnSubscribe will fire when there is a subscription to the Observable:

// Only touch on first value change
form.valueChange.pipe(
 startWithTap(() => this.onTouch())
).subscribe()

// Fire when a component subscribes to the service bus
this.serviceBus.pipe(
  tapOnSubscribe((name: string) => console.log(`New Subscription to ${name}`))
).subscribe()

The tapIf will only fire if a passed method result is truthy:

fromNumber([1, 2, 3, 4, 5, 6]).pipe(
  tapIf((val) => val % 2 === 0), (val) => console.log(val)
).subscribe() // 2, 4, 6

The last operator is mapIfSource which might be a bit of a weird one but I hope might become useful.

The operator takes the value from the source and passes to a predicate method, and depending on the result will map the result of a passed method. A simple example would be:

fromNumber([1, 2, 3, 4, 5, 6]).pipe(
  mapIfSource(
    (value) => val % 2 === 0,
    (value) => val * 10,
    (value) => val * 20
  )
).subscribe() // 20, 20, 60 40, 100, 60

Here, if the result of the predicate is true multiply by 10, otherwise by 20. The method is typed to allow different return values based on the result (so you will have to handle the type later). For example we could even turn it into a FizzBuzz operator:

export function fizzbuzz(): OperatorFunction<number, string | number> {
  return (source: Observable<number>) =>
    source.pipe(
      mapIfSource<number, string, number>(
        (value) => value % 15 == 0 || value % 3 == 0 || value % 5 == 0,
        (value) => (value % 15 == 0 ? `FizzBuzz` : value % 3 === 0 ? 'Fizz' : 'Buzz'),
        (value) => value
     )
  );
}

// And now we use it in our code
fromNumber([1, 3, 5, 15, 16]).pipe(
  fizzbuzz(),
).subscribe() // 1, 'Fizz', 'Buzz', 'FizzBuzz', 16

Hopefully you’ll find these operators useful and feel free to leave feedback and suggestions.

Recently in RxJS Primitives I encountered a situation where one of the methods - concat was initially designed to take an argument list of strings and in the method used rest (...args) parameters, mimicking the signature and passing them to the String.prototype.concat

I’ve created a StackBlitz Project with the code for each step that can be followed along.

I wanted to refactor it to support an array of strings, but found that in the current implementation this is not possible and throws a TypeScript error:

export function concat(...args: string[]): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

fromString('Testing').pipe(concat([' one', ' two'])).subscribe(console.log)

Argument of type 'string[]' is not assignable to parameter of type 'string'.

Due to how TypeScript treats rest parameters, it expects a list of parameters as a single string which is turned into an array-like arguments .

To get around this, we can use TypeScript function overloading .

How to overload functions with rest parameters

My first attempt at writing this method lead to this code which attempts to keep the type information in all implementations:

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: string[]): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

export { concat }

This resulted in an error on the second implementation:

function concat(args: string[]): MonoTypeOperatorFunction<string> (+1 overload)
This overload signature is not compatible with its implementation signature.

It appears that TypeScript cannot convert an Array rest type to an arguments Array-like value, to get around this we need to use the any value in the last implementation :

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: any): MonoTypeOperatorFunction<string> {
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...args)))
}

export { concat }

So now the TypeScript compiler stops complaining, and we can test it using both supported argument types:

fromString('Testing').pipe(concat(' one', ' two')).subscribe(console.log) // Testing one two
fromString('Testing').pipe(concat([' one', ' two'])).subscribe(console.log) // Testing one, two

However, if you look at the output of the result you’ll notice a bug in our Array implementation where a comma appears in the text - the issue is that now the implementation treats the array as the first argument in a list of arguments - and changing the last method to be args:any would remove our rest parameter destructing.

To solve this, we need to use the any type again to destructure our arguments and check for the first one being an array, if it is then we use this as our destructured arguments into the String.prototype.concat method, but if it’s a string then we pass all the arguments using destructuring:

function concat(...args: string[]): MonoTypeOperatorFunction<string>;
function concat(args: string[]): MonoTypeOperatorFunction<string>;
function concat(...args: any): MonoTypeOperatorFunction<string> {
  const inArgs: any[] = [...args];
  if (inArgs[0] instanceof Array) {
    return (source: Observable<string>) => source.pipe(map(value => value.concat(...inArgs[0])))
  }
  return (source: Observable<string>) => source.pipe(map(value => value.concat(...inArgs)))
}

export { concat }

Now the implementation works as intended - our method can accept one or more string arguments, or an array of strings as the first argument - and it’s not possible to mix the two up - if you try add a second array to the arguments, the TypeScript compiler will look for single string arguments.

This week I released version 1.0 of RxJS Primitives to NPM - but the journey to get there was not as easy as I hoped.

In the past I’ve used CircleCI and at work I use Jenkins but with this project I wanted to try out Github Actions .

After some trial an error (and many failed builds) I managed to get the workflow working, I’ve decided to share here the steps taken to hopefully save you from the same pain.

Setting up your monorepo

The NX CLI is a set of tools that make managing your repository of JavaScript and TypeScript code easy. Originally built on top of the Angular CLI the tooling now supports more project types including Node, React and Web Components.

In my day-job most of my work is building Angular Enterprise applications, but I also work on my own open source software and in both cases I use NX, including using it to publish Angular and TypeScript libraries to NPM.

Each way of setting this up is subtly different. In this example I’ll show the steps of how I built a pipeline for releasing my TypeScript RxJS library.

The easiest way to start is to create your repo using create-nx-workspace

npx create-nx-workspace <project-name>

When running this, you will be asked a few questions depending on your setup. For RxJS Primitives I used a plain workspace, and then once created I added @nrwl/node to the project. The default @nrwl/workspace plugin allows the creation of libraries, but does not provide a publishable output (adding package.json) as an option.

Once set up you can now create libraries that can be built and published as NPM modules, using the @nrwl/node plugin type to create them. For rxjs-string I used:

> nx g @nrwl/node:library --name=string --directory=rxjs --publishable

If it’s an Angular library use:

> nx g @nrwl/angular:library --name=my-component --directory=ngx --publishable

After a few seconds, inside the libs folder will be a default output of a TypeScript library, with various tsconfig.json files for testing and building, a Jest config for unit testing, a index.ts file as the entry point and package.json for publishing.

One issue with nx is that with this configuration in the package.json you’ll find a 2-level name for your library (e.g. @tinynodes/rxjs-string) however in the root tsconfig.json file you’ll see the following in the paths property

{
  "paths": {
    "@tinynodes/rxjs/string": [
      "libs/rxjs/string/src/index.ts"
    ]
  }
}

If you are only using this library internally it’s not really an issue, but when you intend to publish the library I recommend changing the path to @tinynodes/rxjs-string to match the NPM import path.

Preparing for publishing

Once you have developed your library, it’s time to publish to NPM! First of all, make sure your public API (Functions, Classes, and Types) are exported in the library index.ts:

// index.ts
export { myFactoryFunction } from './libs/factory';
export { MyThing } from './classes/my-thing'
export { MyInterface, SOME_CONSTANT } from './types/thing-types' 

To see the output of this library you can run nx build library-name, this will output a NPM module to the dist folder.

Setting up Github Actions for Pull Requests

For an open-source library on GitHub, it’s good practice for you to get pull requests from other developers, and to have confidence in those PRs having a pull request checker is ideal.

First create a folder .github at the root of your workspace, and inside the workflows folder. This is the folder that GitHub checks to see if there are any YAML files containing action flows.

Create a file pr_on_master.yml and inside this file set up the following steps:

# File for Pull Request on master branch
name: PR on master

# When a PR is opened to master
on:
  pull_request:
    branches:
      - master
    types: [ opened, reopened, synchronize ]

This first section provides the name and triggers for when the action runs: when a pull request opened or re-opened against master.

Next the steps need set up:

jobs:
  build:
    # Setup OS and Node Version
    runs-on: ubuntu-latest
    strategy:
      matrix:
        # Latest nodes only
        node-version: [ 13.x ]

This sets up a Github Action runner and makes sure NodeJS 13 is installed. Here you can add more versions and run parallel tests against different versions if you plan to support more than one.

# Define Steps
steps:
  # Checkout code
  - uses: actions/checkout@v2
    with:
      ref: ${{ github.event.pull_request.head.ref }}
      fetch-depth: 0

  - name: Use Node.js ${{ matrix.node-version }}
    uses: actions/setup-node@v1
    with:
      node-version: ${{ matrix.node-version }}

The first two steps first check out the code from the pull request branch, and then sets up NodeJS to run.

The next piece of step was the part of setting up the pipeline that took the longest to fix.

To use the nx affected commands it needs a base branch to compare changes against - by default Github checkout does not pull all the branches, only the current one - and this includes the master branch. This command tells git to pull the master branch from the origin:

  # Make sure we have all branches
  - name: Fetch other branches
    run: git fetch --no-tags --prune --depth=5 origin master

Finally, we run some NPM command for installing the dependencies, running linting and test coverage.

  - name: Install environment
    run: npm ci

  - name: Run lint
    run: npm run affected:lint -- --base="origin/master"

  - name: Tests coverage
    run: npm run affected:test -- --base="origin/master" --codeCoverage

Using the affected commands, the pipeline will only run linting and testing against libraries that have changed against the master branch. More steps such as SonarQube or other webhooks can be performed here.

Publishing to NPM

When publishing libraries, there’s a few additional steps needed before we write the pipeline. First of all we need two access tokens, one for NPM and one for GitHub.

NPM Token

The NPM token will be used to publish the library to the public NPM registry. Log into NPM and under your profile go to the “Auth Tokens” section and create a new token with “Publish” access. Next go to your Github repository and under “Settings -> Secrets” add a new token called NPM_AUTH_TOKEN and paste in the value.

If publishing to a private registry follow it’s instructions on generating an API token.

Github Token

To publish changes back to GitHub from the pipeline you also need a personal access token - this can be created under your account settings in “Developer Settings -> Personal Access Token”. The only permissions you need to give this token are the repo permissions.

Add this under “Settings -> Secrets” as ACTION_AUTH_TOKEN (it seems you cannot name them with GITHUB_ in the name at all) 🤷‍♂️

Setting up the action

Like before we are going to add a YAML file to the .github/workflows folder - this time called publish.yml

First set the action up to trigger only when a PR is closed on master:

name: Merge on master
on:
  pull_request:
    branches:
      - master
    types: [ closed ]

The job section is the same as above, but for the steps there is a slight change - to allow merges to master that * don’t* trigger a release, each command will be wrapped in a if block this block checks the commit message for the string [skip-ci] to avoid running these tasks (unfortunatly it seems you can’t just put a block around the entire set of steps so has to be added to all of them)

steps:
  # Checkout code
  - name: Checkout Code
    if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
    uses: actions/checkout@v2

Instead of running lint and test, we’ll now run a set of different tasks - hold on tight because we’re about to dive into some bash code!

Deployment Step

In our publish.yaml add the following step which first exports our registry publishing setting with the auth token we set up earlier, then we call our publish bash script

- name: Deploy
    if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
    env:
      NPM_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
    run: |
      npm config set //registry.npmjs.org/:_authToken=$NPM_AUTH_TOKEN
      ./.github/scripts/publish-libraries.sh

Create the .github/scripts/publish-libraries.sh file and make sure it’s set to executable by typing chmod +x .github/scripts/publish-libraries.sh before committing the script.

The first part of the script is setting up our variables and getBuildType function to check what type of release we are doing. It’s good to use SemVer to publish based on changes that happen in the library and with this function using the following words in the MERGE COMMIT message will determine the release type.

For example: (fix): Fixed that annoying bug in issue #123 will set the release type to patch. The default is minor . Using (feat) will make a major change.

#!/usr/bin/env bash
set -o errexit -o noclobber -o nounset -o pipefail

# This script uses the parent version as the version to publish a library with

getBuildType() {
  local release_type="minor"
  if [[ "$1" == *"feat"* ]]; then
    release_type="major"
  elif [[ "$1" == *"fix"* || "$1" == *"docs"* || "$1" == *"chore"* ]]; then
    release_type="patch"
  fi
  echo "$release_type"
}

PARENT_DIR="$PWD"
ROOT_DIR="."
echo "Removing Dist"
rm -rf "${ROOT_DIR:?}/dist"
COMMIT_MESSAGE="$(git log -1 --pretty=format:"%s")"
RELEASE_TYPE=${1:-$(getBuildType "$COMMIT_MESSAGE")}
DRY_RUN=${DRY_RUN:-"False"}

The script also cleans up the dist folder, and has an optional DRY_RUN to set if you want to test the pipeline before release.

After this add the following:

AFFECTED=$(node node_modules/.bin/nx affected:libs --plain --base=origin/master~1)
if [ "$AFFECTED" != "" ]; then
  cd "$PARENT_DIR"
  echo "Copy Environment Files"

  while IFS= read -r -d $' ' lib; do
    echo "Setting version for $lib"
    cd "$PARENT_DIR"
    cd "$ROOT_DIR/libs/${lib/-//}"
    npm version "$RELEASE_TYPE" -f -m "RxJS Primitives $RELEASE_TYPE"
    echo "Building $lib"
    cd "$PARENT_DIR"
    npm run build "$lib" -- --prod --with-deps
    wait
  done <<<"$AFFECTED " # leave space on end to generate correct output

  cd "$PARENT_DIR"
  while IFS= read -r -d $' ' lib; do
    if [ "$DRY_RUN" == "False" ]; then
      echo "Publishing $lib"
      npm publish "$ROOT_DIR/dist/libs/${lib/-//}" --access=public
    else
      echo "Dry Run, not publishing $lib"
    fi
    wait
  done <<<"$AFFECTED " # leave space on end to generate correct output
else
  echo "No Libraries to publish"
fi

This part of the script allows us to control the build and release - the first line gets a single line list of affected libraries from the current master to the last HEAD in master - if you use only PRs to make changes into master this is very effective - but can break if you make changes directly to master.

This is done instead of using nx affected as there is currently no task for publishing, so this gives a way to provide a loop for both building and publishing.

Both while loops parses this string and splits on the space to allow a loop to be run.

In each loop there is a string replacement ${lib/-//} - this changes the library name (e.g. rxjs-string) into a path (rxjs/string)

The first while loop ensures we are in the correct directory and does npm version - bumping the package.json for release - it then runs the build task for production, and ensures any dependencies are also build.

The second while loop iterates over the same list but runs the npm publish command in each directory, setting the access to public.

Congratulations

If you’ve made it this far, well done - this is quite a long post! At this point your pipeline should have successfully published your NPM module for other developers to use! We’re not done yet though!

Before running this we are going to add some additional steps so that we can publish documentation and changes back to Github:

  - name: Build Docs
    if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
    run: npm run docs

  - name: Commit files
    if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
    run: |
      git config --local user.email "[email protected]"
      git config --local user.name "GitHub Action"
      git add .
      git commit -m "Release [skip-ci]" -a || true

  - name: Push changes
    if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
    uses: ad-m/github-push-action@master
    with:
      github_token: ${{ secrets.ACTION_AUTH_TOKEN }}
      tags: true
      force: true

For RxJS Primitives I used TypeDoc to generate static content to the docs folder, which is then used by GitHub Pages, but here you can set up whatever documentation system you prefer.

Once the docs have been generated we then commit all changes including the package.json version bumps back to git and then finally push it back to master using the GitHub token generated earlier.

Within a few seconds your GitHub Page should update with the latest content, the master branch reflect all changes made.

That’s a wrap! You’ve now successfully published your TypeScript library for other developers to use, along with documentation.

If you’ve found this article useful, let me know - and if you find any issues or improvements please get in touch!