Courier is the information superhighway for making persistent connection between client and server. Since we open sourced Courier Client Libraries for Android & iOS, we’ve been focusing on adding Flutter support using both libraries as foundation.

Courier is our solution to provide a real-time, lightweight, and highly efficient messaging highway between mobile apps and servers using MQTT.

Why Flutter, you ask?

Flutter is a very popular framework for building mobile apps with nearly half a million apps released. Considering that staggering number of apps, we also want to contribute to the Flutter Community by releasing the official Dart Library so Courier can also be used in building Flutter Apps.

• GitHub - gojek/courier-flutter: Dart port of our popular courier library
• courier_dart_sdk | Flutter Package

With Courier Dart library, one can create a long-running persistent connection to any MQTT broker targeting Android and iOS platforms. Under the hood, we call platform-specific native APIs that depends on Courier library:

1. Kotlin/Java on Android: courier-android
2. Swift/Objective-C on iOS: courier-iOS

Features

Quality of Service

All Courier libraries support three QoS levels for reliable message delivery: 0 (at most once), 1 (at least once) and 2(exactly once).

Automatic Reconnect

All Courier libraries are able to reconnect automatically whenever there are network or other unexpected failures using a connection retry policy.

Clean & easy to understand APIs

All Courier libraries provide clean APIs for connect/disconnect, subscribe/unsubscribe and publish/receivefunctionalities. The APIs are simple and easy to understand.

Observability

Courier libraries also provide events for tracking end-to-end message delivery and connection lifecycle.

Disk Persistence

Courier provides QoS 1 & QoS 2 messages in case the user is offline when sending those messages

Event Provider

The client can optionally listen to system events such as connection success, failure, connection lost, message received and many more for analytics tracking.

Documentation

All Courier Libraries are well-documented with examples. We host the documentation in GitHub Pages.

You can read more about Courier from our GitHub pages . We have also covered Android & iOS libraries in detail in our blog posts.

Courier · Information Superhighway

We have also written a five-part series covering Courier background and usage in Gojek for your weekend tech reading!

Introducing Courier - The Information Superhighway Between Mobile & Server - 5 min read

Thanks for reading and we look forward to your feedback. Contributions to the library are welcome!

How to reach out?

You can join our discord server to connect with the team behind this amazing project.

Find more stories from our vault, here.
Check out open job positions by clicking below:

Courier For Flutter Is Here! was originally published in Gojek Product + Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.

How we built a Courier library for iOS with MQTT.

Here is a summary of the previous posts so far in the series:

• Introduction of Courier — An information superhighway between Mobile and Server.
• Courier Message Broker — Benchmark of various brokers and reasoning behind choosing VerneMQ.
• Courier Android Library — Discussed challenges in Android and background to provide clean API, multi message adapters support, adaptive keep alive, and many more!

We’ve already rolled out Courier for our Android Merchant app to send order updates with high reliability and efficiency. We also intend to utilise Courier in the Gojek consumer app to improve user experience.

Introduction

Every time an order is made on GoFood, customers see an Active Order Screen, which looks like this:

The screen shows information such as driver location on the map, a timeline update of the order status, etc. The timeline starts from restaurant order acceptance until the food is delivered by the driver to the customer’s location.

We believe this is the best starting point for us to utilise Courier in Gojek consumer app as we were using HTTP pooling to get the latest food order status update. This was not efficient in terms of power and data consumption. We’re passionate about delivering the best experience, while also ensuring our app’s energy and data consumption is as efficient as possible.

As we started rolling out Courier to Gojek consumer App, we also needed to build the Courier library for iOS which provided the same set of features and API to the equivalent Courier Android library.

Background of Courier iOS Library

The Courier iOS library high level architecture is designed with several goals in minds:

1. Provide an abstract and simple interface for the client to subscribe to topics, receive stream of messages based on the topics subscribed, and publish messages to topics with any QoS.
2. The communication protocol should be agnostic and hidden from the client. With this, we should be able to switch to any communication protocol without much change from the client. (MQTT, WebSocket, etc).
3. The library used should be agnostic, so we can switch to any library without much effort (CocoaMQTT, MQTTClientFramework, etc).

With these goals in mind, we decided to build Courier iOS library instead of just using a MQTT library directly.

MQTT Libraries on iOS

There are 2 most used open source libraries that we can use to integrate MQTT:

1. MQTT-Client-Framework is a native Objective-C iOS library. It uses CFNetwork for networking and CoreData for persistence. It is a complete implementation of MQTT 3.1.1 and supports TLS.
2. CocoaMQTT is a MQTT v3.1.1 client library for iOS written with Swift 5. It uses CocoaAsyncSocket for networking and currently has no disk persistence support as of v1.2.5.

We have explored both libraries and created the comparative analysis table for both libraries below.

On our initial experiment, we decided to use CocoaMQTT on Courier; since it is written on Swift, most of the APIs seems pretty modern and Swifty compared to MQTTClientFramework which uses native Objective-C.

It served us pretty well without any issues for receiving messages from broker. But, as our future use cases will require the app to also publish message to broker, the requirement to support disk persistence and offline message buffering became super important. This will ensure us to persist message sent while the user has poor or no internet connection and send it when the user is reconnected.

As of version 1.25, CocoaMQTT doesn’t have this feature, so instead of reinventing the wheel and wasting resources, we decided to switch to use MQTTClientFramework which provide full QoS 1 & 2 disk persistence and offline message buffering. Based on our testing, it worked really well for both publishing and receiving messages.

Courier iOS Library Features

Courier is not just a simple wrapper on top of MQTTClientFramework. It also provides several main features to increase robustness and reliability for the iOS client app to communicate with the broker. Let me walk you through of all the main features of iOS Courier library.

Clean and Reactive API

Courier provides clean and simple API to connect, subscribe, and receive stream of messages from subscribed topic. With the nature of persistent realtime long run connection that process messages asynchronously, we designed Courier API to be reactive using Combine Publisher and Subscriber pattern.

// Method to get publisher for a subscribed topic
func messagePublisher<D>(topic: String) -> AnyPublisher<D, Never>

// Usage (type of model is generic using type inference)
courier.subscribe(topic: topic, qos: qos)
courier.messagePublisher(topic: topic)
.sink { [weak self] (message: Message) in
   self?.handleMessageReceiveEvent(.success(message))
}
.store(in: &cancellables)

Multi Message Adapter Support

Courier provides multi message adapters that supports various serialisation formats such as JSON, Protobuf, XML, or anything else that can be serialised into data and deserialised into model.

User of Courier can also implement their own custom message adapter by implementing MessageAdapter protocol (example of ProtobufMessageAdapter).

import SwiftProtobuf

class ProtobufMessageAdapter: MessageAdapter {
   

    func fromMessage<T>(_ message: Data) throws -> T {
        // Deserialize to Protobuf Message
    }

    func toMessage<T>(data: T) throws -> Data {
        //Serialize Protobuf Message to Data
    }
}

Subscription Store

Courier provides subscription store to manage currently subscribed topics and pending unsubscribe topics. This is super useful to avoid stale subscriptions in broker when the user connects with clean session flag set to false.

Fallback Policies

Courier provides several built in fallback policies for various Courier lifecycle system events (connection lost, connection failure, subscribe failure, and many more). User can react and provide their fallback logic in case of failure (for example fallback to HTTP polling). The user can also implements their own custom fallback policy and react to various Courier system events accordingly.

Database Persistence and Offline Message Buffer

MQTTClientFramework provides this feature to persist QoS 1 and QoS 2 messages in case the user is offline when sending those messages. It will retry to send the messages maintaining the order when the internet connection is back.

Automatic Reconnect

Courier provides the reconnect mechanism to handle the case where the internet connection is lost and whether it should retry to connect when the internet connection is reachable.

Custom Authentication Provider

In case, the user needs to be authenticated when connecting to the broker using username and password. Courier provides API to delegate the authentication handling from the user side. The user should be able to implement their own custom authentication to get the username and password used for making connection to the broker.

Event Provider

The user can optionally listen to Courier system events such as connection success, connection lost, message received from a topic, and many more). This is pretty useful in case they want to add analytics and metrics for their use case.

Conclusion

Courier is our solution to provide a realtime, lightweight, and high efficient messaging highway between mobile apps and server using MQTT.

With Courier at Gojek, we have just scratching the surface of many possible future use cases that we are planning to implement such as:

1. Chatting.
2. Realtime app metrics & analytics.
3. Any realtime bi-directional data sharing (e.g collaborative cart sharing, etc)

Thanks for reading and stay tuned for our upcoming articles!

Click here to read more stories about how we do what we do.

And we’re hiring! Check out the link below:

Courier — A Realtime Lightweight Messaging Highway in iOS with MQTT was originally published in Gojek Product + Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.