We’ve been heads down for the last few months working on our new professional offerings: SKIE and Touchlab Pro. We’ll be posting about them soon. While that work has been the major focus, our open source projects have been mostly getting critical updates. As our new projects have been nearing release, we’ve finished a bunch of updates to our open source libraries.

KMMBridge​

We’ve had a few tweaks and updates pending for some time. Those have been pushed with version 0.3.7.

One of the less visible but really important changes is a flag that disables a lot of the “stuff” that needs to happen when actually publishing versions. If you opened a project with earlier versions of KMMBridge on a dev machine, several things might happen in the background that really don’t need to. Now they won’t.

Also, if you happen to use S3 as your publish target, there’s a bug in the default AWS SDK. This version should fix that for you.

Finally, the website has been updated: kmmbridge.touchlab.co

Kermit​

Kermit’s API has gone through a fairly significant update. We’ve released 2.0.0-RC3 with the new changes. Feedback is welcome.

New API​

1.x versions had a Logger API that was designed to be called from non-Kotlin environments. For the most part, that meant avoiding default parameters. However, the primary use case is calling from Kotlin. The primary Logger API was updated to be (mostly) source compatible, but now include simpler method definitions with default parameters.

In simple terms, that means all log calls can have message, a Throwable, and a tag specified.

For non-Kotlin environments such as Swift and JS, you can export kermit-simple, which provides extension methods to make calling less verbose from those environments (without default parameter support, you'd need to provide each value on each call).

Custom API​

We had a lot of feedback about Kermit’s API decisions since releasing it. Some has been incorporated, but some is really just personal preference. For example, full names instead of single letter methods for logging calls (info() instead of i()). You can't make everybody happy.

However, most of the functionality has been moved to the module kermit-core. The kemrit module is now fairly small, and just represents the public Logger API. That means you can add your own Logger API on top of the Kermit internals. See Custom Logger API for more detail.

Website Update​

And, of course, the website has been updated: kermit.touchlab.co

CrashKiOS​

CrashKiOS has had an interesting history. The library allows Crashlytics and Bugsnag to report symbolicated Kotlin crash reports on iOS (and other Darwin/Apple targets). Most of this functionality was moved into Kermit as we had several requests to support Kermit integration.

Changes​

However, crash reporting has other features unrelated to logging, such as custom key/value pairs, and automatically catching hard crashes. Also, NSExceptionKt tweaked the method being used for reporting hard crashes. This allows reports to be consolidated rather than the earlier dual report method.

The crash reporting details from Kermit have been moved back into CrashKiOS, and various parts of NSExceptionKt have been integrated, to support crash reporting as a standalone library again. Kermit's support has been simplified. You can add log writers for Crashlytics or Bugsnag, but they both delegate to CrashKiOS.

New Website​

Website also updated: crashkios.touchlab.co

Stately​

Stately was designed to help developers work with the original strict memory model of Kotlin/Native. That memory model has been deprecated. We’ve released one last version of the original Stately built on 1.8.0, for teams that aren’t ready to update everything and who would like to continue using Stately as it was in earlier Kotlin versions.

What Now?​

Since the strict memory model has been deprecated, Stately’s future has been up in the air. We internally still find some of the concurrency pieces useful, and with the new memory model, there are some utility classes we wanted to have available. It is likely that newer libraries will better handle these cases in the future, but we’ve added them to Stately for now.

stately-concurrency contains structures like locks and atomics. There is functionality similar to Atomic-Fu, and we may eventually deprecate this support in Stately, but for now we continue to use it in our projects.

stately-strict contains shared definitions of things like freeze(). This is only for use in projects that still have this code in their projects and don't want to remove it. Most projects will be running on the new memory model, and freeze() won't really do anything.

stately-concurrent-collections is a simple implementation of thread-safe collections. We anticipate somebody will eventually release something more robust, but for now, you can easily include this module to have shared collections in your project.

Stately did not get a website refresh. The future utility of Stately is up in the air, so we didn’t prioritize the website update.

Documentation Sites​

We’ve moved several of our open source projects’ documentation to Docusaurus. I have been pretty skeptical of these kinds of site generators, but finally took a look at Docusaurus after several team members mentioned it as an option.

It is one of my favorite things now. I’ve even moved my personal site to Docusaurus, just for the blog.

There’s too much going on to explain why I love it, but some highlights (some Docusaurus-specific, some just neat hacks)

• Docusaurus let’s you create custom react components and use them directly in markdown, so we can extend the markdown syntax easily
• Dark and light mode
• Code style is highly customizable. Kotlin code has Intellij/Android Studio colors, while Swift looks like Xcode (see Kermit docs)
• Our doc build process auto-replaces version strings into code samples. This isn’t Dosusaurus specific, but a nice add on. The CI process gets the latest published release string and replaces a template string in the markdown. Anything in gradle.properties can be swapped as well.

It’s easy to tweak and extend. It’s just great.

The home page of each doc site is now using a simple react template (from cruip.com). That uses tailwind CSS, which is the other new (to me) web thing I’m in love with now. Although there’s been discussion, Docusaurus doesn’t have a tailwind theme, so the CSS is bit of a blended hack. Once you get started, though, its really easy to develop with it.

Originally published at https://touchlab.co/open-source-updates/ on March 10, 2023.

The Touchlab team is working on working on tools for KMM. For teams getting started, check out KMMBridge

GitHub - touchlab/KMMBridge: KMMBridge is a set of Gradle tooling that facilitates publishing and consuming pre-built KMM (Kotlin Multiplatform Mobile) Xcode Framework binaries. See https://touchlab.github.io/KMMBridge/intro/ to get started.

We have some new things in the works to improve the iOS dev experience, and some experiments a little further down the line for coordinated feature development. If you want to know more about what we’re working on, or maybe want some help building with KMM, reach out at #touchlab-tools in the Kotlin lang slack, or directly at touchlab.co.

Today we are announcing KMMBridge, part of Touchlab’s new Faktory* tooling suite. This set of build tools allows mobile teams to successfully publish shared code faster than ever with Kotlin Multiplatform Mobile.

Specifically targeted at iOS publishing, teams can now implement the Internal SDK Flow approach, today’s best practice to quickly integrate KMM into production app builds.

GitHub - touchlab/KMMBridge: KMMBridge is a set of Gradle tooling that facilitates publishing and consuming pre-built KMM (Kotlin Multiplatform Mobile) Xcode Framework binaries. See https://touchlab.github.io/KMMBridge/intro/ to get started.

KMMBridge allows teams to publish and integrate Kotlin Multiplatform Mobile (KMM) Xcode Frameworks as an internal SDK.

When you’re ready to get started, head to our KMMBridge Quick Start Tutorial.

Why We Built a Team-Focused Tool

Touchlab has spent years working with teams at different stages of their Kotlin Multiplatform journey. Evaluating KMP, putting KMP into production, and yes, even a fair bit of time talking to teams who looked at KMP and decided against it. One of the key lessons we’ve learned seems obvious in retrospect.

Different types of teams work in different ways.

Of course, every team is different. Every situation will be different. However, there are common patterns.

Out of the box, the KMP and the Kotlin/Native toolkit will let you build an Xcode Framework, then integrate it locally into your Xcode project. This works well if everybody on the team builds Kotlin. And in an ideal world, everybody would be able to build Kotlin.

But in the real world, most teams who have succeeded followed the Internal SDK Flow approach.

The Internal SDK Flow Approach

As with adopting any technology on a team, early wins help build momentum and confidence so the team can unlock the full potential over time. The Internal SDK Flow has emerged as the best practice to get faster ROI when adopting KMM. At its simplest, this approach means the Kotlin Xcode Framework is pre-built and published as an internal SDK (a private module shared with your team) so iOS developers can include it without building Kotlin locally.

The Kotlin Xcode Framework is pre-built and published as an internal SDK

Somebody else (a team member, CI, etc.) builds the Kotlin Xcode Framework and publishes it somewhere internal. That Framework is then consumed by Xcode just like any other SDK. The iOS build workflow does not need to add Gradle or Kotlin and does not add to its build time or risk. Our build tools let teams build their way immediately, increasing efficiency and eliminating unnecessary complexity to get started.

For more context, see Nate Ebel’s talk from Droidcon NYC 2022: Adopting Kotlin Multiplatform in Brownfield Applications. It’s a very good overview of the startup issues teams face.

How KMMBridge Works

KMMBridge is a set of Gradle tools that integrate with your Kotlin Multiplatform Mobile project to assist with building and publishing KMM Xcode Frameworks. It is built for the Internal SDK Flow, but you can also use it to publish public SDKs as well.

The basic flow is as follows:

1. Edit and test your shared Kotlin code, then push to your source control repo
2. Kick off a CI build with KMMBridge. It packages, versions, and publishes your Xcode Framework
3. Tell Xcode to update to the new version

The iOS build workflow does not need to add Gradle or Kotlin and does not add to its build time or risk

For iOS devs, there is no need to install or learn anything new. KMMBridge supports Cocoapods and Swift Package Manager. It can publish to a few different back ends out of the box, including Github Releases, with more support coming soon.

Why Use This Approach?

As a leader or a member of a mid-sized-plus mobile team, does any of this sound familiar?

• Most of the Android devs don’t know much about iOS builds
• Most of the iOS devs have never touched Gradle**
• The apps are in separate repos or at least have very different build workflows

Every team that Touchlab has worked with, that has successfully added KMP to their existing iOS app, has started with an internal SDK flow.

Some teams have relatively sophisticated internal module-sharing systems, Artifactory or similar. Most (and you’d be surprised at who) have nothing like that. Even with some kind of artifact sharing in place, there is a lot of custom work that would need to be done to wire up Kotlin Xcode Frameworks to be included into a production iOS build.

KMMBridge will help most teams speed past this and get started productively with KMM.

What Comes After the Internal SDK Flow?

What is the next step when you want to scale and get shared code closer to your feature dev flow?

It can seem simple on the surface but this can get tricky, depending on team structure and how features are developed. One example we’ve seen a bit of: over time both teams could start to edit the code as if it wasn’t shared, and without clear guidance those edits can start to conflict in multiple ways.

Most teams with Kotlin Multiplatform Mobile in production have used one or more Touchlab solutions. If you are looking to scale your KMM dev (or have already done so), we should chat.

*There is more to the Faktory tooling that is in the works, but we decided to carve out this bit now as it is a necessary entry point for teams adopting KMM. As KMM enters Beta, for continued growth and adoption, there are core capabilities that simply need to exist, and the internal SDK flow is one of them.

**Most Android devs try to avoid it, me included (no offense Gradle)

Cross post from https://touchlab.co/kermit-and-crashlytics/

Kermit is a Kotlin Multiplatform logging library. The log is configured on each platform to write to various, potentially platform-specific, outputs, but can be called from shared “common” Kotlin code.

Crash reporting tools like Crashlytics allow you to get error reports from remote devices running your software. These tools are very common in mobile app development, and in most cases are critical for monitoring application health in production.

Crash reporting for KMP involves a setup similar to other platform-specific libraries that you want to use in a KMP context. You configure each platform according to the documentation from the software vendor, then have some common Kotlin code that can interact with the platform-specific library.

Crash reporting is a bit of a special case for iOS. It’s a long story, but the summary is the JVM and Kotlin throw unchecked exceptions up the stack, and if unhandled, eventually wind up in a catch-all handler. On iOS, exceptions don’t work the same way. When a crash happens the runtime essentially stops and a handler is called. The crash library gathers the state of each thread's stack and sends that to the server.

If you call a Kotlin function on iOS and an exception is thrown from somewhere in that Kotlin call stack, the Kotlin runtime will bubble the exception up the stack like it would on the JVM. At the “border”, where you first called the Kotlin code, the Kotlin runtime will trigger a crash. If you look at the crash reports you’ll see konan::abort() . That’s the Kotlin runtime (internally called “konan”) force-killing the process.

That “works”, and if you don’t need symbolicated Kotlin crash info, then you’re done! However, all of the crash reports will lose potentially vital info. Specifically the Kotlin stack and the exception message.

To get that info, we register an uncaught exception handler with the Kotlin runtime, and when a crash originates in Kotlin code, a custom handled error is sent with the Kotlin stack info and exception message.

The only major negative here is that each crash will result in 2 separate reports for the same event. One fatal crash that ends with konan::abort() and a non-fatal report with the Kotlin stack and message. There is no way to send a custom fatal crash report, and no way to modify the fatal crash, so we wind up with 2. Hopefully, as KMP becomes more popular, vendors will add direct support for Kotlin. Alternatively, the Crashlytics client is open source, so maybe somebody from the community wants to give it a shot?

Setup

First let’s get the dependencies squared away. You can see sample code here:

Kermit/samples/sample-crashlytics at main · touchlab/Kermit

Add the Kermit dependencies to the common source set.

implementation("co.touchlab:kermit:1.0.3")
implementation("co.touchlab:kermit-crashlytics:1.0.3")

Just FYI, if you’ll be exporting Kermit to iOS, you’ll need to use api instead of implementation.

You should add CrashlyticsLogWriter to the Logger config. You can do that with a static logger instance, or add it to the global logger:

Logger.addLogWriter(CrashlyticsLogWriter())

The log writer adds log statements to the Crashlytics client, so when you send a crash report, you’ll see those statements in the report. You can configure the minimum severity, the minimum crash severity, and whether tags are included in the log statement. By default, the minimum severity is INFO, the minimum crash severity is WARN, and tags are included.

The minimum severity is the log level that will get logged to Crashlytics. The default again is INFO, so DEBUG statements won’t be sent to Crashlyics, unless you alter the default configuration.

The minimum crash severity is the minimum log severity that will trigger a handled exception being sent to Crashlytics. By default, it is set to WARN, which means that exception log statements of WARN, ERROR, and ASSERT create handled exceptions in Crashlytics.

val ex = SomeException()
Logger.e(ex) { "We have a problem..." }

If you want to log exceptions but not have them create Crashlytics exceptions, make sure to keep them below the minimum.

val ex = NotImportantException()
Logger.i(ex) { "We have a problem..." }

iOS Crash Handler

On iOS, to capture Kotlin crashes, we need a little extra setup. Kermit defines a convenience function to set up the default handler.

setupCrashlyticsExceptionHook(Logger)

Call that function and pass in a logger instance (in this case, I passed in the global Logger ). This function is only defined for iOS!!! You should define a function in your iOS source set and call it on app start (from Swift/Objc).

When a crash happens in Kotlin code and bubbles up to Swift/Objc, Kermit will now send a handled report to Crashlytics before allowing execution to continue and the app to crash.

Configuration Notes

Kermit Crashlytics integration on iOS uses cinterop to talk to the native client. Crashlytics itself needs to be added and linked in your build. The details of how you add and link binary on iOS are a little beyond the scope of this post, but be aware that simply adding the kermit-crashlytics dependency won’t include the iOS Crashlytics SDK.

You have 2 basic choices of framework type: static and dynamic. We tend to use static in our builds, but you can use either.

Static

Static is somewhat easier. You only need to link to the Crashlytics binary when actually building your iOS app, or when running Kotlin tests (and in that case, only if you actually use CrashlyticsLogWriter in your tests, which wouldn’t really make sense).

If your tests are not building because the linker can’t find Crashlytics, you can add our stub dependency. It won’t work if you try to use it, but it will satisfy the linker.

dependencies {
    implementation("co.touchlab:kermit-crashlytics-test:1.0.3")
}

You’ll still need to add Crashlytics to your iOS project, but you can do that with Cocoapods or SPM.

Dynamic

Dynamic frameworks are a little more complicated. The Crashlytics binary needs to be available at compile time. It is relatively easy to do this with Cocoapods.

cocoapods {
    //Other config
    framework {
        isStatic = false
    }
    pod("FirebaseCrashlytics")
}

However, that will cause Kotlin to generate a whole different cinterop definition for Crashlytics. That’s not a huge technical problem, but since we don’t use it, it’s a waste. On our near-term todo list is adding a way to use cocoapods as strictly a dependency integrator with the ability to disable things like the cinterop, but for now, you’ll need to live with the imperfections (or wire up Crashlytics yourself).

I should mention, it is also possible to use pod(“FirebaseCrashlytics”) for static frameworks rather than our test stubs, but that will also generate the extra cinterop code.

Usage

Log messages above the minimum severity will be sent to the Crashlytics log. If a log call includes an exception and is at or above the minimum exception severity, that call will trigger a handled error report with Crashlytics.

For hard crashes on iOS, as mentioned, you will get 2 reports. One “fatal” one “non-fatal”. The non-fatal one has the extra Kotlin stack and info, and you’ll want to be able to navigate between them. To do that, the crash handler generates a random id and adds it to Crashlytics key/value pairs. You can search for matching reports with that key.

Again, not ideal, but functional. When you get a chance, tweet at the various vendors to let them know you really want first-party KMP crash reporting support. For now, you need to search to find associated reports.

Assuming you have your project set up correctly, you should start to see symbolicated crash reports for Kotlin on iOS. Correctly configuring Crashlytics can sometimes be fairly difficult, so make sure to test that your reports are being sent to the Crashlytics back end and that your debug/dsym info is getting there.

If your stack traces look like this, the server doesn’t have some debug info.

The Kotlin stack trace should look more like this, with Kotlin source line numbers.

The Kotlin/Native compiler generates a “standard” Xcode framework, so you shouldn’t need any custom configuration to get debug info uploaded. It is one of those things that can be difficult to configure correctly, Kotlin or no Kotlin, and you don’t want to have a production problem and no way to see what’s happening, so just make sure it’s working as expected!

Hiring!

We’re looking for people to join the KMP team. If you’re interested in the tech, Touchlab is a great place to build your experience.

Careers at Touchlab - Open positions & apply here | Touchlab

Cross posted from https://touchlab.co/kmm-memory-model-when-to-switch/

Touchlab will be defaulting to the “new” memory model for anything that’s in dev and/or not hitting production for a few months. That decision will be reviewed if there’s some wild performance issue or bug, but otherwise, we’re just doing it. On Slack, the team collectively decided to make the switch this afternoon.

Why now? Well, the urgency this morning was largely driven by this.

1.6.x will be the last series of native-mt releases and we are effectively decommissioning native-mt in the favour of the new memory model that is supported out-of-the-box in 1.6.0 coroutines release.

…

We encourage developers to try out and evaluate the new MM: that’s our bet, we are investing our resources, development and testing in it and the quality of regular coroutine releases (including pre-release testing, a priority of the bug-fixes, extensive support of existing features and so on) is much higher than of native-mt builds.

The general vibe is to wait until it’s not “experimental”, but if you’ve been working with KMP and Kotlin/Native for any length of time, you’re probably used to “experimental”. If you wait for something resembling “Beta”, the rest of the community will probably have switched long before you did. Such is the life of an early adopter. Also, assuming the new model works “good enough”, more people using it will speed up maturity through feedback.

That’s for client/app work. There are some decisions we (Touchlab), and other library maintainers, need to make around library decisions.

Large portions of Stately, if not all of it, will probably be archived (discuss). However, we’ll also need at least an OK implementation of concurrent collections. Next, while Atomic-FU covers a lot of what we’d want for concurrency primitives (atomics, locks, etc), we’ll need a review to see if we want to completely switch over. Stately may continue on as a “simpler” version of some of what Atomic-FU does, or maybe we just need better Atomic-FU samples? TBD.

Koin has a very special Kotlin/Native implementation that forces you to inject on the main thread. I actually implemented that, trying to do it in a way that cooperated with the strict memory model. That all needs to be unwound for sure, and function pretty much the way the JVM version does.

If there’s anything else in the library space you think needs a review, reach out. Most libraries will at least need to start testing on the new model. The big concerns I have are: 1) code that explicitly freezes, or 2) code that relies on the strict model to enforce its safe concurrency. Number 1 is more of an immediate problem, as there are a number of libraries that likely do this in code. You may run into it, but it’s not “risky” in the sense that you’ll just crash but you’ll know it, and nothing “wrong” will happen. Number 2 will be an edge case for sure, but part of the point of the strict model was that you could make assumptions that are no longer true, and you don’t really know what assumptions the author made (certainly the runtime won’t). This is, I think, far less likely of a problem, but it kind of gets at the core of what the strict memory model tried to “fix”. Don’t lose sleep over it, but keep an eye out!

If you are super interested in this kind of stuff, well, we are hiring of course :)

Careers at Touchlab - Open positions & apply here | Touchlab

Today we’re announcing the stable release of our logging library Kermit

GitHub - touchlab/Kermit: Kermit by Touchlab is a Kotlin Multiplatform centralized logging utility.

If you’ve been following me, I’ve given a few talks on Kotlin KMP library development. In my previous talks I gave some suggestions on what kinds of libraries the community needs. I went out of my way to say avoid building logging libraries. Even way back in 2019, there were already a few options for KMP, so I assumed that topic was handled.

After doing some digging, though, nothing really felt like it was done the way we wanted. Also, in early 2020, due in part due to circumstances we’re all aware of, we had a few more hands available than planned. We had wanted everybody on the team to get a chance to work on a KMP library, and logging libraries are, if nothing else, conceptually straightforward. After some general architecture chats, the team dove in and made a dev version of Kermit.

The plan was to try it out, get some feedback, then make a 1.0 and tell everybody!

Then we got super busy.

The good news is we got some extra time to think about the library design, and a lot more feedback than you’d think for something we never quite announced.

With that feedback, plus more lessons learned shipping KMP code in the meantime, we sat down and made a plan for Kermit 1.0.

Kermit 1.0

Kermit has a simple API, is easy to extend, and is focused on performance. Getting started is very easy, but Kermit provides more sophisticated configuration as you move beyond dev into releasing your applications.

Rather than a single long post that I’ll never finish, we’ll wrap this up with basic setup, then go deep on a few other topics in future posts. The list is subject to change, but look for posts on basic setup and usage, production setup, crash reporting integration, and a deep dive on the performance considerations that went into the design.

Note: Kotlin 1.6.0 publishing issues

To clarify some confusion around releases, we originally published 1.0.0 for Kotlin 1.5.31, then this post got delayed a bit. When Kotlin 1.6.0 came out, we ran into some Kotlin/Native stuff that needed fixing, but ultimately pushed Kermit 1.0.2 with Kotlin 1.6.0. However, it turns out Kotlin 1.6.0 has Android publishing issues, so please use Kermit 1.0.0 until Kotlin 1.6.10 releases, which should be in a couple weeks.

Getting Started

In your Kotlin Gradle config, add the Kermit dependency in `commonMain`.

commonMain {
  dependencies {
    implementation("co.touchlab:kermit:1.0.0")
  }
}

Without any other configuration, you can use the global Logger with default log writers per platform. For Android, it writes to android.util.Log, iOS logs are written with println, and JS goes to console.

For iOS we replicated a good idea from Napier. Color coding the iOS logs. Format is slightly different, but same concept.

To write logs with the global logger, just do this:

Logger.i { "Hello" }

The log calls are lambdas, which will allow us to avoid creating strings if they’re not needed, but you can also call “regular” methods with a string.

To log an exception:

catch(ex:Exception){
  Logger.e(ex) { "Something happened" }
}

To change tags, do this:

val l = Logger.withTag("DifferentTag")
l.i { "New logger, new tag" }

That’s it for today. Again, keep an eye out for other posts about Kermit. Please share feedback, star the Github repo if you like it.

Work at Touchlab!

We’re hiring! If you want to help define the future of Kotlin, and like living a bit on the edge technically speaking, please reach out!

Careers at Touchlab - Open positions & apply here | Touchlab

Kotlin/Native’s strict memory model has been a common topic of interest and/or concern in the community for some time. The arrival of the new memory model will remove what many consider an adoption blocker. However, creating and testing a new memory model is a significant undertaking.

It is tempting to wait until the new memory model is “ready”, but the community can help accelerate the maturing of the platform by testing the new model and reporting bugs.

We are starting to run our open source library tests against the new memory model. Just turning it on for Kermit found a relatively minor issue. If you publish a library, it would be great if you could turn on the new memory model and see if things work. Although everybody’s config is different, here’s how we do it.

Kermit’s Kotlin version is in the gradle.properties file.

GROUP=co.touchlab
VERSION_NAME=1.0.0-rc4
KOTLIN_VERSION=1.5.31
# etc...

Kermit/gradle.properties at main · touchlab/Kermit

That makes turning on the memory model simple. We can pass in a different Kotlin version and memory model flag, both as gradle properties.

Currently we run tests 3 times. First with Kotlin 1.5.31, which is our published version. Then Kotlin 1.6.0-M1, but with the strict memory model, then Kotlin 1.6.0-M1 and the new memory model. In the Github Actions script, here are the lines:

- name: Build
  run: ./gradlew build --no-daemon --stacktrace
- name: Build 1.6
  run: ./gradlew build -PKOTLIN_VERSION=1.6.0-M1 --no-daemon --stacktrace
- name: Build 1.6 nmm
  run: ./gradlew build -PKOTLIN_VERSION=1.6.0-M1 \
-Pkotlin.native.binary.memoryModel=experimental --no-daemon \
--stacktrace

Param -PKOTLIN_VERSION=1.6.0-M1 sets the Kotlin verison, and -Pkotlin.native.binary.memoryModel=experimental turns on the new memory model.

Also FYI, macos-latest is a little old and has Xcode 12.4. macos-11 has Xcode 12.5, which is required for Kotlin 1.6.0-M1.

Obviously, if you run those once and they work fine, you don’t need to leave them on. We found a minor issue with Kermit, linked above, and I assume it’ll get fixed, or we can just tweak our implementation. I don’t think we’re going to merge the build script changes above. However, once 1.6.0 is live, I think we’ll probably run both memory models for a while.

If you find an issue, I’ve created a simple sample project template.

GitHub - touchlab-lab/kn-nmm-test: Kotlin/Native New Memory Model Test

The project is fairly minimal. Put your failing example in a test. To run code in another thread, call “ThreadHelper.doInBackgroundUnfrozen”. Obviously as the new model matures, the focus will shift more to performance issues than hard “bugs”, but at this point I would expect to see actual bugs here and there.

Ping me in the Kotlin slack or on twitter if you’re having issues setting up tests!

When submitting bugs, make sure to select Kotlin as the project. I did it wrong at first. Just FYI.

Kotlin 1.5.30 officially launched just now. There has been a steady stream of Kotlin releases over the last year or so, each with new features and fixes.

In other words, business as usual.

Since we (Touchlab) are plugging away, day in and out, working on client projects using KMM🐶, it can be easy to lose track of the bigger milestones. Kotlin 1.5.30 feels (numerically speaking) like just another point release, but this is a milestone advance for the platform. 1.5.30 presents a few big features, at least in preview, which will ultimately pave the way for “production ready” mainstream adoption of Kotlin shared code technology.

Kotlin 1.5.30 is a turning point in KMP. Here’s why.

Memory Model Preview

One of the most controversial aspects of Kotlin/Native is the concurrency and memory model. There are a lot of takes on it. Whether it was a good or bad thing, and why, is beyond the scope of today’s post. However, I talked about it much more than anything else over the last few years because for KMM and KMP to grow, people would need to understand it. Still, it’s been a significant blocker for some. That will be changing soon. I would not run the new memory model in production yet 🤞, but you can start experimenting with it now, and expect production deployments to pop up as we settle into 1.6 (I’m guessing :) 🚀.

What is this memory model business you ask? Well, in some languages (Java, Swift, etc), multiple threads can read and write shared state. There are a number of ways in which that can be dangerous and error prone. Many languages don’t let you access state from multiple threads (JS), some even have memory restriction rules built into the language and compiler (Rust). Kotlin/Native introduced a relatively unique model. In concept, it’s interesting, but in practice, it can be confusing and a barrier to adoption.

The new memory model will presumably allow unrestricted shared access to state. There are outstanding questions. Will there be equivalents to the JVM concurrency features (synchronized, etc)? If you write a library that needs the new memory model, will you get an error compiling with the freeze model? That kind of thing, but I’m sure these will be resolved in time.

Check out Russell’s quick look at the memory model preview

What will those changes mean to your code, and what should you do today? For the next 6 months at least, you’ll need a working understanding of the current model, and your code probably won’t change at all when you switch on the new model. It’s relaxing the rules more than changing them. My thoughts from mid-2020.

Compile-time Code Analysis

Compiler plugins have been maturing over time, as have the IR compilers for JVM and JS. That maturation isn’t exactly tied to 1.5.30, but KSP support for Kotlin Native and KMP in general is being added, with K/N support added a few weeks ago.

Kotlin Native is quite static at runtime, so any sort of significant code analysis and manipulation needs to be done at compile time. Because there was no real “annotation processor” equivalent, there are a number of libraries that can’t really be built in the same way that they have been for Android. Things like Dagger are only possible with compiler plugins. Any real mocking will need code transformation. Also, generating Swift code to make integration easier needs stable (and reasonably documented) compile-time code analysis. 1.5.30 doesn’t suddenly turn all of these things on, but the maturity of the plugins, the addition of KSP, and the mutliplatform IR compiler becoming the default will greatly accelerate areas of library development that were significantly more difficult with earlier versions.

Hierarchical Modules/Source Sets

I would say this falls under the general header of maturing and stabilizing tools, but very specifically, the Hierarchical MPP. Especially in native, there can be many targets with very similar dependency api’s, but configuring the IDE and compiler to recognize them has not been fully functional. It has worked OK for simpler situations like ios(), but for libraries with nested hierarchies and many targets, publishing HMPP-enabled libraries hasn’t really been an option.

With Kotlin 1.5.30, that has been (mostly?) fixed! With gentle nudging from @sellmair, we’ve figured out how to correctly configure relatively complex library modules. Over the next few days we’ll be publishing HMPP library versions. Currently published:

• TestHelp: 0.5.5
• Sqliter: 1.0.9
• Stately: 1.1.10

SQLDelight HMPP and M1 Mac Arm support is merged and hopefully launching soon. Possibly Koin and others, although I haven’t checked in for a while…

Our logging library, Kermit, is getting HMPP-related changes, but is also going through a fairly significant overhaul focused on performance and features. You can try a preview version 0.3.0-m1. The api itself has changed somewhat, but should be compatible for most uses.

Also…

M1 architectures are a welcome addition. All of the libraries mentioned above have mac arm architecture targets published.

There’s direct support for XCFramework. We’ve spent a fair bit of time helping clients integrate KMM into their production build and CI environments, and demand for Swift Package Manager is significant. Streamlining XCFramework is critical for that. We’ve had some productivity issues using various external solutions and have started experimenting internally. Hopefully direct support will improve that situation.

Touchlab?

The only work we do now that isn’t KMP is for projects we’ve had for years. We are all-in on the tech, and hiring. If you want to jump into KMP with both feet, think about coming to work with us.

I am also interested in talking to Dev Rel people. We’re not quite hiring for that yet, but looking into it. Would love to get some feedback on what we’re thinking. Please DM me if you’d like to chat.

🐶 and KMP. Doing some Kotlin/JS work too!

🤞 We’re going to switch it on for the refreshed Droidcon app. In person events or not, we’ll publish the app anyway, just for fun.

🚀 Some. Not everybody. It’ll take some time to work out issues, but somebody has to go first, right?

Here’s KaMPKit updated to use SQLCipher

touchlab-lab/KaMPKitSQLCipher

The cipher key is hardcoded in code. Do *not* do that in your app. You need to get your key from a more secure source.

Here’s a video showing edits on KaMP Kit:

SQLiter ran into an issue setting the key for SQLCipher. That apparently used to work, but stopped at some point. There was a PR to fix that, but that was not merged in before 0.7.0 was released. 0.7.0 is used in the current version of SQLDelight, so you’ll need to force this fixed version.

sourceSets["iosMain"].dependencies {
    implementation("co.touchlab:sqliter:0.7.1") {
        version {
            strictly("0.7.1")
        }
    }
    //etc...
}

We also had an issue with our custom cocoapods fork, so if by chance you’re using that, you’ll need to use the latest, which is version 0.12.

To configure your SQLDelight driver for native, you need to supply a custom DatabaseConfiguration for SQLiter.

val schema = YourDb.Schema //SQLDelight Schema
val configuration = DatabaseConfiguration(
    name = "YourDbName",
    version = schema.version,
    create = { connection ->
        wrapConnection(connection) { schema.create(it) }
    },
    upgrade = { connection, oldVersion, newVersion ->
        wrapConnection(connection) { schema.migrate(it, oldVersion, newVersion ) }
    }
    ,key = yourCipherKey
)
val driver = NativeSqliteDriver(configuration)

To get this all to work in iOS, as discussed in Sam’s post, disable sqlite linking:

sqldelight {
    database("YourDb") {
        packageName = "co.touchlab.yourapp.db"
        linkSqlite = false
    }
}

Then add the SQLCipher dependency (if using cocoapods)

cocoapods[ext] {
    summary = "Shared code for YourApp"
    homepage = "https://github.com/touchlab"
    pod("SQLCipher", "~> 4.0")
}

Remember, if you add pod to your Gradle config, you need to build from Xcode and not the Gradle command line. I always forget this.

That *should* be it. If there are other missing details, let me know and I’ll update.

The story of coroutines for Kotlin/Native has been complex for a while now, and with the news that the Kotlin/Native memory model will be changing, coroutines for Kotlin/Native will remain a complex story for some time.

Ktor is built around coroutines. The coroutines situation fundamentally affects how Ktor functions. There have been multiple moving targets in this story, which has created a lot of confusion and a number of bug reports that kind of cross over the same core issues.

In this post, I will attempt to add some clarity to the situation.

The Coroutines Situation

To be clear, when we’re talking about “coroutines”, we’re generally talking about kotlinx.coroutines. It’s technically a separate library, but I can’t imagine a realistic situation where you’d actually use coroutines and not include kotlinx.coroutines.

Initially, kotlinx.coroutines for Kotlin/Native was single-threaded. You could suspend execution, but not schedule work on different threads. This was due to Kotlin/Native’s memory model.

Ktor was designed to run on this single-threaded model. Also, as far as I know on iOS, you would need to initiate Ktor calls from the main thread.

In late 2019, a draft PR emerged which allowed kotlinx.coroutines to communicate across threads. We’ll call this the “MT” branch, for “multithreaded”. From this branch, there have been parallel library releases. The current main release is ‘1.3.9’ and the current multithreaded release is ‘1.3.9-native-mt’.

Unless something changes, the parallel branch and release model will likely stay the same for kotlinx.coroutines until the Kotlin/Native memory model update is complete.

Ktor

Until 1.4, Ktor was still designed to be run with the single-threaded version of kotlinx.coroutines. You *could* run it with the MT version, but you had to be very careful to keep the scope you were in to the main thread and make sure it was never frozen.

It’s a little complicated to explain, and not super useful as it’s now history, but here it is. Coroutines run in a scope, which has (among other things) a Dispatcher and a Job. When you change the Dispatcher to run code on another thread, that scope gets frozen. Once frozen, everything the Job holds will also be frozen. So, if you were doing something on a background thread in the same scope as something you were doing with Ktor, even if Ktor was only in the main thread, you’d get an InvalidMutabilityException.

I think one of the more confusing parts is that Ktor would work until you involved that scope in a background process, after which it would stop working.

To see this in action, watch the video.

To get around this, you either need to maintain a different scope for Ktor, and make sure it never got frozen, or isolate the Job. We did that with KaMPKit. It was a pretty ugly hack, but it worked.

Also, since Ktor depended on the non-MT version, you could unintentionally be using the non-MT version of kotlinx.coroutines even though your dependency configuration explicitly asked for it. You would need to specify that you wanted to use that version.

implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.9-native-mt") {
    version {
        strictly("1.3.9-native-mt")
    }
}

Kotlin and Ktor 1.4 release

The 1.4 release of Ktor actually fixes this issue. We didn’t hear about it for a bit, but I think that was because the issue was confusing to a lot of folks. Although it’s fixed for the MT branch, the situation has some further confusion, as the current release of Ktor pulls in the MT branch of kotlinx.coroutines. If your project expects the single-threaded version of kotlinx.coroutines, this situation can result in some unexpected behavior.

To experiment with Ktor in the different versions, you can check out our little sample app.

touchlab-lab/ktorcoroutines

The master branch has 1.3.72 and shows the failure that was happening. The kotlin_140 branch has the updated versions and a functional Ktor sample.

Issues

Ktor needs to be run from the main thread. There are also issues with Ktor and testing, related to the threading restrictions. We’ll be digging into this more at some point. Just be aware of it.

Also, Russell tells me there’s a logging issue which I’m not even going to dig into right now because I want to publish this today…

TL;DR

Ktor and kotlinx.coroutines were not working well together on native before 1.4. Now they cooperate better. Using Ktor 1.4 will possibly pull in the MT branch of kotlinx.coroutines, so if your project is not designed for that, be aware that it’s possibly an issue.

We’ve updated KaMP Kit to remove the hack we had put in. Ktor is now just regular Ktor

GitHub - touchlab/KaMPKit: KaMP Kit by Touchlab. A collection of code & tools designed to get your mobile team started quickly w/Kotlin Multiplatform