This post isn't meant as a request for pity, or linking to a GoFundMe. The purpose of this post is to show some of the behind the scenes of what it takes to put on a conference and specifically, the financial cost in doing so.

When the tickets for the ServerSide.swift conference went on sale a few weeks ago I got a few comments about the price increases. In truth, the price increases should have happened last year but that's an (expensive) lesson for me to learn. I didn't budget correctly and it left me making a loss of around £25,000 😳!

The iOS world (and for better or worse, the server-side Swift world) is a strange place for conferences. Almost every single conference out there is predominantly based on community, run by community members for community members. No one is getting into running conferences in the iOS world to make money. The result - a huge number of amazing conferences that are a joy to attend! We're so incredibly lucky to have a lot of conferences that are relatively cheap to attend and incredibly welcoming. (The mass of European conferences compared to the rest of the world is an interesting trend which I won't delve into).

Compared to the wider tech world, iOS conferences are definitely an outlier. Most other tech conferences are a lot more expensive, regularly running into 4 figures. Because they're expensive to run! So to demonstrate, here are the rough cost breakdowns for 2024:

Venue

The venue hire was £27.5k for both the days. In our case that thankfully includes security staff to let people in, direct everyone, fire safety, cleaners, everyone it takes to make the venue go smoothly. Also in our case, we have to use suppliers that have a working relationship with the venue. This is primarily to make it easy to coordinate everything and work amongst the venue restrictions that has limited access, certain technical requirements (like an IMAX screen) and is also open to the public usually.

It does however mean the competition is not as deep and we can't just order 200 pizzas from the local pizza joint. The cost also includes things like early access on the first day for load in for the suppliers and the team to ensure we’re ready with breakfast for when doors open.

Catering

The biggest single expense for the conference is catering, which in 2024 cost about £30k, around £100/person/day. £200 of the ticket cost is already eaten up (no pun intended) by the catering cost. And we know to guesstimate numbers 6 weeks in advance so they can ensure they have everything ready. And that's a huge amount of money! But it does include food, chefs, tables and all the staff to serve everything, which is not an insignificant cost.

AV

Finally the third big expense is the AV team who handle cameras, recording, interfacing with the IMAX, microphones, mix desk etc. and all the people required to make sure the tech aspects of the conference run smoothly. The cost for them last year was about £12k.

Misc

And then you have all the expenses you might not consider - T-shirts (£2k), hotels for us and speakers (~£4k), travel for speakers (~£3k), speakers dinner (£500), and then all the little things you forget like site visits, stickers, lanyards and badges (which can get expensive!), tubes to travel around between venue and suppliers, even insurance to cover the conference if someone gets hurt during the conference.

London

Now some of the expense is plainly because we're in London, one of the most expensive cities in the world. We’re lucky having a great venue which is significantly cheaper than some of the other quotes I received, because it’s a public museum.

We could move to another place, but conferences need to be easy to get to usually so people can travel to it and hosting it in London where there are already a lot of companies makes it easier for them to send people. Which means more tickets sold. So it’s a balancing act of where to host the conference. So we could choose an area with cheaper costs but chances are the numbers would take a significant hit.

Total Bill

So the total bill was about £80,000. A blip compared to the likes of Google IO, WWDC or re:Invent but as an individual organising it more than a well-paid job! With tickets ranging from £250-£425 last year, which translated to about £220-£320 of actual income once you factor in VAT and Tito/stripe costs. That’s a lot of tickets to sell to break even. I’m very lucky to have incredibly supportive sponsors but it’s easy to see how you can make a loss on a conference. And remember, I don’t take a penny of income out of the conference, despite the many, many hours of time dedicated to it. If I can break even then it’s a good advertisement for me, which is the business justification for running it. It should in theory drive more consultancy to Broken Hands.

What Can You Do?

So what can you do? Support your community conferences! Buy tickets. Encourage your company to use training budgets to send staff (it’s far cheaper than a recruiter). Consider sponsoring—there are options for every size of business. Whether you’re recruiting locally or promoting a product, it’s one of the most cost-effective ways to support the community you benefit from. We have a lot of really awesome conferences and as someone who’s been to almost all of them, they are all great.

The Rewards

Finally, I should mention the flip side - the rewards of running a conference are immense. The buzz around the conference and seeing your hard work come together and people enjoying something you put on is a great feeling. The connections formed, the friendships, and the knowledge shared are all worth it. Seeing people leave inspired, with new ideas and new friends, is what makes it all worthwhile, which is why me, and many others, keep doing it year after year.

Running a conference is not an easy task, as I keep seem to forget! This year's ServerSide.swift was the 4th edition of the conference and the 2nd time in London. We returned after a long break with a new date and back up to 2 days. Both of these changes definitely paid off and gave the conference a much improved vibe.

The September date provided a much warmer London and less clashing with Thanksgiving and the holiday period. The 2 days allowed us to spread to talks out and provide longer breaks making the conference feel a lot less rushed and provide lots and lots of networking time. I think this was the biggest change of the conference and definitely the one I got the most positive feedback on.

Overall, the feedback was extremely positive. The quality of the talks this year was fantastic with a wide range of talks. We had several success stories that have prompted blog posts, call outs to the community that have results in new AWS documentation, and talks on the fundamentals of the ecosystem that will serve as reference guides for Swift on the server for the years to come. It was fantastic to see so many folks from Apple attend, with engineers from across the organisation. It felt like the in-person WWDCs of old! The social events were packed (we will get more space next year!) and overall the feedback, both in the feedback forms and online was fantastic.

We also got the videos out in record time! Actually having the screen recordings meant that I could power through the videos in a couple of days and they all ended up on Youtube the week after. You can view all of the videos from this year's conference here. And yes, the videos from 2022 are still being worked on as and when I get time.

Next year is already in planning. Expect a similar time of year, the same venue and the same single track conference over 2 days. I would like to bring the panel discussion back if I can make the logistics work. The other thing that I would love to bring back would be the day of workshops and/or office hours. These have worked really well in the past and I'm actively looking into if we can make them work next year, both logistically and financially.

Speaking of finances, I still have a long way to go to make the conference financially viable. Anyone who has attempted to run a conference can tell you they are not cheap endeavours! The move to 2 days, the economic climate with high inflation and less money (meaning less sponsors) meant that there was a significant increase in costs without the corresponding increase in revenue, partly caused by ticket prices being kept close to last year's price. However, I'm hoping over the next year or two we should get to the position where we're at least break event. I'm eternally grateful for the support of this year's sponsors. Without them, the conference would not run at all.

Running a conference is an extremely stressful, time consuming, and rewarding experience! I'm really pleased with how this year turned out and thank you to everyone. The sponsors, those who gave up their time to speak, the orange minions who were running round all conference making sure everything ran smoothly and to everyone who attended. I want to make the conference the conference to attend and with the talks, the unique topics, the people who attend and the venue I think we're well on the way to achieving that goal. I'm looking forward to next year and can't wait to see how it grows. I hope to see many of you there and we should have some dates for diary in the coming weeks.

This is just a short post before a bigger conference retrospective but it makes more sense to get this out first.

You may have noticed that the videos from this year's conference are already up here! It's amazing how quick the edit goes when you have all the slides recorded alongside the cameras! I spent more time encoding the videos than I did in the edit.

This also means that they've beaten the videos from 2022. A few videos have been released but it takes about 15 hours per video to re-record the slides, chop them up, put each slide and transition in at the right time and then finish the edit. So finding time to prepare them amongst everything else that's going on is challenging. Combine that with running a business, maintaining an open source project, organising another conference and a healthy dose of burnout and it's taking some time.

However, I am still working on them, and will keep churning through them until they're up.

ServerSide.swift 2024 recap to follow

Hi everyone! 👋

I know there have been lots of questions and enquiries about the videos from last year's conference, and lots of (justified) criticism about the delay in releasing them and lack of communication. I wanted to give you an update on what's happening and what the plan is.

First, an apology. It has taken way too long to get any videos out and the lack of communication and transparency has only made things worse. Despite all the issues, there's no excuse for the overall delay and I'm sorry for that.

The videos have been...challenging in many aspects. To start with, there was a miscommunication between the AV production team and myself which meant that the on-screen slides were not recorded - you know the actually useful part. This means that I need to rerecord all the presentations with the slides from the presenters and go through and ensure the right slide is shown on screen at the right time. Not only is this a massively time-consuming task, it meant it was impossible to outsource the editing as you need someone who understands the context for the presentations.

The next issue was that the encoding format used by the cameras was unknown to anything other than VLC when you attempted to play them back. I eventually worked out the incantation required to make FFMPEG convert them to a usable format, but it took a lot of trial and error to get there and a lot of time to transcode them all. Then the editing software I use refused to import the video files, so there has been a lot of back and forth with the developers and support team to try and find a way to import the videos.

This combined with an extremely challenging year personally, and the current economic climate which meant I had to spend more time into trying to find work to pay the bills, meant the videos got pushed down the priority list. (Anyone who goes into conferences to make money is in for a shock 😂 - the ServerSide.swift conference is extremely lucky to have amazing sponsors every year to ensure that the conference is viable but we still didn't come close to breaking even last year.)

However, I'm now in a position where all the problems are solved and we can start producing the videos so you should start to see them appear on Youtube in the next couple of weeks. I'll post an update with progress every few weeks until they're all updated (hopefully it will only require one more update).

The next conference

I've also had lots of questions this year about the next edition of the conference and when it will be happening. There will not be a ServerSide.swift this year, however I am planning for one early next year. December is difficult time of year to hit. Not only do you have to compete with Christmas, where many people are away or traveling and many things are much more expensive, you're competing with a lot of other conferences. It's also harder to get sponsorships and for companies to send people as many companies want those budgets decided upon early in the year and they're frequently used up by December.

So I'm planning the next edition of the conference to be sometime around February or March next year. So really only a few months delay! Hopefully there will be more to announce in the coming weeks and months.

Hopefully this should provide a bit of much-needed transparency to ServerSide.swift and if you have any questions fell free to reach out on email, Twitter, Discord etc.

Tim

Nearly two(!) years ago, I wrote about editing Swift packages in Xcode. In this post I'll show you how to edit packages in Visual Studio Code using the awesome Swift for VSCode extension. This extension provides the ability to solve the same issues described in the Xcode post, and arguably in a nicer way.

Swift for VSCode

The Swift for VSCode extension is a community extension maintained by the Swift Server Work Group. It provides all of the functionality you'd expect for a language extension in VSCode, including autocompletion, syntax highlighting, debugging support, test support and many more features. It's an incredible extension that provides a first class development experience when using Visual Studio Code. You really should try it.

When you open a project in VSCode that contains a Swift Package, you'll see something like:

This is just a basic app built from the Vapor template, but has a number of dependencies. You can see the dependencies in the PACKAGE DEPENDENCIES section in the explorer, much in the same way as Xcode displays them. As in the post on Xcode, there are many occasions where you may want to work on packages side by side or edit dependencies.

Editing your dependencies

In Xcode you need to do a bit of a dance by dragging directories into a workspace. It also has a few quirks such as problems caused by adding the dependency to the wrong level (which is an easy mistake to do), or accidentally clicking the Move To Trash button instead of Remove Reference when you're done and having to go rescue the dependency from the trash.

In VSCode, editing packages is much simpler. Right-click on the package you want to edit in the PACKAGE DEPENDENCIES and choose Use Local Version:

Then find the directory containing the package you want to edit and click Select in the dialog. You'll see the package change to your local one:

You can make changes in the local version and use then straight in your project!

Multiple Packages in a Workspace

One really neat feature of the Swift for VSCode extension is being able to have multiple packages in a workspace. You can add as many packages as you want and the extension will automatically generate tasks for building, running and testing each package.

Whilst you may just want to try running your package with a local dependency, the chances are you want to actually work on the local package at the same time. The extension makes this really easy. Right click on the local dependency in the PACKAGE DEPENDENCIES view and click Add To Workspace. The extension will automatically convert your project to a workspace with the two packages in:

You can run the tests for each package in the Test Explorer:

And you can build the different packages in the build menu (CMD+SHIFT+B):

This does require any configuration, the extension does it all for you.

Tidying Up

Once you're done and want to switch back to the remote dependency, you can right click on the package directory and click Remove Folder from Workspace. Then in the package dependency view, right click on the local package and click Revert To Original Version:

And that's it! It's really great to see how well the extension is now working and I'm using it more and more as my daily driver. If you have any ideas about what other features should be added, feel free to open an issue on GitHub.

In this post I'll show you how to replicate data from your DynamoDB tables to Elasticsearch using AWS Lambda, Swift and Soto.

I've been heavily using DynamoDB over the last few months and it's great! For our use case where we have fairly complex queries but we know all the queries we want to do up front, it's fast and efficient no matter how much data we try to query. (This is commonly known as OLTP - or online transaction processing). However, when you need to query data using random access patterns, such as for analytics, it sucks. It's expensive because you have to scan the entire table and it's slow. (This is also know as OLAP - or online analytical processing).

One of the clients I've been working with at the moment are building an online grocery store. Using DynamoDB for storing all the grocery items is a great fit. We know that we want to get all the items for a department, get single items, get all items for a category etc. Setting up our DynamoDB tables is relatively simple and our queries are fast, no matter how many items we have in the database.

As an aside, getting to the point where 'setting up our DynamoDB tables is relatively simple' is a steep learning curve. The whole thought process behind single table design, index overloads and composite keys is completely different to the SQL world. Here's a great post with some pointers and re:Invent videos to help. I'll likely have more to say about DynamoDB in a later blog.

The replication problem

Whilst using DynamoDB for the majority of our queries is great, you'll eventually hit a roadblock any time you want to do search or analytics. For our grocery store items, we want to be able to provide a good search experience to users and allow any future analytics queries to find top performing items etc. DynamoDB is rubbish for doing any kind of search across a table — it's simply not designed for that use case.

Thankfully, Elasticsearch exists. Elasticsearch is a powerful search engine that provides analytics and search capabilities across different indexes. It allows us to provide complex search patterns, fuzzy matching and even autocomplete to users when searching for items, categories or departments.

So how do we get the data from DynamoDB into Elasticsearch and keep it up to date? We do this with a Lambda function.

DynamoDB offers streams, which contain a replication log of every row added, modified or deleted in a table. We can then hook up a Lambda function which gets triggered whenever there's new data in the stream. Then in the Lambda function, we can transform the data into whatever way makes sense for us and insert it into Elasticsearch.

Setting up a Lambda

Almost all of our backend code for the grocery store is written in Swift so it makes sense to use Swift in our Lambda as well! The introduction of the Swift AWS Lambda Runtime makes this really simple to do.

Our Lambda project contains two modules - one for the application start point that hosts a very simple main.swift file and another that contains the application logic. Our main.swift simply looks like this:

import App
import AWSLambdaRuntime

Lambda.run { context in
    return Handler(eventLoop: context.eventLoop)
}

Doing this and splitting our our application logic into a separate module, allows us to write tests for our Handler. Next, in our App module we create a skeleton handler:

public class Handler: EventLoopLambdaHandler {
    public typealias In = AWSLambdaEvents.DynamoDB.Event
    public typealias Out = Void

    let httpClient: HTTPClient
    let awsClient: AWSClient

    public init(eventLoop: EventLoop) {
        self.httpClient = HTTPClient(eventLoopGroupProvider: .shared(eventLoop))
        self.awsClient = AWSClient(credentialProvider: .selector(.environment, .configFile()),
                                   httpClientProvider: .shared(httpClient))

    }

    public func handle(context: Lambda.Context, event: AWSLambdaEvents.DynamoDB.Event) -> EventLoopFuture<Void> {
        // Handler code goes here
    }

    public func shutdown(context: Lambda.ShutdownContext) -> EventLoopFuture<Void> {
        let promise = context.eventLoop.makePromise(of: Void.self)
        awsClient.shutdown() { error in
            if let error = error {
                promise.fail(error)
            } else {
                promise.succeed(())
            }
        }
        return promise.futureResult.flatMap {
            let promise = context.eventLoop.makePromise(of: Void.self)
            self.httpClient.shutdown() { error in
                if let error = error {
                    promise.fail(error)
                } else {
                    promise.succeed(())
                }
            }
            return promise.futureResult
        }
    }
}

This is the same code across all our Lambda functions that are triggered by a DynamoDB stream. We set up an HTTP Client, which is used by both Soto and the Elasticsearch Client. Notice the handle(context:event:) signature - we're only listening for DynamoDB events here and the Swift AWS Lambda runtime converts them to Swift objects for us, making it easy to use.

Processing events

Now that we have our Lambda handler written, we need to do something with the events we get! AWSLambdaEvents.DynamoDB.Event provides an array of records that have changed (they'll be batched up and delivered in chunks, rather than calling the Lambda function on each individual change). We loop through each record and then work out what to do with it.

Each record has an eventName enum property that can either be .create, .modify or .delete. For a delete event, this is relatively simple. We parse the primary and secondary key from the event.record and store this to send to Elasticsearch with a bulk operation.

Creation and modification is slightly different. To start, because we use a single table design architecture, the item being created or modified could be an item, a category, a department etc. So first we need to work out what type it is. Our primary keys are in the form of <ITEM-TYPE>-<ITEM-ID>, e.g. DEPARTMENT-F1BD3ED7-5068-4023-B227-433EE30E373C. The event.record contains a change property with keys and newImage that we can use to get all the information we need. So we simply check the first part of the primary key. Once we know what type it is, we can decode it into our model object.

Since we use the single table design pattern, most of our database rows are stored with helpful key names such as, pk, sk, data1, data2 etc. We take this data and then transform it into a real Department type. The DynamoDBDepartment and Department models all live in a package that's shared across all the microservices and lambdas which reduces code duplication.

A simplified version of our code looks something like:

for record in event.records {
    if record.eventName == .remove {
        guard let id = record.change.keys["pk"], 
          case let .string(idString) = id else {
            return context.eventLoop.makeFailedFuture(
                HandlerError(message: "Failed to get ID from \(record)"))
        }
        do {
            let operation = ESBulkOperation<GroceryItem>(
                operationType: .delete, 
                index: index, 
                id: id, 
                document: nil)
            operations.append(operation)
        } catch {
            context.logger.warning("Failed to handle delete event - \(event)")
        } 
    } else {
        do {
            let item = try convertToItem(from: record.change)
            if record.eventName == .modify {
                let operation = ESBulkOperation<GroceryItem>(
                    operationType: .update, 
                    index: index, 
                    id: item.id.uuidString, 
                    document: item)
                operations.append(operation)
            } else {
                let operation = ESBulkOperation(
                    operationType: .create, 
                    index: index, id: 
                    item.id.uuidString, 
                    document: item)
                operations.append(operation)
            }
        } catch {
            context.logger.warning("Failed to handle event - \(event)")
        } 
    }
}

Transforming Data

All of our data is already pretty denormalized being stored in DynamoDB. But there are some instances where we want to change it before we store it in Elasticsearch. One example of this is when we store grocery items. To make both search better, and presenting results to clients better when search results are retrieved, we store the department and category names in the item, as well as their IDs. So when we get a grocery item in the event record we need to fetch this data from DynamoDB. To do this we use Soto. Soto is a 3rd party SDK for AWS that contains everything you'd ever need to interact with AWS. It's built on top of SwiftNIO and AsyncHTTPClient so integrates well with both Vapor and the Swift AWS Lambda runtime.

Using Soto, we perform a query to get the department and categories for the item and then populate our GroceryItemSearchResult model, ready for insertion into Elasticsearch. We also use Soto to decode the newImage keys from the event record into our DynamoDB model types.

Inserting into Elasticsearch

Once we've performed all the above actions, we have an array of items to delete, an array of items to insert and an array of items to update. To do this we use the Elasticsearch NIO Client, which uses AsyncHTTPClient to make calls to the Elasticsearch API.

All of the changes are wrapped up in the ESBulkOperation type so we can send them in a small number of requests, rather than overloading our Elasticsearch instance when we do a large inventory import. We store the items and departments in a index specific to that particular store. This makes searching inside a store easy to do. We use the ID of the item, category or department for the ID in Elasticsearch, instead of allowing it to create an ID for us. This makes updating items simple as we just overwrite the item at that ID with the new information.

Finally we call elasticSearchClient.bulk(itemOperations) for the different model types and all our data is replicated into Elasticsearch. The Elasticsearch NIO Client also uses Soto because our Elasticsearch instances are hosted in AWS using their managed service. This means that all our requests to Elasticsearch need to be signed. The Elasticsearch NIO Client uses Soto to sign each request so it's transparent to our application (which makes testing easy as well).

Our simplified code above would look something like:

elasticSearchClient.bulk(operations).transform(to: ())

This sends our operations to Elasticsearch and returns EventLoopFuture<Void> to the Lambda runtime so it the stream knows all the records have been processed successfully. If any errors are thrown, then the DynamoDB stream will retry for 24 hours until they are successful.

Wrapping up

Hopefully this post has explained how we replicate our DynamoDB data into Elasticsearch and keep the data up to date. You can use similar Lambda functions to duplicate DynamoDB data into Kinesis Firehose or into any data lake for analytics processing or ML applications.

Further aside - Kinesis Firehose can automatically insert data into Elasticsearch for you and throttle the insertions if you're ingesting more data than Elasticsearch can currently handle. Unfortunately we can't use that because you need a different firehose stream for each index and they need to be known up front. Our indexes are based on stores, which can be dynamically added which is why it's set up the way we have it.

Our Lambda code is pretty specific to our data transformation needs, which is why I haven't shared more of it here. But this should give you a good start to get going. And by using Swift, sharing model code and using libraries to do some of the heavy lifting for you, it can be nice and simple!

Swift 5.2 brought some awesome changes to the package manager thanks to SE-0226 that massively improved the handling of dependencies. Going forward no longer would you face the spinning resolution of doom if you had dependency conflicts. And no longer would you have to download all transitive dependencies if some were only used in testing of your dependencies.

This is great, so you take your Swift 5.1 project, open the Package.swift manifest and edit the first line to read

// swift-tools-version:5.2

Easy. You then open the project in Xcode, or run swift run and are suddenly faced with:

Wait, what. To be fair, at least SwiftPM now offers advice. Earlier versions of Swift 5.2 would just say "Unknown dependency Vapor". So what's going on here?

The Manifest

To understand why we're seeing these errors, we need to understand the three parts of a package SwiftPM uses for resolution:

• The name of package. You define this in your manifest using the name parameter of the Package initialiser. E.g.

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "ConsumerWebsite",
    // ...
)

• The name of the target, defined when you declare a target:

.target(
    name: "App",
    // ...
)

• The final path component in the Git URL.

Swift Package Manager's Dependency Resolution

SwiftPM uses a combination of the package name, target name and Git path when resolving dependencies. Note that it's also case sensitive, which can trip you up. This produces four different scenarios.

Everything matches

In the ideal world, the dependency you're pulling in has a target whose name matches the package name. The package name also matches the Git path (in case as well). A good example of this is Vapor Security Headers. It defines a single target named VaporSecurityHeaders in a single package named VaporSecurityHeaders. The URL is also VaporSecurityHeaders.git. Because everything matches, you can include it as a dependency the old fashioned way:

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "DependencyExamples",
    dependencies: [
        .package(url: "https://github.com/brokenhandsio/VaporSecurityHeaders.git", from: "3.0.0")
        // ...
    ],
    targets: [
        .target(
            name: "App",
            dependencies: [
                "VaporSecurityHeaders",
                // ...
            ],
        ),
        // ...
    ]
)

Different Git URL and Package Name

Unfortunately, some repositories don't match the name of the package. Most use a kebab- or lisp-case for the URL and then CamelCase for the package name. In this case, you need to qualify the name of the package in the dependencies array. For example, if you integrate Vapor CSRF, you'd do something like:

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "DependencyExamples",
    dependencies: [
        .package(name: "VaporCSRF", url: "https://github.com/brokenhandsio/vapor-csrf.git", from: "1.0.0")
        // ...
    ],
    targets: [
        .target(
            name: "App",
            dependencies: [
                "VaporCSRF",
                // ...
            ],
        ),
        // ...
    ]
)

Notice how you specify the package name in the dependencies array using .package(name:url:from:). This is because the package name is VaporCSRF but the Git URL is vapor-csrf.

Different Package Name and Target Name

A recent convention (at least in the server-side Swift world) seems to be using kebab case for package names. This is following what SwiftNIO does. However using kebab case for a target name would mean you end up doing import swift-nio, which Swift doesn't support (and looks weird!). So you using camel case for the target name. So you need to qualify which package the target comes from in your target's dependency array. For example, when including Vapor, it would look like:

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "DependencyExamples",
    dependencies: [
        .package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),
        // ...
    ],
    targets: [
        .target(
            name: "App",
            dependencies: [
                .product(name: "Vapor", package: "vapor"),
                // ...
            ],
        ),
        // ...
    ]
)

Notice how you specify where the target Vapor comes from with .product(name: "Vapor", package: "vapor"). Vapor's package name is vapor, which matches the Git URL. But the target name is Vapor so it needs specifying. This would also be the case if a dependency exposes multiple targets in the package. If you want to depend on one where the name is different to the package name, you must specify where the dependency comes from.

Nothing matches

The final situation is where nothing matches. The Git URL, package name and target name are all different. I haven't actually seen any examples of this in the wild. If you do come across this, you need to combine the two different qualifications, like so:

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "DependencyExamples",
    dependencies: [
        .package(name: "MySuperPackage", url: "https://github.com/brokenhandsio/my-super-package.git", from: "1.0.0"),
        // ...
    ],
    targets: [
        .target(
            name: "App",
            dependencies: [
                .product(name: "MySuperTarget", package: "MySuperPackage"),
                // ...
            ],
        ),
        // ...
    ]
)

This this case, the Git URL is my-super-package, the package name is MySuperPackage and the target name is MySuperTarget. To help the dependency resolver work it all out, you need to qualify everything.

Where to go from here

Hopefully this post helps you understand some of the problems you get when adding dependencies and how to resolve them. You might also find it useful when writing your own libraries to make it easy for people to consume them and write your docs correctly!

The changes were made in Swift Package Manager to allow for better dependency resolution. This helps avoid the resolver getting stuck in dependency hell and never returning. It also means you don't need to build large packages that your dependencies rely on just for tests. Finally, it sets SwiftPM up for future improvements, like dependency caching with the new Llbuild2 build system. I'm looking forward to that coming!

I've just released a new Vapor library - Vapor CSRF. It provides a simple way to protect websites and web forms from CSRF attacks. The project I'm currently working on is the first (very) large project I've done in Vapor 4 so I'm spinning little parts of either into open source libraries like this one or interesting blog posts!

CSRF Attacks

A CSRF (or cross-site request forgery) attack in simple terms is where an attacker tricks a user into making requests that a web application accepts. Imagine a bank website that has a POST request to transfer money into an account. If a malicious site can force the user to send that POST request (when they're logged in) then an attacker could trick a user into transferring money.

CSRF tokens protects against this by ensuring the POST request is legitimate. The website provides a token to the GET request which is then checks when handling the POST request to ensure it matches.

There are other protections you can add to your site as well to mitigate this type of attack. One of the recent protections is SameSite cookies. This is where the browser won't present a site's cookie unless the referrer request has come from the same site. In practice this is more complicated to implement with a good user experience as it has the habit of making it appear like your users aren't logged in when they first visit your site. However it's still a worthwhile protection to investigate adding. Note that not all browsers (especially older ones) support the SameSite attribute.

The CSRF library

The Vapor CSRF library aims to provide a really simple solution using the common technique of CSRF tokens. When you make a GET request to a form, you provide a unique token to be submitted with the form. In the POST request you verify that the CSRF token is provided and matches the expected value. In Vapor this translates to a single call to generate and save a CSRF token to provide to your HTML forms and either a middleware or single call to verify the token.

Check out the GitHub repo for usage details. And feel free to open PRs and issues, I've already had someone offer to write a custom Leaf tag to add the CSRF token as a hidden form input!

Xcode 11's Swift Package Manager integration is subjectively awesome. Especially for people creating and build SwiftPM packages and projects. However there are two main pain points - working with multiple packages at the same time and editing your dependencies.

Working with multiple packages

When you're building a complex app or library, the chances are you'll be working on more than one Swift package. You might have a package that contains models you share between iOS and the server and the main server project. Or you might be building a blogging engine that consists of the main engine, the repository implementations and an example application that integrates everything.

Working with individual projects can be difficult if you're developing everything together. Let's say you want to add a new API that performs a different database operation. You need to add the new route to your engine, and a new function to the Repository protocol and write the tests for that. You then need to commit, tag and release that and hope it works in the way you want with a real database. You then need to go into the repository implementation and update to the latest version. Then you must implement the new protocol requirements and commit, push and tag and release that. Finally you can go into the main application and join it all together to actually use your new route! However, more than likely, you made a mistake or didn't consider a use case that's come up. To rectify this, you have to go and do the whole dance again. What you want is to be able to work on all three codebases at once.

Editing your dependencies

If you're working on a Vapor app, you may come across a bug, or missing feature in one of the new Vapor 4 release candidates. How do you go about adding the new feature or fixing the bug if Xcode won't let you edit the code? You could fork the repository, clone and download it and work in there. But then you're left in the same situation where you're not sure if it works in the real world. Again, you want to be able to work on both codebases at once.

Swift Package Workspaces

Thankfully, Xcode already has a solution for this - the trusty workspace! In Xcode, create a new, empty workspace. Once created, find the directory for the package you want to work on and drag that directory into the project navigator in Xcode. That will create a Swift Package in Xcode and download the dependencies as normal.

However, now drag another package you're also working on, such as one of the dependencies into the project navigator. This replaces the existing project's cloned dependency with your local copy. You can now work on the packages side-by-side, commit and push when needed and generally have a better time!

Note: there is currently an issue or feature in Xcode where it doesn't expose all the schemes for all packages you're currently working on by default. You may need to add these manually. I've raised this as FB7537293.

When you're finished, just delete the package you no longer want to work on. Xcode will detect the change and automatically switch back to the remote dependency. Make sure you select Remove Reference instead of Move to trash otherwise you'll delete your local copy of the package!

This is a post I've been meaning to write for a very long time. I'm conscious that I'm completely biased when it comes to this topic. I have a large amount of potentially misplaced optimism and a vested interest in seeing Swift on the server succeed. But I also believe that I have more insight than most given I work full-time on it. I'm also expecting to generate a lot of, uh, discussion with this post - so hey Twitter, Reddit and Hacker News 👋

I first thought about writing something when coming back from the ServerSide.swift conference. For those who haven't heard of it (and full disclosure, I'm one of the organisers) ServerSide.swift is an international conference for Swift on the Server (no surprises there). It's the one place where everyone from the community can come together because of the focus on the server. It's a great place to find very technical talks like gRPC and testing NIO systems. It was an awesome conference, an amazing line up of speakers and there was a real buzz around the community. Having recently quit my 'proper' job to work on server-side Swift full time, I was feeling pretty pleased with it all!

IBM and Kitura

Then a month later, IBM quietly announced they were stepping away from Swift as one of their open source languages. It was a huge blow for the community, especially everyone who was using Kitura and invested in it. Added to that, the way that IBM made no formal announcement left a bad taste for many.

It was a real shame to see IBM exit the Swift on the server scene. Competition is always good, and IBM should be remembered for the amount of work they put in to make server-side Swift viable. After the announcement, there were lots of people and articles taking this as the sign that Swift on the server is dead. After all, if IBM can't make it work, why would anyone use it?

2020 - The year of the server

To be clear, I'm not saying that everything was (and is) perfect when using Swift on the server. There are teething problems and issues arising from its immaturity. Developing on Linux, crash handling, lack of OS support, debugging, SwiftPM features - all have provided bumps in the road and issues. And up until quite recently, if you didn't want to get your hands dirty so to speak, I wouldn't have recommended using Swift on the server.

However, since January there have been major developments that show real promise for the future. First, towards the end of January, Ted Kremenek posted On the road to Swift 6 on the forums. And rather than being the Swift 6 roadmap, this was essentially the Swift on the server roadmap. Every single goal of Swift 6 is either related to the server or has a direct benefit for it. Things like new OS support, LSP improvements, build times and dependency management improvements. The list goes on, and I haven't even mentioned the aim for a real concurrency model, which for me is the last missing piece for Swift on the server.

To signify the importance of all this, the Swift core team got two new members. Tom Doron - who is responsible for SwiftNIO and the Swift Server Work Group - and Saleem Abdulrasool - who almost single-handedly has dragged Swift onto Windows. Having both on the core team signify the importance of Swift outside the Apple ecosystem to the Swift team.

Then, a few days later, Mishal Shah posted on the forums to announce that there would be nightly Docker images available for Swift. While this doesn't affect most developers, it is important. It allows you to test changes in Swift and Foundation without having to either compile Swift or install a new toolchain. So framework developers can now test code against pre-release versions of Swift without much effort.

It also builds on the monthly releases of Swift for Linux that have been happening for over a year now. Originally started with Swift 4.2.2, it opened the way for getting fixes for Foundation quickly rather than waiting for the next Xcode release. And amazingly, this was a case of Linux getting quicker Swift releases than macOS!

A week after the nightly Docker images announcement, Cory Benfield went on stage at dotSwift to announce the release of Swift Crypto. Crypto is a part of any server ecosystem that you really want to have. Rolling your own Crypto is a bad idea and linking against OpenSSL was causing issues. It was difficult to integrate with OpenSSL from all the different operating systems. And relying on the system-provided OpenSSL would stop Swift from supporting HTTP/2 and HTTP/3 in any reasonable timeframe.

Swift Crypto still isn't without its issues, which is to be expected. Currently Swift Crypto doesn't support either RSA keys or PEM/DER keys. This is problematic in the server world for 95% of use cases! 😅 The current workaround is to embed your own copy of BoringSSL and use that yourself. (And also every dependency that needs these features to embed their own copy of BoringSSL.) Again this is due to Swift's immaturity and something that the Swift Crypto team will fix going forward.

But the good news kept coming. Earlier this month the Google Summer of Code projects for Swift were announced on the forums. Out of the four projects announced, two were directly working on improvements for server-side and Linux.

Then finally, the next day Tom Doron announced support for more Linux distributions. This is a huge step forward for Swift because it makes it easier to run Swift on the OS you're already using. Lots of companies have infrastructure and tooling built around different OSes. By constraining Swift to Ubuntu only (Docker aside), it meant these companies couldn't use it. Now they can and with support for Amazon Linux, it even opens up opportunities for things like Lambda.

To coincide with the announcement (whether by coincidence or not), Amazon release 2.0 of their Smoke Framework. It's really cool to see a company like Amazon go all in with Swift on the server. And if you want to learn more about it, check out Simon's video from ServerSide.swift:

Not just the server

Whilst everything above is great news for the server, there are non-server use cases that are starting to make inroads. The biggest is Swift for Tensorflow. Using Swift for machine learning may seem like an odd choice, and Google choosing Swift as one of their primary ML languages even odder! But there are good reasons why it was chosen. Tryolabs recently did a deep-dive into the reasons, background and benefits of this and it's a great read! And even though this may not be applicable to you (it certainly isn't to me!) it provides two big benefits.

The first is improving the language itself. The S4TF team needed to make some big changes to both Swift and the compiler to allow things like differentiable programming and Python/C++ interoperability. We're now seeing these changes being merged back upstream into Swift itself, where they can be used by anyone. Additionally, the S4TF team recently released Swift Benchmark to benchmark code. It's awesome to see great tooling for Swift developed outside of Apple!

Second, it provides a big mindshare opportunity. A major project using and evolving Swift outside of the Apple ecosystem is important. It proves to outsiders that Swift isn't 'the Apple language' and proves that the core team want Swift to be more than just the language to write iOS apps in.

There's also a large amount of work going on making Swift work on the web, but client side. The SwiftWasm project allows you write Swift and use it in the browser with WebAssembly. This could potentially mean writing Swift instead of JavaScript for websites 🎉 Although it's early days and a lot of work needs to be upstreamed back into Swift, it shows real promise and you can try it out now!

Looking forward

Things are moving quickly with server-side Swift and this year is only accelerating. I suspect we'll see more supported Linux distros and further improvements to the language. Though I don't expect Swift 6 and async/await to land this year. The release of Vapor 4 is another step forward and encompasses a lot of the work of the SSWG. The SSWG are also publishing a number of guides on best practices for building, deployment and debugging. These are definitely worth checking out.

It's important to remember that Swift is still young. It was only open sourced at the end of 2015. Go is a 'new' language, but it was first released in 2009. Rust, another language that's generating a lot of buzz first appeared in 2010. Swift is playing catch-up and you should take that into account when making comparisons. Whatever happens it's going to be an interesting year for Swift on the server. The future is definitely bright!

With the introduction on Xcode 11, Swift Package Manager finally has first class support! This makes working with SwiftPM projects in Xcode so much nicer.

However, it also means that Xcode runs the application in a randomised directory inside DerivedData. For apps that need to load resources, such as template files or .env files (hello Vapor! 👋), this presents a problem. So to work around this you can set a custom working directory in Xcode, so Vapor can work out where to look for files.

In Xcode, OPTION+CLICK the scheme (the thing next to the play button with your app name) to edit the scheme. Select Options and check Use custom working directory and add in the directory to your project path. It should look something like:

That's all you need! Now Vapor can detect env files, Leaf files and also static files in the Public directory when using FileMiddleware.

Welcome to my new site! I needed a place to post opinionated blogs about server-side Swift that didn't make sense on the Broken Hands. I also want to be able to write about non server-side Swift stuff, like the home office and home network, and some of the DIY stuff that's been going on in the garden. But don't expect too much!