For many years of having our App in the App Store, a particular bug alluded us most mischievously. Various reports would come in that distilled down to a simple problem: Customers with plenty of space on their device could not make any changes to the file system. Some of these bugs manifested as follows:

1. Unable to download new files.
2. Unable to upload new files.
3. Unable to duplicate a file.
4. Unable to save changes to an existing file.
5. Unable to delete files.

When we received these reports, we were painfully unaware they were the same bug since they manifested in seemingly unrelated parts of the App. We captured device logs and searched the internet and still found ourselves unable to understand what was happening.

So, we filed a support ticket with Apple to get an engineer’s eyes on this.

After some productive back and forth and a user who could to help us that could replicate the issue, he led us down a path that helped us get the error in question. In this instance, we looked at why UIDocument would fail to save changes to an existing document. He instructued us to override the handleError API and log the outputs.

The following error message revealed itself:

"localizedDescription": "The operation couldn’t be completed.",
"domain":"NSCocoaErrorDomain",
"code":512

Our helper at Apple pointed us to an obscure post in the forums that said the following:

In a nutshell, there is an arbitrary limit set on items in the Caches and tmp directories, that limit is 1000. Meaning, for our longtime customers, when those folders got too big, everything stopped working 🤯. Furthermore, iOS was not automatically pruning these directories as one might expect.

Our first step was to ship a build that had a deeplink to clear Caches and tmp. We sent this to users that experienced the problem and voilà, it worked! 💃Taking this a step further, we decided to write a utility that pruned these folders on each app update. We watched our errors go down to zero in no time.

It is worth mentioning that some developers have reported that this seems to be a problem, mainly when creating entries in these folders where the filename is generated using UUID().uuidString. We definitely make use of this API but if it’s affecting you too, I hope this helped you on your journey.

🧠 Moral of the story

If you find your App failing to make any changes using FileManager, look at your error code and see if it matches 512 and you have a large Caches and tmp directory.

I hope this saves you the time it took us to get to the bottom of this so that you can get your customers back and running in no time.

The curious case of FileManager’s 512 error on iOS was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Detecting changes to a folder in iOS using Swift

In this post, you will learn how you can observe a specific folder for new/deleted/updated files. Some of this code is adapted from Apple sample code that is no longer available online. The final result will look like the GIF below:

🌄 Background

Most of us are familiar with FileManager and contentsOfDirectory. In most cases, that is good enough when peering into the contents of a folder. Sometimes though, we come across situations where we want to know precisely when the contents of a directory/folder have changed.

At Over, we allow users to create project documents that live inside a folder. As the user makes edits to a project, we need to update its thumbnail and, in the case of a new project, add it to the project feed. The coordination required to manage this manually could be highly error-prone. Since cache invalidation is one of the hardest programming challenges, we need a solution that gives us the closest source of truth.

We could use Timer to monitor a folder by polling for changes but, that would incur a lot of unnecessary file system reads and overall feels wasteful.

Enter DispatchSourceFileSystemObject. This nifty GCD object monitors a given file descriptor for events. One can consult FileSystemEvent for the list of events we can monitor, but in this instance, we care about FileSystemEvent.write.

🏃‍♀️Getting Started

We begin with a demo project that displays files in a folder. We will have functionality to add and remove a file.

Our first step is to start a new SwiftUI project, and in the ContentView, add support for showing a list of files.

struct ContentView: View {
    @ObservedObject var folder = Folder()
    
    var body: some View {
        NavigationView {
            List(folder.files) { file in
                Text(file.lastPathComponent)
            }
            .navigationBarTitle("Folder Monitor")
            .navigationBarItems(leading: createFileItem, trailing: deleteFileItem)
        }
    }
}

Above, we have the definition of a view that will show a list of files and update the list on any changes. We can create and delete files and expect the UI to be updated. Ignore what Folder does, for now, that will be explained later. Just know it returns a list of files and publishes changes whenever that list changes.

📹 Monitoring a Folder

When monitoring a folder, two things are essential:

1. A file descriptor pointing at the folder we want to observe.
2. A DispatchSourceFileSystemObject that will use the descriptor to report changes. We can construct one using the DispatchSource.makeFileSystemObjectSource API.

Using them together would look like:

let folderToObserve: URL = //a folder somewhere on device
let descriptor = open(folder.path, O_EVTONLY)
let folderObserver = DispatchSource.makeFileSystemObjectSource(fileDescriptor: monitoredDirectoryFileDescriptor, eventMask: .write, queue: someQueue)
directoryMonitorSource?.setEventHandler { in
    print("Something changed in our folder")
}

In the above, we use eventMask: .write so that we only get events for file writes to that folder. This mask will give us updates that include new files, deletions, and updates (ie, an existing file has been updated).

When it comes to GCD APIs like this, it’s good practice to add a higher-level abstraction. This enables an API that is easier to consume and more customizable for your purposes.

Our abstraction will be called FolderMonitor. Before I share the code, there are some essential concepts to consider when using folder observers. Namely, we need to manage the lifetime of our objects so that we do not keep monitoring files when we no longer need to. We will cater for this by adding functionality to release the folder monitor as required.

With that in mind, this is what our monitor API should look like 👇

class FolderMonitor {
    let url: URL
    var folderDidChange: (() -> Void)?
    init(url: URL)
    func startMonitoring()
    func stopMonitoring()
}

A simple enough API to consume and understand. Now, let’s expand upon it.

First, we need references to the two essentials mentioned earlier:

class FolderMonitor {    
    /// A file descriptor for the monitored directory.
    private var monitoredFolderFileDescriptor: CInt = -1

    /// A dispatch source to monitor a file descriptor created from 
        the directory.
    private var folderMonitorSource: DispatchSourceFileSystemObject?

    /// A dispatch queue used for sending file changes in the 
        directory.
    private let folderMonitorQueue = DispatchQueue(label: "FolderMonitorQueue", attributes: .concurrent)

    let url: URL
    var folderDidChange: (() -> Void)?
    init(url: URL)
    func startMonitoring()
    func stopMonitoring()
}

Now we can implement our startMonitoring function:

func startMonitoring() {
    guard folderMonitorSource == nil && monitoredFolderFileDescriptor == -1 else {
        return
    }
    // Open the folder referenced by URL for monitoring only.
    monitoredFolderFileDescriptor = open(url.path, O_EVTONLY)
    
    // Define a dispatch source monitoring the folder for additions, deletions, and renamings.
    folderMonitorSource = DispatchSource.makeFileSystemObjectSource(fileDescriptor: monitoredDirectoryFileDescriptor, eventMask: .write, queue: directoryMonitorQueue)
    
    // Define the block to call when a file change is detected.
    folderMonitorSource?.setEventHandler { [weak self] in
        self?.folderDidChange?()
    }
    
    // Define a cancel handler to ensure the directory is closed when the source is cancelled.
    folderMonitorSource?.setCancelHandler { [weak self] in
        guard let self = self else { return }
        close(self.monitoredFolderFileDescriptor)
        self.monitoredFolderFileDescriptor = -1
        self.folderMonitorSource = nil
    }
    
    // Start monitoring the directory via the source.
    folderMonitorSource?.resume()
}

Above, you can see the usage of the monitor source to handle both file changes and also cancellation. Isn’t it neat that it will send events for any file that is updated inside the folder!

Closing the loop is easy, to stop observing folder changes:

func stopMonitoring() {
    folderMonitorSource?.cancel()
}

👟 Tying it up to our UI

At this point, we have a capable folder observer. To prepare it for SwiftUI, we can wrap it into a UI-friendly ObservableObject. In this example, all I am doing is observing the files in the Temporary Directory:

class Folder: ObservableObject {
    @Published var files: [URL] = []
    
    let url = URL(fileURLWithPath: NSTemporaryDirectory())
    private lazy var folderMonitor = FolderMonitor(url: self.url)
    
    init() {
        folderMonitor.folderDidChange = { [weak self] in
            self?.handleChanges()
        }
        folderMonitor.startMonitoring()
        self.handleChanges()
    }
    
    func handleChanges() {
        let files = (try? FileManager.default.contentsOfDirectory(at: url, includingPropertiesForKeys: nil, options: .producesRelativePathURLs)) ?? []
        DispatchQueue.main.async {
            self.files = files
        }
    }
}

This gives us an observable that uses a directory monitor and publishes the full list of URLs whenever they change.

Since our UI will be displaying these URLs in a list, we need to make them Identifiable.

extension URL: Identifiable {
    public var id: String { return lastPathComponent }
}

Note to reader: Be careful when extending foundation types like this as it can have unintended consequences in a large application. I am doing this only for illustrative purposes.

Finally, we can finish our UI Demo:

struct ContentView: View {
    @ObservedObject var folder = Folder()
    
    var body: some View {
        NavigationView {
            List(folder.files) { file in
                Text(file.lastPathComponent)
            }
            .navigationBarTitle("Folder Monitor")
            .navigationBarItems(leading: rightNavItem)
            .navigationBarItems(leading: rightNavItem, trailing: deleteFileNavItem)
        }
    }
    
    var deleteFileNavItem: some View {
        Group {
            if folder.files.count > 0 {
                Button(action: deleteFile) {
                    Image(systemName: "trash")
                }
            }
        }   
    }
    
    var rightNavItem: some View {
        Button(action: createFile) {
            Image(systemName: "plus.square")
        }
    }
    
    func createFile() {
        let file = UUID().uuidString
        try? file.write(to: folder.url.appendingPathComponent(file), atomically: true, encoding: .utf8)
    }
    
    func deleteFile() {
        try? FileManager.default.removeItem(at: folderObservable.files.first!)
    }
}

🙇‍♀️ Conclusion

You will notice there is still a lot of room in FolderMonitor for expansion. We could design it so that it returns a diff of the files inside a folder. We could even have it report changes to specific files with a specific extension. Please, feel free to take your monitor on its own journey. The purpose of this post was to shine some light on often overlooked APIs.

Thank you for taking the time to read this. Hopefully it made a difference to your iOS journey this day 💖.

Want to hear more? Be sure to follow us on Medium and Twitter @EngineeringOver

💼 Full Code

The full code for our FolderMonitor looks as follows:

class FolderMonitor {
    // MARK: Properties
    
    /// A file descriptor for the monitored directory.
    private var monitoredFolderFileDescriptor: CInt = -1

/// A dispatch queue used for sending file changes in the directory.
    private let folderMonitorQueue = DispatchQueue(label: "FolderMonitorQueue", attributes: .concurrent)

/// A dispatch source to monitor a file descriptor created from the directory.
    private var folderMonitorSource: DispatchSourceFileSystemObject?

/// URL for the directory being monitored.
    let url: Foundation.URL
    
    var folderDidChange: (() -> Void)?

// MARK: Initializers

init(url: Foundation.URL) {
        self.url = url
    }

// MARK: Monitoring

/// Listen for changes to the directory (if we are not already).
    func startMonitoring() {
        guard folderMonitorSource == nil && monitoredFolderFileDescriptor == -1 else {
            return
            
        }
            // Open the directory referenced by URL for monitoring only.
            monitoredFolderFileDescriptor = open(url.path, O_EVTONLY)

// Define a dispatch source monitoring the directory for additions, deletions, and renamings.
            folderMonitorSource = DispatchSource.makeFileSystemObjectSource(fileDescriptor: monitoredFolderFileDescriptor, eventMask: .write, queue: folderMonitorQueue)

// Define the block to call when a file change is detected.
            folderMonitorSource?.setEventHandler { [weak self] in
                self?.folderDidChange?()
            }

// Define a cancel handler to ensure the directory is closed when the source is cancelled.
            folderMonitorSource?.setCancelHandler { [weak self] in
                guard let strongSelf = self else { return }
                close(strongSelf.monitoredFolderFileDescriptor)

strongSelf.monitoredFolderFileDescriptor = -1

strongSelf.folderMonitorSource = nil
            }

// Start monitoring the directory via the source.
            folderMonitorSource?.resume()
    }

/// Stop listening for changes to the directory, if the source has been created.
    func stopMonitoring() {
        folderMonitorSource?.cancel()
    }
}

Monitoring a folder for changes in iOS was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Originally posted here.

I joined the iOS engineering team at Over early in 2018. If you’re not familiar with Over, it is a photo and video editing app that gives you the power to create visuals that will help you and your business succeed. Our iOS project’s setup is pretty much what you can expect from an iOS app: a mix of Objective-C and Swift (all new code is written in Swift, obviously), a single code repository on GitHub, buddybuild for automated builds and deployments, and a hint of fastlane for some more automation.

When I joined the team, we were using a process called GitFlow to isolate features and release changes to the App Store. While we are not a particularly big team, even eight people trying to coordinate changes in the codebase using GitFlow was daunting. We found ourselves struggling to get releases out, because we were dealing with plenty of merge conflicts and semantic code conflicts right at the end of feature work, when we were supposed to release. Fixing these conflicts resulted in a lot of extra debugging, bug fixing and testing. This made it difficult to gauge when we would be able to release and it also decreased our confidence in releases.

Whilst my first priority is to write code, I have a natural tendency to try and improve all manner of processes and GitFlow raised a red flag for me. In my previous role, the team had also worked with GitFlow before scrapping it in favour of CI. This experience made it easy for me to identify the bottlenecks that GitFlow was creating for us and how CI could help us eliminate them. I explained to our team at Over how CI avoids long lived feature branches (which are a natural product of GitFlow) and instead merges each developer’s work onto the main code branch as soon as possible. We agreed to try it out and are now busy moving towards CI.

This has improved our communication during code reviews and given the team more confidence in telling product managers when a feature will be ready. It has also allowed us to ship more features, more often, leading to better user experiences and ultimately happier users. Here’s how we have been doing it.

🥴 Our old process and the headaches it caused us

GitFlow had been the preferred method for delivery for a while before I joined Over. On the surface, GitFlow is inviting because it allows developers to do their own work without worrying about it getting tangled up with other developers’ work.

When using GitFlow, a developer’s code is kept separate from the main branch, in a feature branch, until just before it needs to be released. At this point, a release branch is created from the main branch and the feature branch is merged into this release branch. The release branch is then deployed to production. All of this seems fine until more than one developer wants to release and get their work into the shared release branch. This tends to result in last minute panic.

The ‘independent work’ benefits of GitFlow seemed alluring but it was letting us down by hiding file level and semantic conflicts until the different long lived feature branches were merged into the release branch. These issues could be hidden for weeks or even months which led to an almost impossible task of accurately being able to determine when we’d be able to ship to production.

🛠 What we did to solve this

While facing similar release issues at my previous job, I was introduced to CI by a software engineering coach. He pointed me to this blog post by Martin Fowler, which gives a great overview of the technique and how to use it. He also does an excellent job at describing how feature branches fit in with CI and why it is bad getting sucked into long lived feature branches (as used in GitFlow) in his article called FeatureBranch. This was something we needed to do urgently, and so, having experienced the benefits of CI before, I pitched the idea of trying it out during one of our informal lunch-and-learn sessions. (A lunch-and-learn is merely a chat we have, about cool stuff we know, while we have lunch 😁) The team was keen and so, we decided to try out CI for our next big release.

🚒 CI to the rescue: What it is and how it works

It is important to note that CI is a practice, not a tool.

When using CI, a developer’s code is integrated into the main branch as often as daily (sometimes even as often as every commit). The main branch is kept in a releasable or stable state and can therefore be deployed to production at any time. This is a far cry from the normal rush we experienced to integrate everyone’s work and get a release out amidst major conflicts.

There are many tools that facilitate CI (in our case it’s buddybuild), but no tool can give you CI since it is a practice that developers in a team need to adopt.

In its purest form, CI works like this:

1. You check out the latest version of the code from the main development branch onto your local machine.
2. Using your local copy, you make all the changes you require, including changes to the production code as well as automated tests.
3. After making changes, you run a build — i.e. you run the app and run the automated tests.
4. If the build and tests pass, you commit the changes.
5. You repeat steps 2 to 4 until all the required work is completed.
6. Next, you pull any newer changes from the main branch, since other people have most likely added code.
7. You rebuild and fix any compilation issues or failing tests.
8. You push your local changes to the shared code repository.
9. Finally, you run a build (and tests) on an integration machine, creating a releasable app.

If the build on the integration machine passes, you have new version of your app ready to be released.

Keeping the changes small and integrating often in this way ensures stable production code with few bugs.

Of course build failures do occur, but since everyone stays up to date with the main branch frequently, such failures never last long.

📋 How we applied CI to our project

Keeping the ‘pure’ CI scenario in mind, our team at Over needed to make some adaptations for it to suit us. Being a fully remote, distributed team, often working asynchronously, we rely on pull requests to get our code into our main branch. This provides a level of visibility that is very important for a team that is not always in the same physical space.

What is a pull request, you might ask? It is a way for developers to request that their code be pulled into a target branch. A pull request shows you all the changes between your branch and the target branch, and that’s why it is often used for code reviews.

In order to accommodate pull requests, we needed to adapt the list above by including a pull request before local changes are pulled into the main branch — i.e. before step 8 above. This is what our process currently looks like:

1. After checking out the latest code from the main branch, we create a feature branch. The goal is for these feature branches to be short lived — a day or two.
2. We make our changes on these local, feature branches.
3. After making changes we run the app and any associated tests (manual or automated).
4. We commit the working changes.
5. We repeat steps 2 to 4 until all the required work is completed.
6. We pull any newer changes from the main branch (since other people have most likely added code).
7. We rebuild and fix any compilation issues or failing tests.
8. When we’re ready, we open a pull request to merge our changes into the main branch.
9. An opened pull request results in an automated build and test cycle handled by buddybuild.
10. Once a pull request is merged, buddybuild kicks off another build and test cycle and then deploys our app.

Getting feedback about the build status on a pull request allows us to fix any broken builds immediately. You might wonder what we do with features that aren’t due to ship soon: Those get the same treatment described above. However, this is where feature flags play a crucial role. Feature flags allow us to have all our code in the same codebase, even if some code is not yet available for general consumption. We even gave our implementation of feature flags its own blog post, which you can read here.

📈 The results we saw

Less file level conflicts? Yes ✅
Less semantic code conflicts? Yes ✅
Less of a mad rush to get releases shipped? Yes ✅
More confidence in shipping our features? Yes ✅
More confidence in estimating when a feature will be shipped? Yes ✅

The above results were promised by CI and it delivered 🥁. Pun intended. We did, however, see two more results that hadn’t necessarily been expected:
• More meticulous code reviews 🧐 and
• More thorough testing 🛠️

Since the new system ensures that all our pull requests are being merged into our main development branch, there are no more chances to try and fix things later. Once a pull request is merged we deem the code ready to be shipped. This has resulted in a greater sense of accountability and clearer communication within our team.

Using CI has opened our eyes to what an effective process can do to help teams communicate more effectively, work productively and be accountable at an individual level. If you’re keen to learn more about CI, check out this article I recently read to hear the reason Dave Farley and Jez Humble declared “we can’t emphasise enough how important this practice is in enabling continuous delivery of valuable, working software“.

Thanks for reading and we hope you enjoyed this article 🤓. We would love to hear your thoughts so leave some feedback down below and be sure to stay up to date by following us on Medium and Twitter @EngineeringOver

Also, We’re hiring! 😉 so come work-at-over!

How Our iOS Team Streamlined Our Processes Through Continuous Integration was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

A while ago we wrote about starting an iOS platform team at Over (read it here). From the start, we decided to tackle various tasks that the iOS team doesn’t always have time to look at. We thought, now would be a perfect time to look back on the success of the platform team and what we have managed to achieve thus far.

The Platform team’s primary purpose is to facilitate the success of the entire iOS team at Over. This ranges from tasks like handling urgent bugs, assisting other squads with platform changes needed, and cleaning up technical debt. Another responsibility we take on is to ensure that non-critical company requests from outside of engineering also get attention.

These are some of the bigger tasks we completed.

🧹 Code Cleanup

During the initial planning for the Platform Team, we identified code cleanup as the biggest need in the iOS Team. 2018 was a busy year for us. It was also marked by a period of growth hacking where we shipped a lot of experimental features. We also had to sunset our Circles tab which, code-wise, was a major feature.

We started off by removing the Circles feature. The majority of the classes were now unused and simply had to be deleted. The end result was about 10 016 lines of code being removed across 80+ files 🤯.

Next up were the experimental features. We shipped around 17 of these in 2018. They where all part of our internal ABTesting service and most of them were built over several one week sprints. Many of these features were discontinued by turning off the feature flags but the code still existed in our codebase. They all varied from a few lines to several thousand lines of code. We did our best to abstract these as much as possible, but some still found their way into parts of our codebase we weren’t comfortable with them being in.

Our biggest concern with these features is that no single person in the team had the full context over all of them. Things started to go wrong when our users began reporting bugs in production on features that were part of these ABTests, but, should have been, essentially “turned off”. 🙈

The first step was identifying each experimental feature and then followed by documenting the context of its impact on the codebase. Once that was done, we started tackling them one at a time to ensure our Pull Request’s (PR’s) were manageable for the rest of the team to review. The end result was 15 PR’s removing a total of 7,474 lines code.

All in all, we removed around 17,500 lines of unused code 🎉

🚑 Continuous Integration Backup Plan

We use Buddybuild for continuous integration as well as automating our App Store submissions. This ensures everyone in the company has access to the very latest snapshot of our codebase in an Adhoc & TestFlight build.

To simplify our environment, our team members Mac’s aren’t configured to build & sign for the App Store. This becomes a problem when the CI server goes down. While this almost never happens, it has before, and on an important release day 😒.

What we wanted was something that was quick & easy to set up, with clear documentation of how to use it.

We had 2 conditions we wanted to achieve.

1. It must be quick to set up on any existing dev machine. (ie. no 3rd Party software to install)
2. It shouldn’t require any changes to the Xcode project.

What we ended up with was a build script able to build the Production and Beta IPA, our private key stored in an accessible but secure location and a wiki article explaining how to configure and run it. All in all, it probably takes about 10 minutes to set up.

The build script makes use of the exportOptionsPlist option introduced in Xcode 9. This lets us manually configure the build independently from what is configured in Xcode. Since we also have multiple Notification extensions, it also lets us define different Provisioning Profiles per bundle identifier.

🐛 Bug & Crash Reporting

For years the suite of services provided by Fabric have been our crash and bug reporting tools of choice. However, Fabric is being shut down very soon. This created an opportunity for us to evaluate other solutions out there for our bug and crash reporting. Our team isn’t sold on the idea of moving to Firebase. A big reason being that Crashlytics will require an extra 8 Google/Firebase libraries to be installed with it. That’s a lot.

After evaluating a few different options, we have now moved to Instabug. It’s still early days, and while it doesn’t give us as good of an overall state of the app that Crashlytics did, it’s going well. We are now also using Instabug for bug reporting and feedback on our internal builds, replacing Buddybuilds feedback feature.

💾 Removing Realm

A few years ago Realm was implemented in Over with different requirements in mind. Those requirements changed and we where left having to support a 3rd Party framework that we don’t need anymore. 3rd Party Frameworks can become a problem around the time of having to upgrade Xcode and Swift. When Xcode 10.2 and Swift 5 came out, we found ourselves in a bit of a pickle. We couldn't immediately upgrade because of an issue with Realm and we had to wait for them to resolve it. This pushed us to reevaluate the urgency of removing Realm from our codebase.

We had 2 features in Over backed by Realm. Neither of them required any complex querying but did make use of Realms object notifications. Since we don’t deal with 1000’s of database entries, it was clear we could replace Realm with a simple caching mechanism.

This meant updating all our older models to conform to Codable. We effectively replaced thousands of lines of code in Realm with a very simple caching class that is able to insert, update, delete and retrieve the objects. This new class is backed by PINCache. As of today, we no longer have Realm as a dependency. This also had a nice side effect of reducing our build times 😃.

🔨 What Else?

Besides these bigger planned tasks, we did manage to help out other departments with various requests and fix some nasty bugs like our Google users getting logged out on every app launch.

🚀 What’s Next?

There will always be more code-cleanup to do, but as we move forward this will have a lower priority now that the biggest culprits are gone. We are now focusing on improving key areas of our codebase to better help the rest of the iOS team.

We’re also looking into where the Platform team can help in Customer Success to ensure issues reported by our users go through the correct flow of being assigned & actioned by Engineering.

As the responsibilities of the Platform team grows, we may also look at expanding the team later this year.

Thanks for reading and we hope you enjoyed this article 🤓. We would love to hear your thoughts so leave some feedback down below and be sure to stay up to date by following us on Medium and Twitter @EngineeringOver

Also, We’re hiring! 😉 so come work-at-over!

iOS @ Over: 3 months into the iOS Platform Team was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

The challenge that all scaling teams face is in dealing with the growing pile of technical debt and performance problems. With a few people in a team, it is easier to coordinate on refactoring and knowledge sharing. With a larger team, collaboration and communication become increasingly complex. Once those teams start moving faster, it becomes even harder to slow down and deal with that growing pile of tech debt.

In this post, we will explain what happened within our team at Over and what led to the creation of the iOS Platform Team 🚀

🏚 The cracks of a scaling team

As our team scaled, so too did our codebase. We develop primarily in Swift and this has a massive impact on our overall build times (hopefully this improves this WWDC 🤞). It would take around 20 minutes for our build server on Buddybuild to build and run all our tests for a Pull Request. A clean build of Over can take up to 3 whole minutes. These times add up each day and it makes things worse when we started noticing how we would switch to Slack, or surf the web while we waited. All that context switching while waiting for builds really hurt our productivity.

Another pain point we experienced was a growing pile of technical debt. One such example is when we replaced our Circles feature with Teams. This was quite a large refactor that was also implemented as a feature flag. This meant that both Circles and Teams were coexisting. Once we finally shipped, the team continued moving on to support Teams and couldn’t really justify the time it would take to delete the old code. After all, shipping meaningful features to our users is our top priority.

The above issues are just the tip of the proverbial iceberg ❄️. Once we took a step back, it was evident that we have a need for a dedicated team that is free from product work and can focus on helping our iOS team be more successful. We identified an individual on our team, Jamie Raad, that was willing to dive into creating the Platform Team and defining the workload and prioritisation effort. Let’s see how that went down… 😃

✍️ Defining our platform team

The platform team exists to ensure the continued success of user-facing feature teams. As the platform team is not tied to feature work and product deadlines, it is free to focus on broader challenges that inhibit the team from either shipping more smoothly or working productively.

Here was a list of tasks we prioritized as a team that we should start with:

1. Improve build times (locally and on Buddybuild).
2. Investigate new crash reporting tools.
3. Remove Realm from the App.
4. Ad-hoc requests from various departments.

The platform team is also responsible for listening to the technical needs of the team and working on new and improved APIs to assist with software delivery. Having a dedicated team also means we have the resources to investigate things such as obscure bugs that are hard to reproduce, image preloading mechanisms and approaches to reusable generic feeds. These are just a few off-the-cuff examples of areas in which the platform team can improve the way we work.

👀 The first set of results

With the platform team set up and ready to go we embarked on our first challenge: Build times. The goal here was to try and determine which of our frameworks took the longest to build so we can figure out exactly where to optimize. Ultimately, being able to reduce build times by at least 50% would be a huge accomplishment.

To get the results we followed these guidelines of passing different diagnostic options into the compiler. We passed in -driver-time-compilation & -Xfrontend -debug-time-compilation. More about them here.

xcodebuild -destination 'platform=iOS Simulator,name=iPhone 8' \

-sdk iphonesimulator -workspace Over.xcworkspace \

-scheme Over -configuration Debug \

clean build \

OTHER_SWIFT_FLAGS="-driver-time-compilation \

-Xfrontend -debug-time-compilation" | \

tee profile.log

We decided not to use debug-time-function-bodies since we’re just looking for overall build times, not the time per function. We then wrote 2 scripts to process the output of each option and this finally gave us an indication of our build times.

Script Results

Our script read through the build logs and calculated the total time of all Swift files inside each CocoaPod and we ignored files in Over. Below is just a short list of the top offenders

RealmSwift

Execution Time: 13.662400000000002

Wall Time: 38.39059999999999

Unbox

Execution Time: 12.765

Wall Time: 30.538700000000006

Alamofire

Execution Time: 9.7638

Wall Time: 26.3308

FacebookCore

Execution Time: 8.8349

Wall Time: 24.321099999999994

AlamofireImage

Execution Time: 3.0501

Wall Time: 10.8478

FacebookLogin

Execution Time: 1.3653999999999997

Wall Time: 5.8422

KeychainAccess

Execution Time: 1.2674

Wall Time: 1.9256

OverLogging

Execution Time: 0.5084

Wall Time: 1.1639

CocoaLumberjack

Execution Time: 0.3282

Wall Time: 0.4215

Armed with this knowledge we knew what we had to do first. After identifying Realm as the slowest building 3rd Party framework we wanted to test the impact of manually adding and configuring it without using CocoaPods. Thankfully, Realm supplies a pre-built binary that you can manually add to your project. The advantage of a pre-built framework is that Xcode will not compile it when building the project. This speeds up the build process versus CocoaPods, which builds and compiles our frameworks every time we do a build after a clean or delete of derived data. This one change had a massive impact on our build times. We saw a reduction of roughly 100 seconds! 🎉

One disadvantage is that several steps are involved when adding frameworks manually. What helps is that Realm supplies a ready-to-go compiled Framework, not all libraries do that and for those cases, we would have to compile them ourselves.

Since this was a cumbersome process, if this continued, Carthage would be a good alternative dependency manager. The advantage being that it will compile the frameworks for us, as well as provide 1 script to strip the architectures from all frameworks.

Nevertheless, the above exercise demonstrated the importance in inspecting our build times and that there are tangible ways to improve them. Our goal was to reduce build times by 50%, which we did 😬. This is good enough for now and we have some really interesting opportunities for reduction in future.

🚀 What’s Next?

The platform team will continue to experiment with build times among many other things on our backlog. We honestly can’t wait to see what we achieve with this new team. From big things like investigating new architectures to small bugs that we have struggled to fix (looking at you iCloud ⛅️).

One really exciting avenue to explore is the potential of such a team to incubate new talent. Since they are free from feature work it could be a great place for a junior developer to join and focus on exploring the codebase and picking up issues that do not have hard deadlines.

Want to hear more? Be sure to follow us on Medium and Twitter @EngineeringOver

We’re hiring! come work-at-over

iOS @ Over: Starting a Platform Team was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Background timers are a very usefool tool in one’s developer toolbox. Most of the time when we want to schedule a repeating unit of work we consult NSTimer . Problem is, NSTimer requires an active run loop which is not always readily available on background queues. The main thread has an active run loop but this defeats the purpose of having our timer run in the background so its a definite no go. So, to get a dedicated background-queue-friendly timer, we use GCD.

Hello DispatchSourceTimer 👋

Like most APIs we deal with from Apple, there is always a lower level counterpart. When it comes to background timers, we have DispatchSourceTimer . The code to construct one is rather simple:

let t = DispatchSource.makeTimerSource()
t.schedule(deadline: .now(), interval: .seconds(1))
t.setEventHandler(handler: { [weak self] in
    // called every so often by the interval we defined above 
})

This will create a repeating timer that will fire events on a default background queue unless one is specified in makeTimerSource(). All we need to do is create one and we have a repeating background timer 🚀…right?

Not So Fast 🐒

Unfortunately, it’s not that easy. Whilst that code will create a timer, you will experience crashes if you ever try deallocating that timer, or if you are looking to implement pause and resume functionality. Lets fix that 👩‍⚕️

Pausing and Resuming

GCD timers can be somewhat sensitive. If you try and resume/suspend an already resumed/suspended timer, you will get a crash with a reason like:

BUG IN CLIENT OF LIBDISPATCH: Over-resume of an object

This tells us we have tried to resume an already resumed timer 💩. Luckily the fix is simple, we just need to balance the calls to suspend and resume, like the documentation states:

As a result, you must balance each call to dispatch_suspend with a matching call to dispatch_resume before event delivery resumes.

Now is a good time to create a safe wrapper to use these timers

class RepeatingTimer {

    let timeInterval: TimeInterval

    init(timeInterval: TimeInterval) {
        self.timeInterval = timeInterval
    }

    private lazy var timer: DispatchSourceTimer = {
        let t = DispatchSource.makeTimerSource()
        t.schedule(deadline: .now() + self.timeInterval, repeating:                             self.timeInterval)
        t.setEventHandler(handler: { [weak self] in
            self?.eventHandler?()
        })
        return t
    }()

    var eventHandler: (() -> Void)?

    private enum State {
        case suspended
        case resumed
    }

    private var state: State = .suspended

    func resume() {
        if state == .resumed {
            return
        }
        state = .resumed
        timer.resume()
    }
    
    func suspend() {
        if state == .suspended {
            return
        }
        state = .suspended
        timer.suspend()
    }
}

This allows us to have a timer that will ensure not to over resume/suspend itself. We should also ensure its accessed from the same thread/queue otherwise we will need to add an internal serial queue to prevent race conditions.

Preventing Deallocation Crashes 💥

One final hurdle before our timer is complete is to ensure it can be deallocated properly. To deallocate a timer, it needs to be cancelled. If this is not done, GCD will trigger the timer and call the event handler on a deallocated object and 💥. We don’t want this. Furthermore, we also don’t want to cancel a timer that has been suspended(paused) because this will also trigger a crash. Unfortunately, this isn’t really documented but I did find a post in the Developer Forums that outlined how to ensure timers don’t crash when they are cancelled and deinitialised.

Our deinitialisation function can now be written as:

deinit {
    timer.setEventHandler {}
    timer.cancel()
   /*
    If the timer is suspended, calling cancel without resuming
    triggers a crash. This is documented here
    https://forums.developer.apple.com/thread/15902
    */
    resume()
    eventHandler = nil
}

And Finally 🏁

Now that we have our timer up and running, using it is simple:

let t = RepeatingTimer(timeInterval: 3)
t.eventHandler = {
    print("Timer Fired")
}
t.resume()

This timer will automatically fire events on a background queue. The reasoning for this is that DispatchSource.makeTimerSource() creates a timer that uses a default background operation queue as you will see in the documentation:

class func makeTimerSource(flags: DispatchSource.TimerFlags = default, queue: DispatchQueue? = default) -> DispatchSourceTimer

If you want to configure the queue it uses I leave that up to you 👍

I included the full timer code in a gist that can be found below. If you won’t be using your timer on a dedicated background serial queue you should definitely include a serial queue inside it to ensure you don’t have any race conditions.

Happy coding 🤸‍♂️

A Background Repeating Timer in Swift was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

The problem we are trying to solve

Whenever we have to deal withUICollectionView or UITableView, we need to register a cell before we use it, for example.

collectionView.register(Cell.self, forCellWithReuseIdentifier: "Foo")

Later, when you need to use it …

let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "Foo", for: indexPath) as! Cell

This is what Apple gives us out of the box but what could be wrong with that? 🤷‍♂ . Well, for starters, we are dealing with a stringly-typed API. We have to register and dequeue our cells based on a string identifier. These two can easily get out of sync and can become a pain to maintain. Not to mention, we will only know its broken in production!

When we dequeue our cells, we also have to make a force cast to the cell type. It would be nice if we could avoid this entirely. Lastly, we have to remember to register the cell. What if we could have it all and automatically register the cell the moment we tried to dequeue them 🤔

The proposed solution

Our aim is to simplify cell reuse and registration to just one line of code

let cell: Cell = collectionView.dequeueReusableCell(for: indexPath)

This seems like a job for generics 😄

The solution here is inspired by this gist.

While generics are a fairly advanced topic, they are one of the most powerful features of Swift. For those new to Generics.

Generic code enables you to write flexible, reusable functions and types that can work with any type, subject to requirements that you define. You can write code that avoids duplication and expresses its intent in a clear, abstracted manner.

A large portion of the Swift standard library is built with generic code.

Since we would like to work with both UICollectionView and UITableView, we can start off by defining a protocol for a reusable view that has a defaultReuseIdentifier.

public protocol ReusableView: class {
    static var defaultReuseIdentifier: String { get }
}

Since cell registration always needs a reuse identifier, let’s use the class name as the default identifier. We seldom need different reuse identifiers for the same view. Let’s use String(describing: self).

extension ReusableView where Self: UIView {
    public static var defaultReuseIdentifier: String {
        return String(describing: self)
    }
}

One thing to be mindful of, is that we can also load our cells from a nib file. So lets go ahead and add a protocol for a nib loadable view.

public protocol NibLoadableView: class {
    static var nibName: String { get }
}

extension NibLoadableView where Self: UIView {
    static var nibName: String {
        return String(describing: self)
    }
}

Now that we have our ReusableView & NibLoadableView protocol, we can implement a generic way to register and dequeue a cell:

extension UITableView {
    func register<T: UITableViewCell>(_: T.Type) where T: ReusableView {
        register(T.self, forCellReuseIdentifier: T.defaultReuseIdentifier)
    }

    func register<T: UITableViewCell>(_: T.Type) where T: ReusableView, T: NibLoadableView {
        let bundle = Bundle(for: T.self)
        let nib = UINib(nibName: T.nibName, bundle: bundle)
        register(nib, forCellReuseIdentifier: T.defaultReuseIdentifier)
    }

This can be used on any UITableView instance. Notice how we have two register functions. One is for ReusableViewand the other one is for NibloadableView . When you register it will always look like table.register(CustomCell.self) and depending on the protocol that the cell conforms to, it will use the relevant one. Try it out 😄

Now that we have a way to register, lets put together a nice way to dequeue our cells

func dequeueReusableCell<T: UITableViewCell>() -> T where T: ReusableView, T: NibLoadableView {
    register(T.self)
    return dequeueReusableCell(withIdentifier: T.defaultReuseIdentifier) as! T
    }

func dequeueReusableCell<T: UITableViewCell>() -> T where T: ReusableView {
    register(T.self)
    return dequeueReusableCell(withIdentifier: T.defaultReuseIdentifier) as! T
    }

Every time we dequeue we call register first and in our experience the performance cost is negligible.

Mission accomplished 🚀

Now that we have everything put together, this is how you would use it

//make your subclass conform to the protocol
extension CustomCell: ReusableView {}

//then you would  dequeue your cell 
func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
   let cell: CustomCell = collectionView.dequeueReusableCell(for: indexPath)
   return cell
}

That’s it! We have removed so much boilerplate code now and our lives are so much easier. We no longer have to worry about registering cells and supplementary views. We never have to think about what string identifiers are in use and best of all, its all typesafe. With just two lines of code you are good to go 🎉

Download the full code here

I hope this helped you out and you learned something new today. If you enjoyed reading this article be sure to throw your hands together for a couple of claps 👏👏 . Feel free to stalk me on Instagram and Twitter 😄.

An easier way to dequeue cells in iOS was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Believe it or not, our iOS team also had this problem. It was not uncommon for us to have 22 pull requests open for review. Let’s put this in context: 22 pull requests in a team of 8 developers. If all of those pull requests are counted as work in progress all of us were doing 2.75 things at once.

It seemed like a few people did most of the reviewing, which also meant getting anyone to review your pull request was a tough process.

I hope this is not the first time you hear this, but multitasking is a myth. Us humans are terrifically bad at it. What we actually do when we “multitask” is task switch (or context switch).

Having recently joined and seeing this as a pain point in the team, I felt we could run two fun exercises to highlight this problem. Enter our first game…

The first game: Proving you don’t multitask well

1. Draw four columns on a page
2. Write one of these at the top of each column: “1–12”, “3–36”, “Jan-Dec” and “7–84”

The objective: Fill out all the columns so that the first column would have 1, 2, 3 etc., the second column will have 3, 6, 9, 12 etc., the third column Jan, Feb, Mar, etc. and the fourth column 7, 14, 21 etc.

Time yourself doing this.

Round 1:

Complete all columns by writing one value in each column, i.e. jumping between columns as you go. (You’ll write 1, 3, Jan, 7 in the first row and then move on to the next row, writing 2, 6, Feb, 14 etc.)

This should take you about 90–120 seconds. Go ahead. I’ll wait.

Round 2:

Complete all columns by finishing one column completely before moving on to the next column. (You’ll write 1, 2, 3 … 12 in the first column, then move on to the next column 3, 6, 9 … 36 until all columns are finished.)

This should take you about 60 seconds.

The result

Sorry to break it to you, but like billions of people, you are also bad at multitasking/task switching/context switching. In round 1 you paid the price of task switching by taking nearly twice as long to complete all the columns. Writing numbers and names of months on a piece of paper is not a difficult task. Imagine if you were task switching between refactoring, debugging a production issue and checking Slack 🤯

The second game: Proving you don’t have to write code to contribute

Now we’re going to fold some origami. Yes, the art of paper folding. This is going to require your whole team. So gather everyone ‘round. You’ll probably need between 4 and 8 people.

Here are the instructions:

We are going to make a production line to create paper boats. Divide the steps from the instructions among the team members like this (assuming 5 people):

The first person does step 1, the next person does steps 2 and 3, the next person does step 4, the next person does steps 5 and 6 and the last person does steps 7, 8 and 9. You can change these up if you have more people, but purposefully create a bottleneck by at least giving the last person more steps to do.

The objective: Make as many paper boats as possible in 3 minutes.

Round 1:

As soon as the clock starts, the first person does their fold, passes it on to the next person and immediately grabs a new piece of paper to fold. Everyone in the queue then continues to make their folds and pass it on to the next person as fast as they can.

After three minutes sound the airhorn and stop. Now, count three metrics: the number of finished paper boats, the number of unfinished boats (any paper with a fold still in the queue) and the number of finished boats of which the team is happy with the quality.

Round 2:

This time each person is only allowed to pass work to the next person if the next person is not currently busy with a fold. To signal this properly have each person sit with their hands in the air if they are not busy with a fold. When someone has their hands in the air, the person before them in the queue will know they can pass work on to them. Some people will find themselves with their hands in the air more often than not.

After three minutes sound the airhorn again and stop. Count the three metrics again: the number of finished boats, the number of unfinished boats and the number of quality boats.

The result

You should have the same number of finished boats after both rounds (or more after round 2).

However, round 2 will have significantly less unfinished boats and also the same number of quality boats (or more).

This is a nice practical example of the difference between a push system and a pull system. In round 1 everyone was pushing work into the system, regardless of capacity. In round 2 work is pulled through the system when there is capacity to perform the task.

Combining the results of the two games

Armed with the results of these two games we realised a couple of things:

1. Writing more code and creating more pull requests were actually harming the amount of work we are capable of delivering.
2. Having more open pull requests than we have people on the team slows down our “production system”.
3. If we are constantly switching between writing code and reviewing code (without finishing either) we are increasing the time it takes to finish both tasks.

How we improved

We have limited the number of “allowed” open pull requests to six. If we submit a pull request and hit the limit (or pass it) we don’t continue to our next coding task without reviewing some pull requests first.

We have seen a dramatic reduction in review time for our pull requests which results in a very welcome increase in output and all of us have more time to review other pull requests.

Not only do we have more time to review, but reviewing has become a shared responsibility within our team.

As a bonus we feel more like a team than we ever did, since everyone is involved, collaborating and sharing knowledge on our open pull requests.

Try it out

These games and insights have made a significant difference to our development process. I hope it can inspire and help you to build great products. Like Over 😃

Further reading

Lean Software Development: An Agile Toolkit

A must-read about the concept of waste in a software production system — how to recognise it and how to avoid it.

Queueing Theory

The mathematical analysis of how work flows through a system with queues. The article explains how to avoid queues and how to implement work in progress limits (like we did) to avoid the pitfalls of unavoidable queues.

Limit your pull requests and see what happens. This is our story. was originally published in Over Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.