When I was reading some of these stories, I was amazed by how different and extraordinary these women are. They live in different countries, have different education, work in various cybersec areas like building security software, breaking systems, reverse engineering, training developers, managing risks, etc. — they are unique, yet connected by cybersecurity field.

Here is my story:

Epic Women in Cyber — Anastasiia Voitova

Background

Usually, I use Apple Notes app between all my devices. I write ideas for blog posts and tweets there, draft my conference talks and keep track of things I do during a week (for internal reports at work). I often “lock” sensitive notes.

Time to time I try other applications for note-taking and drafting. I’m very concerned about my privacy (not to mention corporate policy, which adds extra layers of tinfoil), and despite the fact that I don’t store passwords and credit cards in note-taking apps anyway, I don’t want publishers to have access to my notes.

This is a short summary of data security flaws I noticed among 6–7 popular note-taking apps. In most cases, these are not “bugs”, but rather “design caveats”. It’s nothing too complicated — for an experienced mobile developer with a security background, it’s just running the typical checks — which security controls are in place and are implemented well.

I’m not going to publish app names or specific bugs — those apps are great, developers are doing a great job, and security is complicated. I’ve suggested help to the developers, so there’s a small chance that some apps will become more secure soon 🤞

1. No protection at all

If notes are stored as plaintext and are synced to other devices as plaintext — that’s not my kind of application ¯\_(ツ)_/¯

2. Password protection only

Apps that allow users to “lock”/”unlock” note using their custom password, Touch ID or Face ID. Until a password is not entered, users can’t read note content.

This is rather cool because apps can use native Apple LocalAuthentication framework and rely on the authentication mechanism of iOS itself.

How to improve?

— Touch ID / Face ID is the second factor, and if a device doesn’t have these capabilities, or user can’t use them now, users should fallback to enter a password. The application should show password screen, and save the password for later usage (but not infinitely, and store securely), until users attempt to unlock next note. It might be a good idea not to store password exactly, but to use key derivation functions (like PBKDF2, bcrypt/scrypt) to derive hash from a password. On unlock — calculate hash again and compare hashes. The app should store the hash in a Keychain.

— Cache password. If a user has several notes “locked”, once a password is entered (or Touch ID or Face ID), the app can cache it for X minutes in a way, that user won’t need to re-enter password again. However, after X minutes, the app should flush the cache and ask password again (pattern is called “repeated authentication”).

— Wrong password attempts. If a user can’t input correct password for 3–5 times, increase the time before attempts (like iOS itself does). This pattern is called “incremental authentication delay”.

— Reset / change password. There should be a way to change the password (set up a new one after a correct input of an old password or Touch ID / Face ID). Reset should work in a similar way.

— Password syncing. Imagine that a user has two devices with the same application, and decides to update their password on one of them. Of course, the password should be updated on both devices / apps. iCloud Keychain might help in this case (be careful, it can be hacked too), or you can build secure cloud storage where the app can store password hashes.

Why it’s not enough?

Password protection is not an encryption.

The content of the note is still plaintext, even if the access is protected. If the application stores the notes in local file storage, they might appear in backups, can be accessible if device access is compromised, and can be leaked from cloud system that syncs/stores them.

3. Bad network security (rely on TLS only)

Of course, all apps now use TLS to send network requests to the backend server. However, TLS is not enough if someone wants to read your notes. In my talks, I describe in more details why sometimes and in some countries, we can’t rely on TLS itself.

During my testing, I could easily intercept and change network requests — which allows me to not only read notes content, investigate API, send not-allowed network requests, but also to unlock some app features available after subscription only. (I was able to access Pro mode features just by changing the value of “is_subscription_active” to true in the response of account_status request ¯\_(ツ)_/¯).

How to improve?

— Use CloudKit and its API instead of your network layer. A simple and easy way to transfer a problem to Apple’s shoulders.

— Use strong TLS certificate settings and rotate certificates often (90 days, as Let’s Encrypt suggests).

— TLS/SSL pinning: hardcode server’s public certificate inside the app and compare it with certificate received from server connection. So if someone nasty (like me 😅) wants to intercept requests, their certificate differs from hardcoded one, so the app breaks the connection. Keep an eye on updating hardcoded TLS certificates in the right time: hardcode several certificates in the app (current and future ones), and rotate the certificate list on new releases.

— Extra layer of encryption. Unfortunately, certificate pinning can be broken even on non-jailbroken devices. For transferring really sensitive data you might want to add an extra layer of encryption like Themis Secure Session, libsodium, noise protocol, but then you need to manage the keys to additional transport layer really well.

4. Bad storage encryption

As we learnt above, password protection is not encryption. From the user perspective, the application works in a similar way — user inputs password / Touch ID / Face ID to “lock”/”unlock” a note, but this time note’s content is encrypted.

To build proper encryption and key management schemes, developers should answer a question — “what is the worst: data leakage or data loss”?

First means that app should not have a way to decrypt data if the user forgot their password (and there are no other ways to prove their identity), second means that app should have a ‘backup’ way to decrypt the data.

How to improve?

— Define exactly what data to encrypt. One application, that I tried, encrypted my notes, but at the same time it generated preview image with note content, that was stored as file next to the encrypted note. Totally visible, a picture, in plaintext. 🙄

— Do not hardcode encryption key inside your app. Never ever. Don’t store it in user defaults or CoreData/SQLite/Realm. Use Keychain (but Keychain can be broken, so protect encryption keys with user-derived keys).

— Use KDF. As described above, users come up with bad and short passwords. So, use KDF to make passwords stronger (longer, more random), or use libraries for storage encryption that use KDF under the hood.

— Access password != encryption key. Imagine that user decided to update their password: if the app uses exactly this password to encrypt the notes, it should decrypt notes with old password and re-encrypt them with a new one (what will happen if the user forgot password? 😉). It’s better to separate user password from encryption key: app should generate a long random encryption key, and store it in the Keychain (or iCloud Keychain). Before encryption/decryption app asks user password / Touch ID / Face ID to make sure that user is really a note-owner, unlocks Keychain, reads encryption key and decrypts the note.

— Change / reset password — depending on your answer to the “what is the worst” question, techniques differ.

— Syncing access passwords and encryption keys. When the application runs on several devices under the same user profile, it needs to sync keys. Sending encryption key in a plaintext (even above TLS) is a bad practice. Use iCloud Keychain (but it can be broken too), encrypt encryption key with a temporary password before syncing (hehe, this is the point where cryptographic rabbit hole opens 🌀).

— Use strong cipher. AES GCM 256. Forget about CBC mode, it’s hard to get CBC right (many attacks exist). GCM mode has built-in authentication, so if the message was changed, it can’t be decrypted.

— Do not manipulate paddings, IV, and salt during encryption, if you don’t know why they are used. Use “hard-to-misuse” cryptographic libraries that are built by cryptographers for developers. As maintainer of Themis, I can’t recommend it enough.

— Check compatibility. If your app should work on multiple platforms, make sure that encryption cipher and keys format are the same. Ensure that notes encrypted on iOS app can be decrypted on Android app (and vice versa, for each platform. Better write tests for that). Use libraries that are built to support multiple platforms ‘out of the box’.

— Versioning. You might want to change the encryption algorithm later, so it’s better to add some information about the encryption version. Add it as a prefix to the encrypted note content, or save as a separate field in note object. Trust me, you’ll need this later.

5. Lack of end-to-end encryption

The best of the best way to encrypt all my notes and to hide them from prying eyes is to use E2EE. In this case, notes are stored and transferred encrypted, in a way that only the owner (me) can access them.

I’d rather pay a subscription fee for the app that uses E2EE, than use a free app that scans my notes and sells data to ads companies (face-khe-khe-book).

E2EE might be really tricky if users have many devices, or if they want to share their (encrypted) notes with other users. Fortunately, I have some workshop-ready applications for mobile platforms, that use Themis as encryption engine and Firebase as a server. For a large distributed applications with E2EE data collaboration — there’s another open source cryptographic engine Hermes (free for non-commercial use) that was built specifically to handle complex cases.

That’s it for today!

If you want make your apps more secure — you are on the right track, check links from this post, follow my talks and tweets (my DMs are open). If you would like to get more personalized help — ping me. Me and my team not only build software for data security, we run training for developers and even work as external security engineering team.

Found interesting ideas and useful links? 👏 me!

Below you’ll find my typical reply, that I’ll keep updating when changing my mind.

📕 I can really recommend a Serious Cryptography book. It’s awesome, easy-to-understand and covers many questions starting from discussing Randomness to Quantum Computing.

📒 Of course, the Practical Cryptography book by Schneier is a must have as well: deep, serious, practical.

📺 Also, on Coursera there’s a Cryptography 1 and 2 courses. They are quite deep (I mean quite complicated).

🖥 Aaaand I like Applied Cryptography lessons on Udacity (they are easy and interactive!).

🕹 Aaaaand crypto101 is a pdf book, explaining both basic crypto-algorithms and more complex crypto-systems.

📝 pFarb/awesome-crypto-papers — many links on different concepts, articles, books. Great starting point when you’ve learnt simple things and want to continue with more complicated ones.

💻 sobolevn/awesome-cryptography — more practical links here: libraries, tools, easy-to-integrate encryption bits into your infrastructure. Tip: select ‘boring crypto’ libs.

🦊 vixentael/my-talks — my own talks about applied cryptography/security for developers. I touch many different aspects, from secure app architecture to tips-and-tricks every dev can do. Each presentation has ‘home reading list’ on the last slide 😊

Sometimes you can find 💎 like cat explaining HTTPS, or explain-me-like-I’m-five series about Zero Knowledge Proof, E2EE, or great video about public-key-cryptography based on mixing colors.

Hope it helps 🤟

This is a small handout doc following my talk “Zero Knowledge Architecture for mobile applications”, which I gave at several conferences during the autumn of 2017.

Sensitive data narrative

If you want to hear the story about sensitive data we have and how we use it, please read through the slides or watch the video.

We have sensitive data and we can’t avoid sharing it.

We cannot trust data providers by default and we don’t want to think about data security all the time. So we choose to retain control over sensitive data while we store and share it. How? Read below :)

Zero Knowledge Architectures

ZKA is a design principle.

In simple words, everything you do on a Zero Knowledge system is encrypted before it is sent to the server and the key to the encryption is also never revealed to the vendor.

• Anatomy of a zero-knowledge web application
• Why We Will No Longer Use the Phrase Zero Knowledge to Describe Our Software | SpiderOak

The first important principle of ZKA is end-to-end encrypted clients that perform crypto computations. The server knows nothing about the nature of the data. By the way, sometimes ZKA is referred to as No-Knowledge architecture.

Second, all operations are on encrypted data. It means that if you’re going to add a new record to a database, you should add it in encrypted form. If you want to share a piece of data, you should share it in encrypted form. You even perform a search inside the encrypted data.

These principles don’t add additional security in a common sense of this term, but rather guarantee that client-side encryption is used properly. Zero knowledge algorithms and protocols ensure that no keys, passwords, files, or any other sensitive material ever gets transferred in an unencrypted or reversible form. There is no point in time when encryption keys or unencrypted files are visible to the servers or service administrators.

Why mobile?

ZKA relies on cryptography and requires trust to the device that runs crypto code. Mobile has a quite trustworthy runtime environment, compared to the browser or most desktops. Still it is not entirely without risks.

Where is ZKA used?

In messaging, we can use end-to-end encryption. Apart from clients, nobody can read your secret communication. Clients have means to verify trust to each other and to the server to ensure that cryptographic protection is working properly right now. End-to-end trust without leaking anything to storage or transmission layer is the basis of ZKA. You already know many examples of E2EE chats.

In authentication, we can use interactive crypto protocols known as Zero Knowledge Proof protocols. ZKP enables two parties to compare a secret without exposing it, efficiently avoiding leakage of secrets during transmission.

What about sharing data?

We know how to collaborate securely when a document is one blob of data, but modern document is actually a large tree-like structure, so will everyone be able to see everything?

Let’s say, you want to share sensitive data with some users. The naive approach is to encrypt the data several times — once for each user, using their keys.

So if you are sharing data with five users, you need to encrypt the data five times with different keys and transfer these keys to each user. This approach leads to data duplication: you now need to store five times more and if you need to update the data or to change the access policy, you need to decrypt and encrypt some or all the records again.

Does this sound like ZKA? Well, yes. Does it sound easy? Not really.

But there are better approaches to data sharing: you can provide access to specific blocks of encrypted data for particular users. Accessing or encrypting or re-encrypting only what you need; and when you need to change access policy (who has access to what), you have to re-encrypt just one small access block.

As an implementation of such approach, we have an open source library called Hermes. It has C-core, and it is available for many platforms.

Credit history example

Credit history is a beautiful example of sensitive data that is shared among multiple entities. Credit Bureaus store your credit history and send it to your banks by their request. A credit history has lots of sensitive details about your life and your interest, so it’s preferable to minimize its sharing.

Usually, credit history is stored in encrypted form, as a single blob of data, so when it is leaked, you lose everything. Splitting the file into separate encrypted blocks and allowing banks to manipulate only specific blocks minimizes the risks.

Please, see the slides or the video to grasp the whole example with credit history and a cat.

How to implement a ZKA kind of collaboration on shared data

Let’s describe the implementation of secure data collaboration.

Key wrapping

Split a single blob of data into smaller encrypted blocks. Decouple storage keys from the user keys, encrypt data with storage keys — they will protect the data.

Manage privileges

Store storage keys in containers; cryptographically control that users can change stored keys by comparing actual ownership tags (basically, authenticated MAC) to prove that they can perform an operation.

Control requests

Since the data is available only through the crypto layer, you should make sure that your architecture translates every operation over your dataset into crypto API commands; that no one puts data in a plaintext bypassing your crypto core.

Mitigate remaining attacks

Crypto is not a magic wand; it just narrows the attack surface. You still need backups; you need to use ZKP to protect against replay attacks, you need to ensure code trust, secure keys lifecycle, and use traditional things like intrusion detection, monitoring, and traffic inspection.

Cossack Labs / Zero Knowledge Protocols without magic

Other use cases for ZKA

You can use ZKA approach everywhere you want to provide access securely to small blocks of data shared among different clients.

Keybase launches encrypted git

• complex documents with comments or detailed spreadsheets (i.e. Google Docs, Dropbox Paper, etc.). In many cases, users shouldn’t have access to the whole document.

• file systems are a perfect example of small blobs of data, structured and shared with different access control rights.
• document store protection. If every blob in a database is a protected one, and access rights are protected, then you get an end-to-end document store, where every document or field’s rights can be granted to everybody. Imagine MongoDB with custom queries on secure data for the untrusted web apps and trusted queries from your mobile apps.

How difficult is it to implement ZKA in your products?

Well, it depends. Of course, custom implementation will take a lot of time, but there are plenty of existing solutions:

• Axolotl for messaging,
• Themis’ Secure Cell and Secure Session,
• Hermes.

Are there other implementations? Yes!

Check out LAFS. It is a secure storage system that stores your files encrypted separated by chunks on different servers.

Or check out ZeroKit that has integration with CareKit.

Recap

Zero Knowledge Architecture is a design approach that solves problems of trusting the server and the transmission environment. It’s rather easy to solve when interactions are simple and gets rather complicated when we’re talking about databases or collaboration. But it’s not impossible now: I’ve shown you some examples of solving everyday problems using ZKA. I believe that we will collaborate on data more and more with each year, and it’s wise to prepare our products and to protect our users.

More links to follow

• Explain Like I’m 5: End-to-end Encryption
• Zero Knowledge Proof: Explain it Like I'm 5 (Halloween Edition) | HackerNoon
• Zero Knowledge Proofs: An illustrated primer

Looking for something else?

Handy list of my other talks

vixentael/my-talks

Today we going to talk about key management. We start our journey step by step: discovering infrastructure layout, digging into ideas of threats, trust, and keys. Discussing key management system: what is a key generation, how to access and revoke keys, and many other exciting theoretical things! But of course, we will apply this knowledge to the mobile apps.

Watch Video

Scroll slides

Establishing trust

How do you know that you should trust someone in a real world? How do you know, that your friend over the phone is the one you should trust? You recognize them by their voice or typical words. How about a stranger? Probably you would use a secret keyword.

In shiny worlds of computers, there’re many ways to build trust to parties you don’t know and can’t easily recognize: sharing passwords, asymmetric keys, zero knowledge proof.

Usually, we establish trust on servers, mobile, data in transit via public channels, and so on. I mean, real things. Like cables and stuff. Our infrastructure is full of keys and data, all that in cables, yeah. Those cables are important — they’re out of your control, and whatever you reach outside your office — you rely on these creatures.

Why do we need trust?

– To protect the data in those cables, where evil CIA and badass crackers are looking for your secrets. Using special super mathematical techniques, we provide guarantees of confidentiality, authenticity, and integrity towards protected data, bound to trusted keys or secrets. It’s provable and unhackable if you manage the keys correctly.

Since “keys are what we trust” and they control trust in the system, managing them is managing trust. So, let’s talk about key management.

Keys

– What is a key? – It’s an array of bytes.

Based on what kind of process is going on, we need different arrays of bytes with different properties:

1. We use secret keys for symmetric ciphers.

2. Public/private keypair for asymmetric ciphers

3. Passwords are strings that you can remember. It’s a poor key, by the way. We should use KDF instead of user password itself.

4. One-time PINs are small and easy to transfer key with risky properties. It’s an additional trust token, like that thing you get via SMS from your bank when logging into their system.

– What kinds of keys we know in our shiny iOS world?

App tokens, server tokens, user passwords, certificates, well, familiar things, aren’t they? They all are keys.

– What do we need them for?

Keys protect the data!

They let you access the data, verify data authenticity and integrity. Of course, mostly we deal with user-generated data that users are putting into our apps. However, we also care about access to external resources (like 3rd party services or your web servers), and identifiable data of other users in our app (for example, user info from those who like your tweets, if any).

– What do we protect data from?

Oh well, you know, all those threats we can and can’t handle. Like data tampering and leakage, a man in the middle, active and passive, and of course really common problems like rubber-hose cryptanalysis.

Keys are arrays of bytes that are stored somewhere, and they unlock protected data and facilitate trust. Of course, the attacker wants them. Basically, keys are small chunks of data, and they’re subject to threats too.

Keys are small chunks of data and should be protected too!

What can happen to the keys?

1. Attackers can steal the keys, and that’s bad (reasons are obvious, no?).

2. What’s worse, stolen keys can be replayed — they can be used to access something protected.

3. And, if the attackers are lucky enough, and you’re not, keys can even be replaced: attackers can throw in their own evil keys to access resources.

So we need to build a system which protects and manages keys.

Key management system: goals and actions

In key management, there is one practical goal: trust and security are preserved yet the system is usable. If you don’t do that, you’ll end up with another ultra-secure super-paranoid end-to-end system nobody’s willing to use :)

Any key management system consists of several processes, which are linked sequentially: from generation to exchange, from exchange to storage and access, with revocation to control compromised and outdated keys, and rotation to ensure key lifetime. Sometimes, this system should contain service processes too: like backups and admin access to encrypted data.

Generation and exchange

The goal of key generation is to create mathematically strong trust tokens: ones that stand against brute force and smart enumeration. We need to use good random number generator and an algorithm, combined with a secret, to produce a key. To ensure it does not leak, we need to produce it where user inputs secret, or where it is safe to store the key.

Basically, keep it as close to usage and storage location as possible.

How about a password? A password is not a very good key: it’s easy to brute force, and usually, users come up with poor passwords. Key derivation function (KDF) helps to convert the password to the cryptographically strong key so that attackers will be sad.

After we generate the keys, next step is exchanging the keys between trusted parties. Key exchange is an essential process in establishing trust: we need to exchange or distribute keys securely.

Storage and access

Key storing is a big deal: we need to store them in a way which minimizes the risk. One big point: never store the keys with the data encrypted with these keys.

Once the keys are stored, we want to access them to actually use them. Achieving the balance is tricky: keys should be protected, but still easy to access legitimately.

Rotation and revocation

Now that we know that keys are generated and stored… do they live forever? Of course no!

The key lifecycle is a biography of a single mr. Key — from birth to oblivion.

The lifecycle will specify when a key should no longer be used for encryption, when a key should no longer be used for decryption, when a key is a key no more.

And of course, you don’t want everything to be encrypted with one key, right? If you leak it, you’re done. You want to have several keys — one per user, or maybe one per group, or at least one per certain number of records. It’s all about controlling the risk.

Now, what if key should be changed?

You need your system to support re-encryption with new keys. What if somebody finds an attack for the algorithm you’re using? You need to change it. To do that, you need to decrypt the data, generate new kinds of keys, encrypt and store the data, sometimes eliminate unused records, sometimes ensure correct purging of non-encrypted data from memory. Wheeeww. A lot of things to consider for key rotation.

Now that we’re mostly done with theoretical ideas, there’s a practical consideration.

Apart from user features, the system has certain maintenance procedures, which require special access to data, like backups, admin access, password resets. These things are better implemented through additional keys and separate protection layer for them.

Key management in practice

There are three main ways to establish trust on mobile.

1. On channel exchange. You connect first to a server, record it’s certificate, and when you connect again, you can compare the certificates. This away, apart from trusting the certificate itself, you trust the server too.

2. Mediated exchange. You know remote party’s identifier and can ask someone you trust for their public key. For example, you may use keybase.io to exchange public keys in a trusted fashion.

3. Trusted channel exchange. Two methods above rely on many assumptions: you have to trust somebody. The most simple way is always just handing the key or secret to the party you’re establishing trust with. For example, to build trust with Facebook in your app, you need to authenticate via app token. To do that, you register the app in Facebook dev portal and copy app token into your code. That is exactly trusted channel key exchange — via you!

Storing or not storing?

So people, where do you store the keys?

~Code?~ Keychain!

Keychain is great, but sometimes you can’t use it, or you don’t want to use it (by the way, Keychain will break if the device is jailbroken).

The easiest way to avoid leaking keys — don’t store them at all! :)

But if I need to store..?

Mostly, we deal with two kinds of keys: user-defined and app-defined.

App-defined keys are built in. You already have them in the app, but they should be stored and accessed correctly.

User-defined keys are something that user inputs or based on user input. Those keys should be put in right place on right time.

Obfuscation is fun, but don’t rely on it

There are different ways to obfuscate the keys: store them as hex chars, or replace characters inside key strings.

Obfuscation depends on social engineering, and the goal is to confuse the attacker: save certificate, but rename it to mp3 file, for example. Or split key to different pieces and put them in different places. Experiment.

However, a well-trained attacker will break this defense in minutes. Better spend your time on serious defenses.

Store encrypted

Easy:

1. Encrypt keys during development

2. Store encrypted keys only

3. Decrypt before using

This technique works well with app-defined keys. You may ask: how to store keys that were used to encrypt keys? :)

The answer is: not every key should be stored. You can generate or calculate encryption key based on something you already know, something like bundle id or next 4-digits of the result of division 27 by 13.

Poison keys and Honeypot

Another prominent technique in the school of deception known as poison keys.

Make attackers think they have the right keys, make them use those keys. And be alarmed: attackers are poking in your app, naive enough to try fake keys on you.

This technique works well with honeypot: store these poison keys in obvious places and just watch the logs.

For example, store a fake key in the plist file, or put “server.cert” in the app bundle. Sometimes, the best you can do to prevent attacks is actually detect them timely. This is such case. By the way, you still have to protect the real keys really well.

Key-points

For every fragment of your system, trust is applied differently. There are no religious choices. Best solutions come from assessing your real architecture and fitting the tools to it.

Read more

• Cryptographic Storage Cheat Sheet [OWASP]
• Key Management Cheat Sheet [OWASP]
• Managing Keys, Certificates, and Passwords [Apple]

Wanna see more of my security slides?

We’ll talk a bit about the most undervalued part of mobile security: ideas and concepts. Another name for this talk could be “Everything Will Be Broken,” but what should we do?

Intro: this is the picture

Let’s take a look at the problem domain. What’s on the landscape? The picture shows our typical infrastructure — an iOS app talking over some network connection to a server where we have some custom logic serving our tasks.

So, what do we care about while we’re making apps? User experience, fast & continuous delivery, and getting things done. And Swift, of course. Swift is very exciting!

What don’t we care about? Server crap. Everything not iOS is magical and unknown :)

Imagine you put on the Security Wizard Hat. What will you see?

This is what should be important for mobile developers if they care about security:

– iOS runtime and user-held secrets are sources of trust. Apple is good, mostly. Trust Apple (…mostly).

– Beware on the Net, the dragons are near. Limit trust to the server: there’s plenty of unknown dangers.

– Be attentive to server logic: it’s easy to fuck up, and there is no sandbox around.

Why should you care about that?

Because you’re not alone.

In the app, the FBI is knocking at your door. On the net, the NSA and Russian Hackers are here to get your data. On the server… oh well, everyone will try to attack your home base.

The question is, what can we do about it?

What is in our control? The brains.

In other words, the amount of mental effort we put into our app security and the amount of knowledge about protecting things we have at our disposal.

Why does the problem exist?

What exactly is wrong with the way we see app security now? Well, a lot. Security is hard; it is an engineering discipline, unlike familiar subjects we know.

“It is secure” is not a valid statement; instead, we can just say, “It has not been broken yet.”

Not very reassuring, right? There are three main reasons why the problem exists: speed, openness, and ignorance.

We need to get things done. Fast. We live in the design-driven world, where short iterations are bound to the functions visible to the user. In the world of constant MVP, security is outside that bare minimum we do to meet deadlines. It accumulates technical debt, which is a synonym to bugs and failures.

Using the OpenSource libraries and third-party APIs are easy and convenient, but you can’t be sure that they’re secure enough.

Usually mobile developers know little about security. And — unfortunately — we don’t think about these “security things” a lot because it requires another type of thinking we rarely encounter.

Everyone makes security mistakes

You may not notice, but there’re lots of security vulnerabilities found every month. The more applications we create, the more applications we use every day, the higher the chances that we’ll use vulnerable software.

Apps for car remote controls are amazingly popular now. Unfortunately, lots of them skip the authentication stage, so the attacker can access private data and even unlock the car. You can find some examples & links in my slides.

Everything will be broken. Hacked cars are just more scary than hacked photo services, right?

But even Apple makes security mistakes. In the last few months, twice in the iMessage — the core infrastructural app — huge vulnerabilities were found.

The first one allows the attacker to recover messages in plaintext via redirecting application flow using javascript.

The second one concerns the attachment transfer protocol, allowing the attacker to grab your photos by enumerating cryptographic keys via the silent flood of malformed messages, precisely a quarter million messages. That’s a really long and tedious hack, but hacking via key enumerating in 2016 looks rather striking.

(Now the desktop version of iMessage has screen sharing with remote control. Wanna guess what problems come next?)

Taking a look at the bigger picture

There’s no official statistics for iOS applications, but there’s some for iOS itself. According to the National Vulnerability Database, a record-breaking number of vulnerabilities was registered in 2015.

Unfortunately, mobile developers make things even worse!

Last year’s notorious AFNetworking bugs turned hundred of apps into an insecure state immediately. But what’s even more exciting: everybody knows that without SSL pinning your apps might not tell friend from foe; only one tenth of popular apps use SSL pinning.

Why does this even happen?

Is this miserable state of things our fault? Not really; it’s just the way things are. But there are some problems we can solve if we understand what makes our mobile platforms unique.

We think mobile and backend are in a classical client-server relationship. But they’re not. Mobile apps don’t have many features we are used to having in classical apps. And mobile apps require very specific server behaviors, not all of which are good security-wise.

In the end, the problem is that if it works — it does not mean it’s secure.

There are many things we don’t notice: Apple or ecosystem took care of them already. But looking closer at your app and device, we see the disparity between client and server in many things. The mobile app is not a ‘classical’ client, but a thin client, talking over several layers of abstractions. This, together with the aforementioned human problems, gets us into a sad state of things…

Mobile security is hard and still undeveloped. Security relies on the shared wisdom of previous generations. In our world, there is no yet. And we find it hard to import common knowledge for some reasons. One of them is that our ecosystem is really very different, another is that risk models are blurred even for us.

What exactly are we risking?

What bad could happen to our mobile apps? Potential attackers are looking for three things: data, identity, and control.

Security people will notice I’ve skipped something in the next slides: local access. iPhone’s architecture makes JailBreak hard enough for a remote attacker, and it’s no better than smashing your fingers for a local one.

Data is anything sensitive for the user. Either data that is useful (and the user wants to keep it safe) or data that is private (and the user wants to hide it from other people’s eyes).

Why do we keep it on the phone in the first place? Because the user needs it, and, if done right, it is safer than on backend.

Identity is anything that could be used to impersonate the user, like tokens or passwords (a password to your system is actually not only your security risk — users reuse the same passwords across many systems, so any password is a valuable asset). The server is not talking to your phone or your app, it recognizes you by your identification tokens and that’s what attackers want.

Why do we even want to store that on the phone? Your phone is a good place to store data access credentials if you plan to access the data from your app: you can execute code around the credentials in a sandbox!

Control — the abilities that are bound to your phone: for example, being able to redirect application flow to make it authorize some backend action. Remember those cars, right?

What should we do?

We face serious threats, this means more work and trying to figure out sophisticated boring technologies invented by people who don’t know anything about mobile tech. However, we have many benefits; we are going to achieve some decent level of security only by understanding what is important.

Understand the strong sides

Things Apple got right for you in the first place.

User trust relationships: you have direct contact with users over (almost) bare metal. What the user types, we can trust most of the time. But don’t trust fingerprints too much: these keys are not easily rotated yet are easily stolen, so they are not good keys.

Trust the device: iPhone is quite well protected, well you know that story with the FBI. What is stored in the local storage is safe enough from anything around. So you have to take care only of your own process, which is good news.

Narrow scope: iOS apps don’t frequently open network ports for listening because we get push notifications for the outside world to ping us. We don’t run 3rd party code embed on runtime. And it’s really complicated to push buffer overflow payload using an iPhone keyboard.

Low collateral risk: unlike the server, you don’t run a gazillion of processes with shared resources. Processes are sandboxed, so privilege escalation and external control flow hijacking are less likely.

Trust: the result of authentication

Now, let’s talk more about trust — the core thing in security.

Trust server less: A server has a lot of moving parts, dependencies and people you don’t know running it. Your app, once compiled, is considerably safer than your NodeJS backend. The server is your backend, but it is subject to more traditional attacks, and should have as little trust as possible by default.

Explicit trust. Trust is given as the result of authentication, not as some default decision. App flows should require constant verification of trust.

Involve users: users are good carriers of secrecy, which is hard to steal. Users operate with physical devices, which are a good auth factor. We’ll talk about using multiple factors to authenticate users later.

Echelonization: add more layers of defense

Echelonized risk management is an idea from ancient warfare. If the system has only one protected perimeter, no matter how strong it is, it will fall, even if it promises all security guarantees in the world.

Okay, what is the solution? Every layer has its own defense, defenses are connected, trust and threats are calculated for each system layer, and for the whole system, too.

It means for mobile apps:

– authenticate important things manually, even within the app’s flow;

– verify credentials and certificates more than once and compare results;

– use multi-factor authentication, use different factors in different combinations;

– protect data with keys, which are stored elsewhere.

Compartmentalization: limit access to everything

The third important principle you should know. The policy of limiting access to the information, making things private by default and allowing access only to entities that need this info to perform their tasks.

Access only the data you need. Where you need. When you need. Never work with full records, only fields you need. Don’t transmit everything over the network. If anything goes wrong, you’ll leak only a part of the data, a minor credential or an insignificant ID.

Make sure trust tokens and protected data are stored separately and are not easily stolen together. Read the corresponding wiki page for deeper understanding.

Practical techniques for securing mobile apps

It may look like very complicated theoretical things. However, most of this hard theory makes a lot of sense in practice.

Classic security techniques

First of all, do all these traditional things:

– protect transport well, pin certificates;

– authenticate everything: both user and server;

– encrypt data in motion and at rest;

– protect keys well.

These techniques will stop most of the attackers. But not all. Let’s talk a bit about advanced ideas.

End-to-end encryption

End-to-end encryption allows you to echelonize risk, and it compartmentalizes sensitive data.

If you don’t trust your backend to store unprotected data, it can’t be stolen from there.

Real end-to-end has only users and their credentials as a source of trust. Servers and network act as the medium only, so their state does not matter much. Their malfunction/compromise is only as bad as Denial of Service attack.

Good end-to-end exists for both data in motion and data at rest. Take a look at the scheme above and read this useful guide, if you have doubts about what iOS crypto library to use in your app. For data in motion, there’re plenty of specialized protocols, with ephemeral keys and strong crypto. With data at rest, you can pick any easy AES wrapper, and just manage the keys correctly. But being quite biased, I advice using Themis Secure Cell anyway :)

Multi-factor authentication

Multi-factor authentication is a very important idea: authenticate users only when they show several proofs of who they are. Talking in a more scientific way, there are three large classes: “thing you know” (password), “thing you have” (mobile phone), “thing you are” (fingerprint). The important idea is that all credentials are independent of one another and cannot be derived from one another.

MFA follows compartmentalization: it involves checking isolated, unrelated things you have to ensure it’s you. If they’re properly isolated, chances are they won’t be stolen at the same time.

MFA means that it’s not enough to use different authentication methods from one class. You should combine methods from different classes: like a fingerprint (thing you are) and password (thing you know), or password plus Google Authenticator, or voice confirmation of your password over a phone (all three).

Zero knowledge proof

Sometimes, even a single request is data leakage. Imagine you need to enter your passport ID to log into some governmental system — now, if the connection or remote party was compromised, attackers know your passport ID. Sometimes you need to transmit sensitive authentication data, but don’t have a trusted channel of communication at all. In this situation, establishing trust is a complicated task, isn’t it?

Zero Knowledge Proof is a protocol, which allows two parties to compare some secret without sending (leaking it). It has real mathematical proofs and was invented by scientists a few decades ago. You may find this small example useful for better understanding of the protocol itself.

If the remote party doesn’t have the same credential as you, the request will fail without leaking the request data. It works for passwords, document ids, and any private credentials with the public identifier. By the way, it combines really well with MFA. Unfortunately, I’m not aware of any iOS implementations other than the library I’m contributing to, so it’s a shameless plug :)

Combining things

Well, remember echelonization? No single tool is good enough. Let’s combine them.

A typical secure app layout has SSL over a network, and it somehow encrypts data on both ends. But, well, it’s barely enough. Such a system may be still vulnerable to MitM and data leakage.

Step 2 makes trust model consistent and compartmentalize things from one another. Let’s keep most of the trust in the user’s hands: encrypt all stored data by keys derived from the user’s password. So only the application can decrypt data only upon password input. As for traffic, encrypt everything with ephemeral keys, which are derived from public keys. This way, the transport and the data itself are separated and have their own layers of protection.

Now, let’s add more novel techniques to enable even better security guarantees: by introducing several authentication factors we will make identity forgery even more complicated and minimize the risks of stealing even encrypted data. Moreover, when the parties only negotiate to exchange the data, adding zero knowledge protocol helps us to stay strong in a case where the whole system gets compromised, and the remote party gets replaced by an evil twin.

Feeling more secure now? :)

Key points

I would be happy if you tap on links from the text/slides and read them. Obviously, one post (even long read) is not enough to describe everything. You may also be interested in reading my other slides, where I discuss some approaches in more depth, like SSL pinning, ephemeral keys and so on. Then put that Security Wizard Hat on and analyze your system from security point of view. Find possible threats and implement a thing or two.

If you have any questions or comments, please, feel free to contact me via Twitter or mail.

Additional reading

– Text, slides and video from my previous talk “Data protection for mobile client server architecture”

– How Apple Security works

– Why you should avoid ssl for your next application

– Choose crypto iOS library that works for you

– OWASP: iOS application security testing cheat sheet

– Zero knowledge proof illustrated primer

Slides

Video

Coming soon. I’ll post it here as soon it is ready :)

P. S.

If you like my post and sketches, tap on 💚!

Originally published on Stanfy’s blog on May 2016.

That’s the follow-up of several talks about data protection for mobile applications I gave at the end of 2015. Geography is wide: twice in Ukraine (mdday.lviv.ua, CocoaHeads Kyiv #7); once in Amsterdam, Netherlands at do{iOS} conf; and in Minsk, Belarus at CocoaConfBy.

We, mobile app developers, usually consider security to be a complicated topic, and that is why security talks are rather tedious. I also watched lots of talks like those. While some security concepts are very difficult to understand, it’s also almost impossible to remember and implement them. I decided to take the other way round and make my talk about data protection easy to grasp, with a plenty of cute pictures.

Look on the slides, watch the video of my talk at do{iOS} conf, or read the whole story.

Real world security vs cyber world security

Let’s start with a real-world security. Meet two dodo birds: Alice and Bob. They are rare species and want to secure themselves from Eve, a Fennec Fox with looooong ears intended to listen all birds’ secrets. When birds notice Eve, they fly away to the bushes.

Well, in the real world things are clear: when somebody wants to hear your secrets — you can physically move away. Avoiding dangers is a part of our nature.

In the cyber world information security requires us to solve puzzling problems for which we don’t have intuitive instruments.

We only know the general direction:

• make communication confidential
• and limit access to the stored goods

It can be surprising, but every program is a potential target for attackers who will try to find a security vulnerabilities in our apps.

We — developers — are responsible for keeping things secure from bad guys.

We really care about users’ data. We want to protect data in the storage (saved to the database, files, NSUserDefaults, etc) and on it’s way from an app to a server.

Protect the data on its way to the server

Developers tend to think mobile apps are quite safe from malicious attacks in their neat sandboxed environments. This is only partially true: data-in-motion is still at risk, and the risk is high: all sorts of man-in-the-middle and passive eavesdropping attacks are still efficient, if you don’t protect the data the right way.

Why is this happening? Security is thought to be tangled and hard to implement. Instead of thinking about building the app based on the fundamental security principles, developers prefer to copy the quick solution from StackOverflow or to implement already existing cryptoalgorithms themselves. It leads to buggy software, full of vulnerabilities and security holes that can be easily cracked.

Thanks to the Apple ATS policies, now almost every iOS9 application uses HTTPS. HTTPS relies on SSL/TLS protocol, that — unfortunately — has a number of security issues and a lot of moving parts, which are easy to break.

Using HTTPS alone doesn’t make your app very secure: there are lots ways to break SSL encryption. From the attacker perspective, it looks like Alice wears a paper hat and Bob encircles himself with a handmade fence, and they feel secure. But they are not.

It’s very important to do SSL right to be more secure: disable old protocols (like sslv3), use long encryption keys and pin certificate inside the client. Well, you can read more in that tiny cheat sheet.

SSL pinning is one of the easiest things to implement inside iOS app that complicates MitM attacks. You just need to pin server’s certificate inside the app, encrypt it and store in a secure place. Well, of course, don’t forget to update certificate before it expires. Many apps are already using SSL pinning, for example, Paypal. As usual, there are amazing libs that can be helpful! For example, it takes only three lines of code on Swift to pin certificate using Pitaya lib.

But using SSL encryption is not enough. SSL was proven to have weak Forward Secrecy, which means that if the encryption key is leaked, all past and future(!) messages can be decrypted. Forward Secrecy is based on using ephemeral keys — keys that are constantly changing — and even if current key is revealed, only small portion of messages is decrypted.

Encrypting data with persistent keys causes the illusion of safety, but it’s just an illusion.

Always encrypt crucial data with ephemeral keys that are re-generated each session, and no one will be able to grab it using the soldering iron.

Before sending data, client and server negotiate about temporary key and use it only during current — rather short — session. Opening new session requires to generate new temporary key. It sounds quite tricky, but there are cool libs for that! Themis and OTRKit libs provide easy client and server interfaces to add encryption to your current client-server architecture.

Keep users data correctly

Okay, enough about data in motion, securing data in storage is also important. iOS has own crypto, protecting user’s data while your device is locked. But what if device is stolen and unlocked?

Wherever you store sensitive data, please, make sure it’s encrypted.

You don’t need to implement everything yourself. Realize possible security risks for your specific app, build the stout architecture based on CIA principle (confidentiality, integrity, authentication), use right tools, and own brains — this is a proper way to create the app that will take lots of time to break.

It’s quite simple: pick the good library, generate the encryption key (aka master password), encrypt data before writing to the storage, decrypt it after reading back. And of course, don’t put your master password plaintext inside your app or in the NSUserDefaults. Use Keychain (SSKeychain and Valet are nice wrappers) or generate master password ‘on the fly’ using custom math formula and KDF.

To summarize, securing your app is rather simple:

– use HTTPS and make sure you’ve disabled old and weak ciphers;
– implement SSL pinning, but encrypt the certificate and keep it safe;
– encrypt data in motion with the additional layer of encryption using the ephemeral keys;
– encrypt data before putting it in the storage and remember to store master key secure or to compute it “on the fly”.

What to see next

• Look at my slides to catch the full story.
• Read amazing blog posts by nabla-c0d3, pay attention on the audit of CryptoCat iOS app.
• Watch the great talk “Swift-ly Secure” by Seth Law.
• Follow Moxie Marlinspike and watch all his talks :)

Let me know if you care about security as much as I do =)

Data Protection For Mobile Client-Server Architectures was originally published in Stanfy Engineering Practices on Medium, where people are continuing the conversation by highlighting and responding to this story.

But we know what to do next!

Have you tried to use Parse.com for your apps? I have. And I liked it! But there’re a couple of things to keep in mind before you blindly sinking into ‘codeless backend’.

This is my story, based on more than 3 years experience of building apps with Parse →

Backend without code. Parse.com: myth vs reality

P.S.

Just want to have all my stories in one place. Take this rubber duck picture for your inconvenience.

This is the second part of post series “How to Estimate mobile projects”. Read also Part 1 — A step by step guide where we described the initial process of breaking down features and estimations.

The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.

— Tom Cargill, Bell Labs

Changes and uncertainty

Sometimes, a project’s details change during development. Some changes are small (changing fonts or colors), some changes are large (adding an extra page or swapping screens). Changes themselves are not a problem, but they certainly affect the initial estimation. Sometimes, in order to make small changes, you need to rewrite a large piece of the system.

Read this developer horror story, which describes how many steps you need to take in order to change the text in the application that is already published in the AppStore.

The future changes are the things that nobody can estimate at the beginning of the project. Therefore, we must keep in mind that initial estimation becomes irrelevant when the project’s scope changes, which is the case in 90% of all projects.

Reinventing the wheel

The world of iOS development has lots of libraries that significantly reduce development time. We use libs like AFNetworking/RestKit, ReactiveCocoa, Runes, Themis, SDWebImage, SVProgressHUD, other UI controls, our own libs and so on. We can focus on the really interesting or complex components, and implement more valuable features before release. The more cool problem-solving libraries are in our arsenal, the more accurate our estimation is.

The mobile world is evolving so fast that we often google before implementing large components of the application, for example, how fast our favourite database is in the new version of iOS. Such research reduces the chance of stumbling upon unexpected problems during development and allows us to be in the know.

Some tasks are more time-consuming than they appear

Never underestimate the challenges that are associated with the real-world features. Among them are such things as time operations, especially with time zones, and specific hardware capabilities like cameras, GPS, push notifications or Bluetooth. It takes more time because development on a real device is “slightly” slower than on the simulator (although Apple tries to make our life easier every year).

In the same category we include integration with other physical devices such as an external microphone or smart watch.

Normally, embedding Apple services such as In-App purchasing and iAd doesn’t take a long time, and most developers think “it’s easy”, but it may take a long time to test, so we usually mark previously mentioned tasks as “time-consuming”.

In addition, the application won’t submit itself to the Store. We add several hours for submission to the overall estimated time. This is especially sad when you have to take lots and lots of screenshots for each localization, but now we can open up the fastlane tool, and things go much faster. Needless to say that it’s easy to forget something in the pre-release atmosphere, so we have created a ‘before submission checklist‘ to keep things in one place.

Must-have features that are always forgotten

What features are present on 80% of the applications, but are often forgotten during the estimation? Screens with a bunch of texts, such as the FAQ, TOS, license agreement and disclaimer. Alert-reminders asking users to rate an application and leave some comment in the store. A “contact us” button to send an email. These features are ‘easy’, but you should put them on the list, too.

Gathering usage statistics is very important in order to make the right decisions in the future. The simplest way is to track each screen opening and the pressing of every button. But sometimes you need to collect more complex events to calculate conversion or to understand whether the design works as expected. To do this, you need to think about what kind of events statistics will be useful and to integrate analytics tools (like Flurry, Google Analytics or Mixpanel).

Communication and documentation also take time

Developers usually estimate only the technical side of the story. Time spent on communication is also important. For example, often it’s necessary to design server-client API before starting the development phase, but sometimes API documentation is being designed on-the-fly. It takes time to coordinate teams, especially if they work remotely. In this case we don’t estimate ‘communication’, but ‘creating/updating API doc’.

According to the agile manifesto, working product is more important than comprehensive documentation, but it doesn’t mean that docs should not be created. When we estimate complicated features we don’t forget to add points to create the documentation. Talking about client-server interactions we often create sequence diagrams. Reading such diagrams instead of digging through the code trying to figure out what’s going on, saves hours, but creation of the diagram also takes time.

Unpredictable team factors

Teams make mistakes.

More often the team underestimates the complexity of the task, but sometimes they can overestimate it. A few tips on how to reduce this risk to a minimum.

Permanently increase the level of the team (gain more Exp!). More projects estimated and done means more XP gained. Besides, it is important to conduct the retrospective team meetings to try to figure out the reasons for inaccurate estimates.

Also, communicating with other teams, listening to their experience and mistakes, helps. And of course, reading a variety of developer resources daily or weekly to monitor community’s pitfalls and trends is also helpful.

Teams change.

Swapping horses when crossing a stream is obviously a bad idea. It’s not a developer’s fault if one team estimated the project, but another one is going to develop it.

Unfortunately, it happens. In this case, it is necessary to discuss and re-estimate the project with the new/updated team. Well, it seems like a waste of time, but in fact, it makes it easier to navigate in the project timeline and to understand the schedule before it gets too late.

Iterate and re-estimate

We talked about rough estimates in Part 1. In fact, agile project development involves continuous planning and evaluation sessions every sprint or two. Those features that were rough estimated should be re-estimated in more detail, taking into account the knowledge of the system that has emerged.

Features estimation, implementation, and estimation analysis are a part of the development cycle. The team gains XP and reduces the risks associated with every new project estimation.

Well,

in the developers’ world, there are many factors that affect the project’s estimation at the different stages. Unfortunately, it’s impossible to take all of them into account. Fortunately, we know many of these factors and know how to minimize risks.

On the one hand, it is crucial to improve the team constantly, creating various and complex projects, sharing practices with other teams and absorbing their experience. On the other hand, development should move by small iterations, constantly keeping one’s eye on the ball to see the possible divergences from the plan as early as possible.

Read More

Part 1 — How to Estimate Mobile Projects. A step by step guide

P.S.

Found useful? Enjoyed my pony sketches? Feel free to like and share post! :)

Originally published on Stanfy’s blog on October 12, 2015.

How to Estimate Mobile Projects 2. Why Estimation Fails and How to Prevent This was originally published in Stanfy Engineering Practices on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the first part of articles series How to Estimate mobile projects. Read also Part 2 — Why Estimation Fails and How to Prevent This to learn about challenges and pitfalls that may occur.

If anyone asks two dev teams to estimate one project, most likely we will get two different estimates because each team’s judgement will be based on their experience and expertise.

In a perfect world, before engineers take off with an estimation,the project is already divided into functional modules, each is well-described, and interactions between them are documented, too. This project is evaluated by a team of experienced developers and they start development instantly. But the real world is not so rosy.

Start development from estimation

There is a widespread opinion that project estimations are a waste of time because they are very volatile in their nature: projects frequently turn out to be more challenging, new demands arrive, new features are added, members of the team come and go.

It is especially true in the case of a startup or an innovative project — where lots of things are to be validated or even completely pivoted. It’s hard to argue with harsh reality, but it’s always fun to look back at the end and realize how the project began and how naive we were.

Estimation of the project is the first step to a successful release.

The estimation is important: it explains how much time is needed to complete the project and get to release. It also helps to move from deliberation of the idea to the elaboration of the detailed plan; to estimate the complexity of system pieces; and to prioritize them.

Dig into the project’s problem domain

Here at Stanfy we estimate projects at different stages. The first stage is a rough estimation when the project only appears on the horizon. Here we explore the project’s industry (medicine, e-commerce, video streaming), and define users and what functions they may need.

We really need to understand the underlying domain and the problem we’re trying to solve.

At the rough estimation stage we devise how the user will work with the app and create the preliminary list of features. In order to make this list more accurate, we design low-fidelity wireframes, which show the user’s flow screen by screen inside the app. Wireframes allow us to put things in perspective: how the app’s features are correlated with the screens, and how the screens are connected to provide an intelligible user experience.

The next step is picking out the most tricky technological pieces: for example, whether the application has an offline mode with synchronization or a barcode scanner or live chat or video streaming. It allows us to understand what technological challenges may arise along the way, and what constraints we should keep in mind.

For example, integration with Spotlight Search is available only starting from iOS9; we should take this into account if we are going to create an app for iOS8, too. The more tricky features an application has, the more the roughly estimated time may differ from the actual one.

At this stage most customers think carefully about their lives, money and plans, and make a decision about whether they are ready to move forward or want to stop. After that comes Stage 2 of estimation, which we call ‘make it as accurate as you can’.

Design detailed screens

Having a detailed design is very important during the estimation phase. Steps to detailed screens are simple: low-fidelity wireframes, high-fidelity wireframe, interactive prototypes, designs. Lo-fi wireframes are created in the previous step, and they can be transformed into high-fidelity ones. Hi-fi wireframes show more details on the screen: layout, buttons, content placeholders. They pay respect to the design guidelines for every platform: hi-fi wireframes for iOS look different from those for Android.

Hi-fi wireframes can be easily transformed into interactive prototypes. One can open prototypes on mobile phones and interact with them like with real applications. Such prototypes are remarkably helpful for developers during the estimation phase; they help to understand an application’s navigation better and visualize its overall architecture.

Anyway, the result of estimating a project which has only wireframes — even interactive ones — is not completely accurate numbers.

The next stage is to transform wireframes into the set of designed screens. Looking at designs, developers understand the app even better: which components to use and whether it’s necessary to write custom elements or it’s enough to customise native ones.

A good design corresponds to guides: the Apple Human Interface Guidelines and the Android Design Guide. The sets of native components for the web, desktops and mobiles are each different, so if the designer uses complex or non-native components, it increases the resulting estimates. For example, the radio button is native for Web applications, but it has the analogues for mobile — a switch or toggle button. Nevertheless, estimating the prototypes is not as accurate as estimating already rendered designs.

Create a detailed feature list

It is hard to estimate the bare idea, so it’s necessary to create a features list or user stories that make up the project. Features are functional pieces that bring value to the user.

Describing features is something outside of the scope of the estimation and has to be done by the product owner’s team. Our development team clarifies and supplements the descriptions with the technical details.

For example, let’s consider a feature such as login via email. It consists of the login screen itself, keyboard behavior (it’s better to send data when the user presses the ‘done’ button on the keyboard than when he needs to hide the keyboard and look for the ‘send’ button behind it), data validation, sending requests to the server, handling responses, showing loading, showing error messages, etc.

Even the login feature that seemed so small hides many details, and a project is a set of such features. Usually we create a simple spreadsheet where features are divided into blocks, with less or more detailed descriptions of each.

Estimate effort

To estimate the feature means to find efforts (time and cost) that are required to get this feature done. There are different methods of estimation, but we use analogy based estimation, correlating the feature with its difficulty: easy, normal, hard, infinity — like game levels!

Let’s see examples.

• Easy: Login, forgot password, FAQ, help screens. Usually, static or dynamic screens, without too complex a design, often without sending requests to the server.
• Normal: Home screen, registration, tables, collections, custom controls. Usual screens with server requests, or custom controls.
• Hard: Third-party API integration, audio/video recording, streaming, chatting, custom gestures, synchronization, offline mode.
• Extreme. Usually they’re unclear and huge features. It’s better to divide them into smaller ones.

Sometimes it’s hard to define where the line is between groups. The rule is simple: round upward if you have doubts.

Let’s look at an example. Almost every mobile application uses tableview to show content: like showing tweets or cat photos. In the mobile world, the table is a list of the cells, which contain texts, pictures, dates and so on.

A table may look very simple, but from a developer’s perspective a simple table involves:

• Design cells (which can contain a lot of information and extra buttons and have flexible content depending on height).
• Loading data (send requests to the server, parse response, create data models)
• Loading more, pull-down-to-refresh.
• Showing loading, empty, and error states (like connection errors).
• Adding/removing items from the table. In my experience, removing items from the table can be very tricky — many apps crash due to data inconsistency.
• Merging content. Sometimes you will need to merge local content with data from the server.
• User interaction when tapping on the cell itself or on a button on the cell (usually the background changes to indicate the selected state and a new screen opens).
• Image caching for pictures in cells (if any).
• Nice animation when cells appear on the screen while scrolling (image fading or light bouncing, depending on the project style).

Looks complicated, right? And after all, a table should look nice and be scrolled smoothly.

But the good thing is that most of these features only need to be written once and can be reused with different tables inside one project or in different projects.

We are working on the list of components that are frequently used in many mobile applications, describing different user scenarios and corner cases. It’s useful to have this list on hand during project estimation to keep in mind unique peculiarities of every component.

The result of this stage is the list of the estimated features. The more hard or even ‘extreme’ features there are in the list, the more time the project takes.

Sort features by priority

Some features are more important than others. Therefore, we divide them into “must have”, “nice to have” and “cool, but later.” “Must-haves” are core features — without them the application won’t serve its purpose.

This technique is called the MoSCoW method; it allows you to focus on the most important requirements to provide users a valuable product as soon as possible.

Prioritizing has to be done by the product owner’s team right after creating the features list. Priorities help developers to focus on the most important features (without spending time on the deep study and evaluation of the features in the bottom of the icebox).

If we are talking about a chat application, for example, then the chatting itself is a core feature, but stickers’ support and sending audio messages can be added in the next release.

Often, a list of “cool but later” features is changed right after the first release when users give their feedback. It may turn out they really want a geolocation function to share with friends, and stickers can be postponed.

R&D features

Of course, there are features that we can’t estimate the value of at first. For such features, we do a little research, google and collect links to understand which way to go and what to expect. At the end of the estimation phase, we highlight such R&D features and consider that they may increase the overall estimated time.

For example, the integration with not very popular third-party services could be rather tricky. If a service is well documented, just to integrate it and be sure that it works correctly can make up around 20% of the overall work. But often, ‘correctly’ doesn’t mean ‘good enough’ — and you devote the remaining 80% of the time to customizing, improving or even hacking your app.

Do not estimate tech practices!

Some things we do not add to the features list even though we do them during development. Among these are engineering practices such as pair programming or writing tests on business logic or backend API. We also “automatically” integrate crash analytics tools (like Crashlytics) and use the continuous integration/continuous delivery services to regularly build and deliver the latest version of the app.

The time is required to set up such things, but usually this time is generously repaid even before the product is released. For example, first we must spend some time to obtain keys from the crash detecting system and configure your application to use them; but then analyzing the crash reports may save you an unbelievable amount of time in finding and fixing crashes before users notice them.

Solutions instead of the problems

During the estimation phase, understanding the challenges ahead is to some extent guesswork, as no problem is a real problem yet, and you have plenty of time to plan, simplify features, reiterate design and proceed on a clear path. If the proposed budget or schedule seems inadequate to do the work, we propose adjusting upward or downward the cost, schedule or features.

In essence, planning is about trying to understand the challenges ahead beforehand. In good teams, it is each member’s responsibility to maintain a special tone during estimations — talk about solutions to ‘possible problems’, not about ‘these monsters ahead’.

So,

the estimation process is a team job: the client describes his own vision, the product owner’s team clarifies the idea and creates and prioritizes the features list. Developers are immersed in the project area, trying to find out as much as possible about the project and the end-users. They estimate features, adding a technical point of view and trying not to forget anything.

Some features are typical for many applications, so they are easier to estimate (for example, you can like posts in almost every social app). It’s more complicated to estimate unique features that no one else has done before. Of course, much depends on the experience of the team and how many different unique applications they have developed.

Read more

Part 2 — Why Estimation Fails and How to Prevent This

P.S.

Found useful? Enjoyed my pony sketches? Feel free to like and share post! :)

Originally published on Stanfy’s blog on October 12, 2015.

How to Estimate Mobile Projects 1. A Step by Step Guide was originally published in Stanfy Engineering Practices on Medium, where people are continuing the conversation by highlighting and responding to this story.