ADVANCED SOFTWARE OBFUSCATION TECHNIQUES AND APPLICATIONS

ADVANCED SOFTWARE OBFUSCATION TECHNIQUES AND APPLICATIONS

Abstract

Obfuscation is an important software protection technique that prevents automated or human analyses from revealing the internal design and implementation details of software. There has been a strong demand for advanced obfuscation techniques from software vendors to confront threats like intellectual property thefts and cybersecurity attacks. This dissertation approaches the software protection problem through obfuscation in three different aspects, i.e., techniques, applications, and experiences. The dissertation first introduces translingual obfuscation, a novel software obfuscation technique that makes programs obscure by “misusing” certain features of programming languages derived from highly abstract computation theories. For programs written in imperative languages, which are popular but relatively easy to reverse engineer, translingual obfuscation translates part of a program to another language which has a much more complicated programming paradigm and execution model, thus increasing program complexity. The evaluation shows that this advanced obfuscation technique is suitable for protecting software in desktop and server computation environments. It provides effective and stealthy obfuscation effects with only modest performance cost, compared to one of the most popular commercial obfuscators on the market.

As for mobile software, its development, deployment, and execution are significantly different from those of traditional desktop software, while must less is known about the practice of software protection on this emerging platform. Therefore, the dissertation takes a first step to systematically studying the applications of software obfuscation techniques in mobile app development. With the help of an automated but coarse-grained method, we computed the likelihood of an app being obfuscated for over a million app samples crawled from Apple App Store.

We then inspected the top 6600 most likely obfuscated instances and managed to identify 601 obfuscated versions of 539 iOS apps. By analyzing this sample set with intensive manual effort, we made various observations that help reveal the status quo of mobile obfuscation in the real world. As such, the dissertation can provide insights into understanding and improving the situation of software protection on mobile platforms.

Finally, the dissertation reports field experience of applying obfuscation to multiple commercial mobile apps, each of which serves millions of users. In this case study, we leveraged the knowledge learned from the empirical study. The dissertation discusses the challenges of software obfuscation on the iOS platform and our efforts in overcoming these obstacles. This report can benefit many stakeholders in the mobile ecosystem, including developers, security service providers, and administrators of mobile software ecosystems such as Apple and Google.

Table of Contents

List of Figures            ix

List of Tables xi

Acknowledgments     xii

Chapter 1

Introduction  1

1.1       Demand for Software Protection . . . . . . . . . . . . . . . . . . . .       2

1.2       Advanced Software Obfuscation Techniques         . . . . . . . . . . . . .   7

1.3       Obfuscation in Mobile Software Development . . . . . . . . . . . . .         10

1.4       Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .          12

Chapter 2

Related Work 15

2.1       Software Obfuscation . . . . . . . . . . . . . . . . . . . . . . . . . .   15

2.1.1    Cryptography Obfuscation . . . . . . . . . . . . . . . . . . .       15

2.1.2    Heuristic Obfuscation . . . . . . . . . . . . . . . . . . . . . .          16

2.2       Software Deobfuscation . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3       Programming Language Translation . . . . . . . . . . . . . . . . . .    19

2.4       Empirical Studies on Mobile Apps and Software Obfuscation . . . .           20

Chapter 3

Advanced Obfuscation for Desktop Software        22

3.1       Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .            22

3.2       Threat Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .          26

3.3       Misusing Prolog for Obfuscation . . . . . . . . . . . . . . . . . . . .       27

3.3.1    Prolog Basics . . . . . . . . . . . . . . . . . . . . . . . . . .     27

3.3.2    Obfuscation-Contributing Features . . . . . . . . . . . . . . 28

3.3.2.1 Unification      . . . . . . . . . . . . . . . . . . . . . .            28

3.3.2.2 Backtracking . . . . . . . . . . . . . . . . . . . . .  30

3.4       Technical Challenges . . . . . . . . . . . . . . . . . . . . . . . . . .    32

3.4.1    Control Flow  . . . . . . . . . . . . . . . . . . . . . . . . . .     32

3.4.2    Memory Model . . . . . . . . . . . . . . . . . . . . . . . . .    32

3.4.3    Type Casting . . . . . . . . . . . . . . . . . . . . . . . . . .      33

3.5       C-to-Prolog Translation        . . . . . . . . . . . . . . . . . . . . . . . .        33

3.5.1    Control Flow Regularization . . . . . . . . . . . . . . . . . .      33

3.5.1.1 Control Flow Cuts . . . . . . . . . . . . . . . . . .           34

3.5.1.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . .       35

3.5.2    C Memory Model Simulation            . . . . . . . . . . . . . . . . .        36

3.5.2.1 Supporting C Memory-Access Operators . . . . . .      36

3.5.2.2 Maintaining Consistency . . . . . . . . . . . . . . .     37

3.5.3    Supporting Other C Features           . . . . . . . . . . . . . . . . .        39

3.5.3.1 Struct, Union, and Array       . . . . . . . . . . . . . . 39

3.5.3.2 Type Casting . . . . . . . . . . . . . . . . . . . . .  40

3.5.3.3 External and Indirect Function Call . . . . . . . .           40

3.5.4    Obfuscating Translation . . . . . . . . . . . . . . . . . . . .         40

3.6       Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .       41

3.6.1    Preprocessing and Translating C to Prolog . . . . . . . . . .       42

3.6.2    Combining C and Prolog . . . . . . . . . . . . . . . . . . . .         44

3.6.3    Customizing Prolog Engine  . . . . . . . . . . . . . . . . . .      45

3.7       Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .           45

3.7.1    Potency . . . . . . . . . . . . . . . . . . . . . . . . . . . . .         47

3.7.2    Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . .       49

3.7.2.1 Resilience to Semantics-Based Binary Diffing        . . .        50

3.7.2.2 Resilience to Syntax-Based Binary Diffing . . . . .     51

3.7.2.3 Comparing Babel with Code Virtualizer     . . . . .     53

3.7.3    Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .            54

3.7.4    Stealth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .         56

3.8       Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .           58

3.8.1    Generalizing Translingual Obfuscation       . . . . . . . . . . . .     58

3.8.2    Multithreading Support        . . . . . . . . . . . . . . . . . . . .   59

3.8.3    Randomness . . . . . . . . . . . . . . . . . . . . . . . . . . .    59

3.8.4    Defeating Translingual Obfuscation . . . . . . . . . . . . . .            60

Chapter 4

Status Quo of Obfuscation in Mobile Development           62

4.1       Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .           65

4.1.1    The ARM Architecture . . . . . . . . . . . . . . . . . . . . .          65

4.1.2    The iOS Mobile Operating System   . . . . . . . . . . . . . . 66

4.1.3    The Objective-C and Swift Programming Languages        . . . .      66

4.1.4    Technical Challenges of the Study . . . . . . . . . . . . . . .  67

4.1.4.1 Obfuscation Detection and Analysis            . . . . . . . .           67

4.1.4.2 Static Third-Party Libraries . . . . . . . . . . . . .   68

4.1.5    Inferring Developer Intentions . . . . . . . . . . . . . . . . .   69

4.2       Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .          69

4.2.1    Considered Obfuscations . . . . . . . . . . . . . . . . . . . .        70

4.2.2    Mining Obfuscated iOS Apps            . . . . . . . . . . . . . . . . .        72

4.2.3    Per-App Inspection . . . . . . . . . . . . . . . . . . . . . . .            73

4.2.3.1 Detecting Obfuscation . . . . . . . . . . . . . . . .       74

4.2.3.2 Identifying Obfuscated Third-Party Libraries        . . .        75

4.2.4    Cross-Validation        . . . . . . . . . . . . . . . . . . . . . . . .        75

4.3       Detecting Symbol Obfuscation . . . . . . . . . . . . . . . . . . . . .         76

4.3.1    An NLP-Based Detection Model . . . . . . . . . . . . . . . .    76

4.3.2    Implementation . . . . . . . . . . . . . . . . . . . . . . . . .  77

4.4       Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4.4.1    Characteristics of Obfuscated Apps . . . . . . . . . . . . . . 79

4.4.2    Obfuscation Patterns . . . . . . . . . . . . . . . . . . . . . .           84

4.4.3    Impact of Distributor Code Review . . . . . . . . . . . . . . 90

4.4.4    Effectiveness of Obfuscation . . . . . . . . . . . . . . . . . .      93

4.5       Implications of the Results . . . . . . . . . . . . . . . . . . . . . . .            94

Chapter 5

A Case Study on Real-World Mobile Obfuscation  97

5.1       Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   98

5.2       Obfuscation Algorithms        . . . . . . . . . . . . . . . . . . . . . . . .        99

5.3       Implementation Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . 104

5.3.1    Inline Assembly . . . . . . . . . . . . . . . . . . . . . . . . . 104

5.3.2    Heterogeneous Hardware . . . . . . . . . . . . . . . . . . . . 106

5.3.3    App Maintainability   . . . . . . . . . . . . . . . . . . . . . . 106

5.4       Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.4.1    Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.4.2    Overhead       . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

5.4.2.1 Size Expansion . . . . . . . . . . . . . . . . . . . . 111

5.4.2.2 Execution Slowdown . . . . . . . . . . . . . . . . . 112

5.5       Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

5.5.1    Dilemma of Security and Transparency . . . . . . . . . . . . 113

5.5.2    Other Protections . . . . . . . . . . . . . . . . . . . . . . . . 114

Chapter 6

Conclusion     116

Appendix A

Additional Potency Evaluation Data For Babel      118

Appendix B

Publications During Ph.D.    121

Bibliography  124

List of Figures

5.1       Obfuscation configuration examples . . . . . . . . . . . . . . . . . . 100

5.2       Example of obfuscation utilizing LLVM IR inline assembly . . . . . 103

5.3       Effectiveness of disassembly disruption . . . . . . . . . . . . . . . . 111

List of Tables

5.1       Performance of IDA Pro Function Recognition . . . . . . . . . . . . 108

5.2       Binary Size Expansion Due to Obfuscation . . . . . . . . . . . . . . 112

A.1 Program Complexity of Babel-Obfuscated Binaries at Different

Obfuscation Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

A.2 Program Complexity of CV-Obfuscated Binaries at Different Obfuscation Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Chapter 1

Introduction

Software protection is critical in the software industry. According to a study [23] by the Software Alliance, software piracy leaded to 52.2 billion dollars of unlicensed software installation on desktop and server computers, in 2015. Since software piracy is usually linked to malicious incidents, a total loss of 400 billion dollars were caused, directly and indirectly, by software piracy in the same year. On mobile platforms, the situation is even more severe. Concerns on security breaches targeting mobile apps have kept rising in past years. It was reported that the piracy rates of popular mobile apps can approach to 60–95% [21] . Research by Gibler et al. found that a surprisingly large portion of mobile applications are “copies” of others [73] .

This introductory chapter discusses the motivation of the research in the dissertation. In particular, we discuss the role of obfuscation techniques in software protection and explain why advanced obfuscation techniques and their appropriate deployment are critical to software developers. The chapter also summarizes the contributions of the dissertation.

1.1       Demand for Software Protection

Besides these traditional intellectual property theft problems, the industry is also facing new security threats as there are now many businesses heavily relying on mobile devices to operate across the globe. In most cases, mobile software is easier to reverse engineer than desktop software. Although, from the research point of view, there exist various challenges in automated reverse engineering that cannot be easily addressed, leading to beliefs that reverse engineering is not a realistic threat to common mobile software vendors. In reality, however, many of such challenges can be practically addressed or circumvented

Take iOS apps as examples. Since most iOS apps are built with the standard toolchain provided by Apple, the shapes of their binary code are utterly uniform. This is a highly desired situation for reverse engineering. By analyzing the common code patterns and developing corresponding analysis heuristics, modern binary analysis tools have grown reasonably proficient at decompiling iOS apps, making reverse engineering much less laborious than before. Figure 1.1 is an example that demonstrates the quality of the decompilation result for a popular open source iOS app. The decompilation is done by IDA Pro [15] , the most widely used reverse engineering toolkit in industry. As can be seen, the generated pseudocode is almost identical to the original source code, except for the language implementation details which are implicit in the source code but recovered by the decompiler, e.g., the self pointer. To experienced reverse engineers, these differences are negligible.

In addition to the support of increasingly mature analysis tools, reverse engineering is made even more effective on iOS due to its development and production environment. The majority of iOS apps are written in Objective-C, a C-like, object-oriented, and fully reflexive programming language developed by Apple. In Objective-C, method names are called selectors and method invocations are implemented in a message forwarding scheme. When a method is called on an object, the language runtime will dynamically walk through the dispatch table of the class

1 @implementation TSAnimatedAdapter 2 …

3

4          − (BOOL)canPerformEditingAction:(SEL)action { 5          return (action == @selector(copy:) 6            | | action == NSSelectorFromString(@“save:”));

7          }

8

…

@end

(a) Original Objective-C source code

1 // TSAnimatedAdapter − (bool)canPerformEditingAction:(SEL) 2 bool cdecl −[TSAnimatedAdapter canPerformEditingAction:]

3 (struct TSAnimatedAdapter ∗self, SEL a2, SEL a3) { 4   bool result; 5  if ( “copy:” == a3 ) 6   result = 1;

7          else

8          result = NSSelectorFromString(CFSTR(“save:”)) == ( QWORD)a3; 9     return result;

10        }

(b) Pseudocode obtained from decompiling the binary

Figure 1.1: Decompiling an open source iOS app [14] with IDA Pro

of the object to find a method implementation whose name matches with the selector. If no match is found, the runtime will repeat the procedure on the object’s base class. This process is similar to the prototype system of JavaScript, except that the method name matching in Objective-C is conducted through class metadata while the similar process in JavaScript is conducted through the properties of each object. Naturally, the message forwarding scheme requires the Objective-C compiler to preserve all method names in program binaries. Method names are extremely useful information when analyzing large software binaries, for it allows human analysts to infer program semantics and quickly identify critical points worth in-depth inspection among a huge amount of code.

On the Android platform, there is a similar problem since Java is also a fully reflexive language. Having realized the potential risks, Google integrated a method and class name scrambler into the Android development toolchain [24] . In contrast, iOS developers do not get any support from Apple, leaving all code completely unprotected by default. Furthermore, Apple now advises iOS developers to submit apps in the form of LLVM intermediate representation, which is even less challenging to analyze than ARM machine code. Overall, reverse engineering iOS apps can be made very effective if developers do not take actions of prevention.

The lack of technical challenges in analyzing unprotected mobile apps grants adversaries strong reverse engineering capabilities and allows their malevolent attempts of exploiting mobile apps for illegal benefits to succeed with a high chance. App-specific vulnerabilities can certainly be devastating if their presences are learned by attackers. For example, a previous version of Uber’s mobile app was found vulnerable and therefore can be exploited to get unlimited free rides [1] . On the other hand, besides those specialized threats, there also exist attacks that are generally applicable to many apps. We describe three common kinds of them, besides the typical intellectual property theft problem.

Man-in-the-middle attacks. By tricking users into connecting mobile devices to untrusted wireless networks or installing SSL certificates from unknown sources, attackers can intercept and counterfeit the communication between apps and servers [80] . After analyzing how apps process the data exchanged with servers, attackers can potentially control app behavior by forging certain server responses.

Repackaging. It has been reported that some cybercrime groups are able to reverse engineer popular social networking apps and weaponize them for stealing sensitive user information [48] . By developing information-stealing modules and repackaging them into genuine apps, attackers managed to create malicious mobile software with seemingly benign appearances and functionality. Contacts, chat logs, web browsing histories, and voice recordings are common targets of theft.

Fraud, spam, and malicious campaigns. Nowadays, many apps employ anomaly detection to identify suspicious client activities and prevent incidents like fraud, spam, and malicious campaigns. This is usually achieved through collecting necessary information about users and their devices and fitting the collected data into anomaly detection models. Since the data are harvested on device, attackers can reverse engineer the mobile apps and find out what kinds of data are being collected. In this way, they may be able to mimic normal user behavior by fabricating false data of the same kinds on rooted and jailbroken devices.

During the past few years, our industry collaborators have encountered many incidents of categories described above, among which the most concerning ones are the increasingly prevalent large-scale malicious campaigns. According to a report on fraudulent campaigns conducted in China [5] , the business of “click farming” has formed a billion-dollar underground economy, in which hundreds of well organized collusive groups have participated. The technological means used to support these campaigns are also evolving quickly. Campaign runners can now programmatically control hundreds of mobile devices without the involvement of human labor, while

Figure 1.2: Programmatically controlling massive iOS devices as a service (http: //shemeitong.com/index.php/anli/show/46.html).

each device can host over 50 instances of the same mobile app. Figure1.2 shows an example of such technology.

Since the third quarter of 2016, our collaborators in the mobile software industry have captured a large-volume of suspicious activities being conducted around the resources and services offered to mobile app users. Through information crossvalidation, we detected that there are millions of suspicious iOS devices, many of which are virtually faked, constantly trying to log into the account system of the apps, committing massive promotion operations like clicking links to a certain product, posting comments to a certain page, and exhaustively collecting bonuses provided to daily active users. Many of these activities have violated end user terms and affected the quality of the services.

To detect the malicious campaigns and nullify their impacts, app developers need to precisely identify those bot-like users through extensive data analysis. Since data collection must strictly respect user privacy, only certain types of data can be collected for this purpose, which attackers can easily guess out. For the sake of data genuineness, we have to ensure that malicious groups cannot tamper with the on-device data collection process through reverse engineering the corresponding program logic, which requires effective software protection techniques to be deployed.

1.2       Advanced Software Obfuscation Techniques

In general, obfuscation takes effect by hindering the program analysis capabilities of adversaries, which are the basics of many malicious activities targeting commodity software. There are two types of obfuscation techniques, the first of which originates from the research of cryptography and seeks to build mathematically hard-to-analyze programs. Different notions have been proposed to formally define effective obfuscation. In general, however, theoretically secure obfuscation algorithms are either impossible to craft or too expensive to be used for protecting software in production, depending which notion is considered. The second type of software obfuscation is based on heuristics rather than theoretical theorems. Typically, heuristic-base obfuscation are semantics-preserving program transformations that aim to make a program more difficult to understand and reverse engineer. In contrast to theoretical obfuscation, heuristic obfuscation has no guaranteed resilience and is potentially vulnerable to unknown attack methods. On the other hand, it is much more practical than obfuscation based on cryptography constructs and has been employed in real-world software development. The idea of using obfuscating transformations to prevent reverse engineering can be traced back to Collberg et al. [51, 52, 108] . Since then many obfuscation methods have been proposed [94, 106, 116, 125, 45, 145] . On the other hand, malware authors also heavily rely on obfuscation to compress or encrypt executable binaries so that their products can avoid malicious content detection [129, 127] .

Currently the state-of-the-art obfuscation technique is to incorporate with processlevel virtualization. For example, obfuscators such as VMProtect [28] and Code Virtualizer [7] replace the original binary code with new bytecode, and a custom interpreter is attached to interpret and execute the bytecode. The result is that the original binary code does not exist anymore, leaving only the bytecode and interpreter, making it difficult to directly reverse engineer [77] . However, recent work has shown that the decode-and-dispatch execution pattern of virtualizationbased obfuscation can be a severe vulnerability leading to effective deobfuscation [54, 126] , implying that we are in need of obfuscation techniques based on new schemes.

To help the software production community withstand the threats from malicious reverse engineering, this dissertation delivers research results that can advance the status quo of software protection by obfuscation. The research is two fold. Firstly, we propose a novel and practical obfuscation method called translingual obfuscation, which possesses strong security strength and good stealth, with only modest cost. The key idea is that instead of inventing brand new obfuscation techniques, we can exploit some existing programming languages for their unique design and implementation features to achieve obfuscation effects. In general, programming language features are rarely proposed or developed for obfuscation purposes; however, some of them indeed make reverse engineering much more challenging at the binary level and thus can be “misused” for software protection. In particular, some programming languages are designed with unique paradigms and have very complicated execution models. To make use of these language features, we can translate a program written in a certain language to another language which is more “confusing”, in the sense that it consists of features leading to obfuscation effects.

In this dissertation, we obfuscate C programs by translating them into Prolog, presenting a feasible example of the translingual obfuscation scheme. C is a traditional imperative programming language while Prolog is a typical logic programming language. The Prolog language has some prominent features that provide strong obfuscation effects. Programs written in Prolog are executed in a search-and-backtrack computation model which is dramatically different from the execution model of C and much more complicated. Therefore, translating C code to Prolog leads to obfuscated data layouts and control flows. Especially, the complexity of Prolog’s execution model manifests mostly in the binary form of the programs, making Prolog very suitable for software protection.

Translating one language to another is usually very difficult, especially when the target and source languages have different programming paradigms. However, we made an important observation that for obfuscation purposes, language translation could be conducted in a special manner. Instead of developing a “clean” translation from C to Prolog, we propose an “obfuscating” translation scheme which retains part of the C memory model, in some sense making two execution models mixed together. We believe this improves the obfuscating effect in a way that no obfuscation methods have achieved before, to the best of our knowledge. Consequently in translingual obfuscation, the obfuscation does not only come from the obfuscating features of the target language, but also from the translation itself. With this new translation scheme we manage to kill two birds with one stone, i.e., solving the technical problems in implementing translingual obfuscation and strengthening the obfuscation simultaneously.

We have implemented translingual obfuscation in a tool called Babel. Babel can selectively transform a C function into semantically equivalent Prolog code and compile code of both languages together into the executable form. Our experiment results show that translingual obfuscation is obscure and stealthy. The execution overhead of Babel is modest compared to a commercial obfuscator. We also show that translingual obfuscation is resilient to one of the most popular reverse engineering techniques.

1.3       Obfuscation in Mobile Software Development

Apart from developing new obfuscation techniques, the dissertation also aims to advance the application and deployment of software protection particularly on the emerging mobile platform. The prosperity of smartphone markets has raised new concerns about software security on mobile platforms, leading to a growing demand for effective software obfuscation techniques. Although software obfuscation has been intensively studied for the traditional desktop computing environment, the status of mobile application obfuscation is yet to be reviled. As far as we have learned, little emphasis is put on investigating how benign software authors take obfuscation as part of their development process in the real world, which is critical for software obfuscation techniques to be practical. Therefore, both the academia and the industry are interested in a comprehensive study on the latest status of mobile app obfuscation so that the strength of mobile software protection techniques can be understood and improved. This dissertation aims to fulfill this demand.

The mobile ecosystem is currently dominated by two major platforms, i.e., iOS and Android. The two systems are very distinguishable with respect to the development, deployment, and execution of mobile applications. iOS applications are written in C and C-like programming languages and compiled to native machine code before installed and executed on devices. In contrast, Android applications are mostly written in Java and compiled to byte code. Before Android 5.0, Android applications are executed in a managed environment called the Dalvik virtual machine. Starting from 5.0, application bytecode will be further compiled into native code right before being installed onto the devices. Either way, Android developers cannot directly obfuscate their applications on the native code level. In general, obfuscating Android application is more specific to the Java programming language, which is a unique research topic [65, 153] .

In this dissertation, we focus on studying the iOS mobile system. By the time of this research being conducted, there are more than one million iOS applications published. By analyzing the latest versions of this large set of applications, we are able to get a comprehensive understanding of how software obfuscate is practiced in the iOS ecosystem, where millions of developers and hundreds of millions of users are involved.

As a step forward to investigating the applications of obfuscation techniques in real-world software development, we collaborate with mobile app developers in the industry and develop obfuscators to protect multiple commercial iOS apps with millions of users, leveraging the knowledge obtained from the empirical study. The dissertation reports our experience of obfuscating multiple commercial iOS apps with millions of active users. To date, there exist various supposedly effective obfuscation techniques that may fulfill the demand of the mobile software industry. However, the techniques themselves do not automatically lead to effective and practical software protection, especially for mobile apps. Oftentimes, the hardware and software environments of mobile platforms impose harsh restrictions on the types and configurations of obfuscations that can are applied to mobile apps. Additionally, obfuscation must not affect the regular development, distribution, and maintenance of mobile apps, which usually requires further customization to be made for the adopted obfuscation techniques.

1.4       Contributions

In summary, we make the following contributions in this dissertation:

On developing novel software obfuscation techniques,

We proposed a new obfuscation method named translingual obfuscation. Translingual obfuscation is novel for utilizing exotic language features instead of ad-hoc program transformations to protect programs against reverse engineering. Our new method has a number of advantages over existing obfuscation techniques, which will be discussed in depth in later chapters.

We implemented translingual obfuscation in a tool called Babel which translates C to Prolog at the scale of subroutines, i.e., from C functions to Prolog predicates, to obfuscate the original programs. Language translation is always a challenging problem, especially when the target language has a heterogeneous execution model.

We evaluated Babel with respect to all four evaluation criteria proposed by Collberg et al. [52] : potency, resilience, cost, and stealth, on a set of real-world C programs with quite a bit of complexity and diversity. Our experiments demonstrate that Babel provides strong protection against reverse engineering with only modest cost.

On investigating the status of software obfuscation application in real-world mobile software development,

We are the first to conduct a comprehensive empirical study targeting mobile software obfuscation. Our research focuses on iOS, an influential mobile platform that did not receive enough attention from the academia in contrast to Android.

We developed a scalable detection algorithm to estimate the likelihood of an iOS app being obfuscated and applied it to a large quantity of apps crawled from App Store. After manually analyzing the 6600 most likely obfuscated instances, we identified 539 truly obfuscated iOS apps with a total of 601 different versions. As far as we know, this is the first scientifically collected sample set of obfuscated iOS mobile apps. We plan to share these samples with the community in the future.

To overcome the limitations of existing automated software analysis on obfuscated binaries, we invested over 600 man-hours in manually examining the obfuscated iOS apps, extracting detailed information about how these apps are protected by different obfuscation algorithms. The human effort assured the accuracy of our analysis and therefore the credibility of our findings.

We made various observations about the characteristics of obfuscated apps, the obfuscation patterns applied, and their resilience to reverse engineering. Our findings can shed light on future research on mobile software protection.

For applying obfuscation to iOS apps with large user bases, we help mobile developers form a deeper understanding of software obfuscation and avoid common pitfalls that may appear when obfuscating iOS apps, we discuss our learned lessons on the following topics,

Why iOS apps are in urgent need of the protection of software obfuscation, from an industrial point of view,

What restrictions are imposed by the iOS platform on obfuscation techniques,

How the centralized app distribution process can impact practice of obfuscation, and

How to balance obfuscation and app maintenance.

The report will benefit both mobile app developers, distributors, and researchers aiming to develop advanced obfuscation techniques applicable to mobile software.

[1] Findings and conclusions in this dissertation do not necessarily reflect the view of the funding agencies.

ADVANCED SOFTWARE OBFUSCATION TECHNIQUES AND APPLICATIONS