TING SU (苏亭)

Ph.D, Postdoctoral Scholar Department of Computer Science ETH Zurich, Switzerland E-mail: tsuletgo AT gmail DOT com (always valid), or, tingsu AT ethz DOT ch (work)

I am now a postdoctoral scholar in ETH Zurich, Switzerland in the group of Prof. Zhendong Su. Before that, I got my Ph.D in computer science from East China Normal University (ECNU), Shanghai, China (2011-2016), and was surpervised by Prof. Jifeng He and Prof. Geguang Pu. I also had one-year memorable visit in University of California, Davis (UCD), USA (2014-2015), and was advised by Prof. Zhendong Su. I worked as a postdoctoral research fellow with Prof. Yang Liu in Cyber Security Lab (CSL), NTU, Singapore (2016-2019). I also worked as an software engineer in Synopsys, and a research scientist in Rolls-Royce@NTU Research Lab (2016-2017).

My research focuses on developing effective methodologies, techniques and tools to improve software quality, reliability and security.

• Mobile App Analysis/Testing/Debugging/Security/Development
• Symbolic Execution (Test Data Generation in particular, motivated/contributed to SmartRocket Unit)
• Testing Compilers/Program Analyzers/Deep Learning Systems
• Formal Verification

My research work received generous support from Swiss NSF, Google, NTUitive Gap Fund, and won 3 ACM SIGSOFT Distinguished Paper Awards (ICSE 2018, ASE 2018, ASE 2019), 1 Best Research Prototype Tool Award (NASAC 2017, First Place), and 1 Golden Medal of ACM Student Research Competition (ICSE 2016, First Place).

Research Tools/Artifacts

Stoat Star

News

• I will join ECNU as a Professor in Fall 2020.
• I joined the Program Committee of ICST'21, OOPSLA'20 (AEC), ICSE'20 (SEIP), ICECCS'20, ICST'20 and Internetware'20, welcome your submissions.
• We honorably received a Google Faculty Research Award (2019-2020) to support our work on fuzzing mobile apps (me as the Co-PI, with PI: Prof. Zhendong Su)!
• I received a highly competitive research grant (as the sole PI) from Swiss NSF's Spark project (2019-2020) on automated fuzzing techniques for mobile applications!
• Gave an invited talk of our ACM SIGSOFT Distinguished Paper "Large-Scale Analysis of Framework-Specific Exceptions in Android Apps" at ISEC 2019, the flagship software engineering conference in India, held in Pune, India.
• Gave an invited talk "Pushing the limits of automated testing for mobile applications" at A-MOBILE 2018, an ASE workshop, held in Montpellier, France.
• We have released the dataset of Android app crashes/exceptions from our large-scale fault study (Our findings, Who used our dataset?).
• We have released Stoat (including all source code), a fully-automatic, guided model-based GUI testing tool for Android apps. Get the quick idea.
• We have released SmartUnit, a commercial automated unit testing tool for embedded systems. Contact us to know more.

Publications (Google Scholar)

1. Why My App Crashes? Understanding and Benchmarking Framework-specific Exceptions of Android apps
   Ting Su, Lingling Fan, Sen Chen, Yang Liu, Lihua Xu, Geguang Pu, and Zhendong Su
   IEEE Transactions on Software Engineering (TSE), 2020 (Impact Factor: 6.520), pdf, data
   
   An extension of our ICSE'18 paper (won an ACM SIGSOFT Distinguished Paper Award), includes:
   (1) a survey on how app developers analyze, debug/reproduce, detect framework-specific exceptions,
   (2) the reproducibility of existing app testing tools, and
   (3) a comprehensive dataset of android app exception bugs.
   


2. Improving Automated GUI Exploration of Android Apps via Static Dependency Analysis
   Wunan Guo, Liwei Shen, Ting Su, Xin Peng and Weiyang Xie
   36th International Conference on Software Maintenance and Evolution
   ICSME 2020, pdf


3. FREPA: An Automated and Formal Approach to Requirement Modeling and Analysis in Aircraft Control Domain
   Jincao Feng, Weikai Miao, Hanyue Zheng, Yihao Huang, Jianwen Li, Zheng Wang,
   Ting Su, BIn Gu, Geguang Pu, Mengfei Yang, Jifeng He
   28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
   FSE/ESEC 2020 (Industry Track), pdf


4. An Empirical Assessment of Security Risks of Global Android Banking Apps
   Sen Chen, Lingling Fan, Guozhu Meng, Ting Su, Minhui Xue, Yinxing Xue, Yang Liu, and Lihua Xu
   42st International Conference on Software Engineering
   ICSE 2020, pdf


5. The Forgotten Case of the Dependency Bugs: On the Example of the Robot Operating System
   Anders Fischer-Nielsen, Zhoulai Fu, Ting Su, Andrzej Wąsowski
   42st International Conference on Software Engineering
   ICSE 2020 (SEIP), pdf, video (YouTube).


6. Finding and Understanding Bugs in Software Model Checkers
   Chengyu Zhang, Ting Su*, Yichen Yan, Fuyuan Zhang, Geguang Pu and Zhendong Su
   27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
   ESEC/FSE 2019, pdf, slides, bug data
   (*Corresponding author)

Highlights: (1) We found 62 critical bugs in three state-of-the-art software model checkers (CPAchecker, Seahorn and CBMC);
(2) Searhorn and CBMC have integrated our tests as standard benchmarks.


7. Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
   Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen and Changjie Fan
   34th IEEE/ACM International Conference on Automated Software Engineering
   ASE 2019, pdf, press

ACM SIGSOFT Distinguished Paper Award


8. Deep Differential Testing of JVM Implementations
   Yuting Chen, Ting Su*, Zhendong Su
   The 41st ACM/IEEE International Conference on Software Engineering
   ICSE 2019, pdf


9. ReCDroid: Automatically Reproducing Android Application Crashes from Bug Reports
   Yu Zhao, Tingting Yu, Ting Su, Yang Liu, Wei Zheng, Jingzhi Zhang, William G.J. Halfond
   The 41st ACM/IEEE International Conference on Software Engineering
   ICSE 2019, pdf, Tool (ReCDroid)


10. StoryDroid: Automated Generation of Storyboard for Android Apps
    Sen Chen, Lingling Fan, Chunyang Chen, Ting Su, Wenhe Li, Yang Liu, Lihua Xu
    The 41st ACM/IEEE International Conference on Software Engineering
    ICSE 2019, pdf


11. Are Mobile Banking Apps Secure? What Can be Improved?
    Sen Chen, Ting Su*, Lingling Fan, Guozhu Meng, Minhui Xue, Yang Liu, Lihua Xu
    The 26th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering
    FSE 2018 (Industry Paper), pdf
    (*Corresponding author)


12. Efficiently Manifesting Asynchronous Programming Errors in Android Apps
    Lingling Fan, Ting Su*, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu
    The 33rd IEEE/ACM International Conference on Automated Software Engineering
    ASE 2018 (acceptance rate: 19.9%), pdf
    (*Corresponding author)


13. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
    Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang
    The 33rd IEEE/ACM International Conference on Automated Software Engineering
    ASE 2018 (acceptance rate: 19.9%), pdf, www
    
    ACM SIGSOFT Distinguished Paper Award
    


14. Large-Scale Analysis of Framework-Specific Exceptions in Android Apps
    Lingling Fan^#, Ting Su^#*, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu and Zhendong Su
    The 40th International Conference on Software Engineering
    ICSE 2018 (acceptance rate: 20%), pdf, data, slides, bibtex, award, press1，
    (^#Equal Contribution, *Corresponding author)

ACM SIGSOFT Distinguished Paper Award

Highlights: (1) largest and most comprehensive fault study: collected 8,243 framework-specific exceptions (crashes) from 2,486 open-source Android apps by using Stoat and Github, and analyzed their characteristics, manifestation, and fixes.
(2) motivated several follow-up research: bug detection, fault localization and patch generation.


15. From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation
    Chunyang Chen, Ting Su*, Guozhu Meng, Zhenchang Xing and Yang Liu
    The 40th International Conference on Software Engineering
    ICSE 2018 (acceptance rate: 20%), pdf, tool (UI2code)
    (*Corresponding author)


16. SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry
    Chengyu Zhang, Yichen Yan, Hanru Zhou, Yinbo Yao, Ke Wu, Ting Su*, Weikai Miao and Geguang Pu
    The 40th International Conference on Software Engineering
    ICSE 2018 (SEIP), pdf, slides, tool (SmartUnit)
    (*Corresponding author)
    
    Highlights: (1) SmartUnit has been applied in several industry-scale embeded control systems, tested millions lines of C code.
    (2) SmartUnit fully automatically supports unit testing for statement, branch, boundary value and MC/DC coverage.
    


17. AndroVault: Constructing Knowledge Graph from Millions of Android Apps for Automated Analysis
    Guozhu Meng, Yinxing Xue, Jing Kai Siow, Ting Su, Annamalai Narayanan, Yang Liu
    arXiv, 2018


18. Guided, Stochastic Model-Based GUI Testing of Android Apps
    Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, Zhendong Su
    The 11th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering
    FSE 2017 (acceptance rate: 24%), pdf, slides, tool (Stoat), Stoat (Google Site), bibtex, press (in Chinese)
    
    Highlights: Stoat has (1) contributted to these popular apps: WeChat (1 bug), Gmail (1 bug), and Google+ (2 bugs). All these bugs were reported and confirmed/fixed.
    (2) tested 6000+ open-source and commercial Android apps in the past one year, and detected 5800+ fatal crashes.
    (3) won Best Research Tool Award (First Prize, pdf) in NASAC 2017 (National Software Application Conference) held by CCF.
    (3) won NTUitive Gap Fund Grant.
    
    Best (First Place) Research Prototype Tool Award
    


19. A Survey on Data-Flow Testing
    Ting Su, Ke Wu, Weikai Miao, Geguang Pu, Jifeng He, Yuting Chen, Zhendong Su
    ACM Computing Surveys (CSUR), 2017 (Impact Factor : 6.748), pdf, data, bibtex


20. Automated Coverage-driven Testing: Combining Symbolic Execution and Model Checking
    Ting Su, Geguang Pu, Weikai Miao, Jifeng He, Zhendong Su
    SCIENCE CHINA Information Sciences, 2016 (Impact Factor : 1.628, invited paper), pdf


21. Automated Requirements Validation for ATP Software via Specification Review and Testing
    Weikai Miao, Geguang Pu, Yinbo Yao, Ting Su, Danzhu Bao, Yang Liu, Shuohao Chen and Kunpeng Xiong
    The 18th International Conference on Formal Engineering Methods
    ICFEM 2016, pdf


22. FSMdroid: Guided GUI Testing of Android Apps
    Ting Su
    The 38th International Conference on Software Engineering
    ICSE 2016, ACM Student Research Competition, pdf, Press
    
    Golden Medal (First Place) of ACM Student Research Competition
    


23. Coverage-Directed Differential Testing of JVM Implementations
    Yuting Chen, Ting Su, Chengnian Sun, Zhendong Su, Jianjun Zhao
    ACM SIGPLAN Conference on Programming Language Design and Implementation
    PLDI 2016 (acceptance rate: 16%), pdf, slides, talk video


24. Combining Symbolic Execution and Model Checking for Data Flow Testing
    Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, Zhendong Su
    37th {IEEE/ACM} International Conference on Software Engineering
    ICSE 2015 (acceptance rate: 18.5%), pdf, ppt, tool (CAUT)


25. Fm-QCA: A Novel Approach to Multi-value Qualitative Comparative Analysis
    Ke Wu, Shiping Tang, Geguang Pu, Min Wu, Ting Su
    8th International Conference on Knowledge Science, Engineering and Management
    KSEM 2015, pdf


26. Automated Coverage-Driven Test Data Generation Using Dynamic Symbolic Execution
    Ting Su, Siyuan Jiang, Geguang Pu, Bin Fang, Jifeng He, Jun Yan, Jianjun Zhao
    Eighth International Conference on Software Security and Reliability
    SERE 2014 (acceptance rate: 30%), pdf, slides


27. Runtime Verification by Convergent Formula Progression
    Yan Shen, Jianwen Li, Zheng Wang, Ting Su, Bin Fang, Geguang Pu and Wangwei Liu
    21st Asia-Pacific Software Engineering Conference
    APSEC 2014, pdf


28. Modeling and Verification of AUTOSAR OS and EMS Application
    Yunhui Peng, Yanhong Huang, Ting Su, Jian Guo
    Seventh International Symposium on Theoretical Aspects of Software Engineering
    TASE 2013, pdf
    


29. The Semantics and Verification of Timed Service Choreography
    Yongxin Zhao, Hao Xiao, Zheng Wang, Geguang Pu, Ting Su
    International Journal of Computer Mathematics
    IJCM 2013.

Awards

• Google Faculty Research Award (Google, 2019-2020)
• ACM SIGSOFT Distinguished Paper Award (ACM, ASE 2019)
• ACM SIGSOFT Distinguished Paper Award (ACM, ASE 2018)
• ACM SIGSOFT Distinguished Paper Award (ACM, ICSE 2018)
• Nomination for Distinguished PhD Thesis, Selected by China Computer Federation (CCF), 2017
• Best Research Prototype Tool Award (First Prize, NASAC 2017), CCF, 2017
• First Prize in ACM Student Research Competition (ICSE 2016), ACM, 2016
• Outstanding PhD Thesis Cultivation Sholarship, ECNU, 2016
• ACM SIGSOFT Travel Award (ICSE 2015)
• National Scholarship for Doctorate, China, 2014
• Graduate Student Overseas Visiting Scholarship, ECNU, 2014

Academic Services

PC Member: OOPSLA 2020 (AEC), ICSE 2020 (SEIP), ICECCS 2020, ICST 2020, Internetware 2020, ICST 2020 (Industry Track), TASE 2020, TAIC-PART 2020, NASAC 2019, A-Mobile 2019, ICST 2019 (Industry Track), SOFTENG 2019, TASE 2019, INTUITESTBEDS 2019, TAIC-PART 2019

Invited Reviewer of journals:

• Empirical Software Engineering (EMSE)
• Automated Software Engineering Journal (ASE)
• Information and Software Technology (IST)
• Journal of Computer Science and Technology (JCST)
• IEEE Transaction on Reliability
• Frontiers of Computer Science
• Software Quality Journal (SQJ)

Sub-Reviewer of conferences: FSE 2019, TACAS 2019, SANER 2019, ICSE 2019, ASE 2018, ISSTA 2018, TACAS 2018, Internetware 2017, ATVA 2016, PRDC 2015, ATVA 2015, TASE 2015

Teaching

• Master Seminar (Research Topics in Software Engineering)
  Spring 2020, ETH (Lecturer)
• Master Seminar (Research Topics in Software Engineering)
  Fall 2019, ETH
• CZ2006 Software Engineering
  2017/2018 Semester 1, NTU
• Tools of Program Analysis, Testing, and Verification
  2015 Semester 1, ECNU

Other Links

Github: github.com/tingsu

A Bibliography of Papers on Android app testing (Google sheet).

CAUT: a Dynamic Symbolic Execution (DSE)-based test generator for C language

A Bibliography of Papers and Tools on Data Flow Testing, maintained by myself.

A List of Code-based Test Generators, maintained by Zoltán MICSKEI.

A Bibliography of Papers on Symbolic Execution Technique and its Applications, maintained by Saswat Anand, another link

A List of Android research papers and tools, maintained by impillar.

Interesting Articles

Google Coding Style Guides, Research News, The Morning Paper

researchers: Philip Guo, Koushik Sen

labs: xuanwu_lab, SEAL

material: Introduction to Software Testing, Latex for ECNU PhD Thesis, Advice for researchers and students by Michael Ernst,


Last Modification: 2020.1