TING SU (苏亭)

Ph.D, Postdoctoral Scholar Department of Computer Science ETH Zurich, Switzerland E-mail: tsuletgo AT gmail DOT com

I am now a postdoctoral scholar in ETH Zurich, Switzerland in the group of Prof. Zhendong Su. Before that, I got my Ph.D in computer science from East China Normal University (ECNU), Shanghai, China (2011-2016), and was surpervised by Prof. Jifeng He and Prof. Geguang Pu. I also had one-year memorable visit in University of California, Davis (UCD), USA (2014-2015), and was advised by Prof. Zhendong Su. I worked as a postdoctoral research fellow with Prof. Yang Liu in Cyber Security Lab (CSL), NTU, Singapore (2016-2019). I also worked as an software engineer in Synopsys, and a research scientist in Rolls-Royce@NTU Research Lab (2016-2017).

My research focuses on developing effective methodologies, techniques and tools to improve software quality, reliability and security.

• Mobile App Analysis/Testing/Debugging/Security/Development: ICSE'19 (ReCDroid), ICSE'19 (StoryDroid), ASE'18 (APEChecker), FSE'18 (Ausera), ICSE'18 (Stoat+, Exlocator), ICSE'18 (UI2code), FSE'17 (Stoat), ICSE'16 (FSMdroid)
• Symbolic Execution (Test Data Generation in particular): ICSE'18 (SmartUnit) CSUR'17, SCIS'16, ICSE'15 (CAUT), SERE'14 (CAUT)
  motivated/contributed to SmartUnit/SmartRocket Unit (a commerical automated unit testing tool)
• Testing Compilers/Program Analyzers: FSE'19 (MCFuzz), ICSE'19 (classming), PLDI'16 (classfuzz)
• Testing Deep Learning: ASE'18 (DeepGauge)
• Formal Verification: APSEC'14, TASE'13, IJCM'13

Fortunately, my research work has received 3 ACM SIGSOFT Distinguished Paper Awards (ICSE 2018, ASE 2018, ASE 2019), 1 Best Research Prototype Tool Award (NASAC 2017, First Place), and 1 Golden Medal of ACM Student Research Competition (ICSE 2016, First Place).

Research Tools/Artifacts: Stoat (Github repo)

News

• We honorably received a Google Faculty Research Award to support work on functional testing of mobile apps (me as the Co-PI, with PI: Prof. Zhendong Su)!
• I received a highly competitive research grant (as the sole PI) from Swiss SNF's Spark project on automated functional fuzzing techniques for mobile applications!
• I joined the PC of ICSE'20 (SEIP), ICECCS'20, ICST'20 and Internetware'20, welcome your submissions.
• Gave an invited talk of our ACM SIGSOFT Distinguished Paper "Large-Scale Analysis of Framework-Specific Exceptions in Android Apps" at ISEC 2019, the flagship software engineering conference in India, held in Pune, India.
• Gave an invited talk "Pushing the limits of automated testing for mobile applications" at A-MOBILE 2018, an ASE workshop, held in Montpellier, France.
• We have released the dataset of Android app crashes/exceptions from our large-scale fault study (Our findings, Who used our dataset?).
• We have released Stoat (including all source code), a fully-automatic, guided model-based GUI testing tool for Android apps. Get the quick idea.
• We have released SmartUnit, a commercial automated unit testing tool for embedded systems. Contact us to know more.

Publications (Google Scholar)

1. An Empirical Assessment of Security Risks of Global Android Banking Apps
   Sen Chen, Lingling Fan, Guozhu Meng, Ting Su, Minhui Xue, Yinxing Xue, Yang Liu, and Lihua Xu
   42st International Conference on Software Engineering
   ICSE 2020


2. The Forgotten Case of the Dependency Bugs: On the Example of the Robot Operating System
   Anders Fischer-Nielsen, Zhoulai Fu, Ting Su, Andrzej Wąsowski
   42st International Conference on Software Engineering
   ICSE 2020 (SEIP)


3. Finding and Understanding Bugs in Software Model Checkers
   Chengyu Zhang, Ting Su*, Yichen Yan, Fuyuan Zhang, Geguang Pu and Zhendong Su
   27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
   ESEC/FSE 2019, pdf, slides, bug data
   (*Corresponding author)

Highlights: (1) We found 62 critical bugs in three state-of-the-art software model checkers (CPAchecker, Seahorn and CBMC);
(2) Searhorn and CBMC have integrated our tests as standard benchmarks.


4. Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
   Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen and Changjie Fan
   34th IEEE/ACM International Conference on Automated Software Engineering
   ASE 2019, pdf, press

ACM SIGSOFT Distinguished Paper Award


5. Deep Differential Testing of JVM Implementations
   Yuting Chen, Ting Su*, Zhendong Su
   The 41st ACM/IEEE International Conference on Software Engineering
   ICSE 2019, pdf


6. ReCDroid: Automatically Reproducing Android Application Crashes from Bug Reports
   Yu Zhao, Tingting Yu, Ting Su, Yang Liu, Wei Zheng, Jingzhi Zhang, William G.J. Halfond
   The 41st ACM/IEEE International Conference on Software Engineering
   ICSE 2019, pdf, Tool (ReCDroid)


7. StoryDroid: Automated Generation of Storyboard for Android Apps
   Sen Chen, Lingling Fan, Chunyang Chen, Ting Su, Wenhe Li, Yang Liu, Lihua Xu
   The 41st ACM/IEEE International Conference on Software Engineering
   ICSE 2019, pdf


8. Are Mobile Banking Apps Secure? What Can be Improved?
   Sen Chen, Ting Su*, Lingling Fan, Guozhu Meng, Minhui Xue, Yang Liu, Lihua Xu
   The 26th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering
   FSE 2018 (Industry Paper), pdf
   (*Corresponding author)


9. Efficiently Manifesting Asynchronous Programming Errors in Android Apps
   Lingling Fan, Ting Su*, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu
   The 33rd IEEE/ACM International Conference on Automated Software Engineering
   ASE 2018 (acceptance rate: 19.9%), pdf
   (*Corresponding author)


10. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
    Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang
    The 33rd IEEE/ACM International Conference on Automated Software Engineering
    ASE 2018 (acceptance rate: 19.9%), pdf, www
    
    ACM SIGSOFT Distinguished Paper Award
    


11. Large-Scale Analysis of Framework-Specific Exceptions in Android Apps
    Lingling Fan^#, Ting Su^#*, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu and Zhendong Su
    The 40th International Conference on Software Engineering
    ICSE 2018 (acceptance rate: 20%), pdf, data, slides, bibtex, award, press1，
    (^#Equal Contribution, *Corresponding author)

ACM SIGSOFT Distinguished Paper Award

Highlights: (1) largest and most comprehensive fault study: collected 8,243 framework-specific exceptions (crashes) from 2,486 open-source Android apps by using Stoat and Github, and analyzed their characteristics, manifestation, and fixes.
(2) motivated several follow-up research: bug detection, fault localization and patch generation.


12. From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation
    Chunyang Chen, Ting Su*, Guozhu Meng, Zhenchang Xing and Yang Liu
    The 40th International Conference on Software Engineering
    ICSE 2018 (acceptance rate: 20%), pdf, tool (UI2code)
    (*Corresponding author)


13. SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry
    Chengyu Zhang, Yichen Yan, Hanru Zhou, Yinbo Yao, Ke Wu, Ting Su*, Weikai Miao and Geguang Pu
    The 40th International Conference on Software Engineering
    ICSE 2018 (SEIP), pdf, slides, tool (SmartUnit)
    (*Corresponding author)
    
    Highlights: (1) SmartUnit has been applied in several industry-scale embeded control systems, tested millions lines of C code.
    (2) SmartUnit fully automatically supports unit testing for statement, branch, boundary value and MC/DC coverage.
    


14. AndroVault: Constructing Knowledge Graph from Millions of Android Apps for Automated Analysis
    Guozhu Meng, Yinxing Xue, Jing Kai Siow, Ting Su, Annamalai Narayanan, Yang Liu
    arXiv, 2018


15. Guided, Stochastic Model-Based GUI Testing of Android Apps
    Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, Zhendong Su
    The 11th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering
    FSE 2017 (acceptance rate: 24%), pdf, slides, tool (Stoat), Stoat (Google Site), bibtex, press (in Chinese)
    
    Highlights: Stoat has (1) contributted to these popular apps: WeChat (1 bug), Gmail (1 bug), and Google+ (2 bugs). All these bugs were reported and confirmed/fixed.
    (2) tested 6000+ open-source and commercial Android apps in the past one year, and detected 5800+ fatal crashes.
    (3) won Best Research Tool Award (First Prize, pdf) in NASAC 2017 (National Software Application Conference) held by CCF.
    (3) won NTUitive Gap Fund Grant.
    
    Best (First Place) Research Prototype Tool Award
    


16. A Survey on Data-Flow Testing
    Ting Su, Ke Wu, Weikai Miao, Geguang Pu, Jifeng He, Yuting Chen, Zhendong Su
    ACM Computing Surveys, 2017 (Impact Factor : 6.748), pdf, data, bibtex


17. Automated Coverage-driven Testing: Combining Symbolic Execution and Model Checking
    Ting Su, Geguang Pu, Weikai Miao, Jifeng He, Zhendong Su
    SCIENCE CHINA Information Sciences, 2016 (Impact Factor : 1.628, invited paper), pdf


18. Automated Requirements Validation for ATP Software via Specification Review and Testing
    Weikai Miao, Geguang Pu, Yinbo Yao, Ting Su, Danzhu Bao, Yang Liu, Shuohao Chen and Kunpeng Xiong
    The 18th International Conference on Formal Engineering Methods
    ICFEM 2016, pdf


19. FSMdroid: Guided GUI Testing of Android Apps
    Ting Su
    The 38th International Conference on Software Engineering
    ICSE 2016, ACM Student Research Competition, pdf, Press
    
    Golden Medal (First Place) of ACM Student Research Competition
    


20. Coverage-Directed Differential Testing of JVM Implementations
    Yuting Chen, Ting Su, Chengnian Sun, Zhendong Su, Jianjun Zhao
    ACM SIGPLAN Conference on Programming Language Design and Implementation
    PLDI 2016 (acceptance rate: 16%), pdf, slides, talk video


21. Combining Symbolic Execution and Model Checking for Data Flow Testing
    Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, Zhendong Su
    37th {IEEE/ACM} International Conference on Software Engineering
    ICSE 2015 (acceptance rate: 18.5%), pdf, ppt, tool (CAUT)


22. Fm-QCA: A Novel Approach to Multi-value Qualitative Comparative Analysis
    Ke Wu, Shiping Tang, Geguang Pu, Min Wu, Ting Su
    8th International Conference on Knowledge Science, Engineering and Management
    KSEM 2015, pdf


23. Automated Coverage-Driven Test Data Generation Using Dynamic Symbolic Execution
    Ting Su, Siyuan Jiang, Geguang Pu, Bin Fang, Jifeng He, Jun Yan, Jianjun Zhao
    Eighth International Conference on Software Security and Reliability
    SERE 2014 (acceptance rate: 30%), pdf, slides


24. Runtime Verification by Convergent Formula Progression
    Yan Shen, Jianwen Li, Zheng Wang, Ting Su, Bin Fang, Geguang Pu and Wangwei Liu
    21st Asia-Pacific Software Engineering Conference
    APSEC 2014, pdf


25. Modeling and Verification of AUTOSAR OS and EMS Application
    Yunhui Peng, Yanhong Huang, Ting Su, Jian Guo
    Seventh International Symposium on Theoretical Aspects of Software Engineering
    TASE 2013, pdf
    


26. The Semantics and Verification of Timed Service Choreography
    Yongxin Zhao, Hao Xiao, Zheng Wang, Geguang Pu, Ting Su
    International Journal of Computer Mathematics
    IJCM 2013.

Academic Services

PC Member: ICSE 2020 (SEIP), ICECCS 2020, ICST 2020 (Industry Track), TASE 2020, TAIC-PART 2020, NASAC 2019, A-Mobile 2019, ICST 2019 (Industry Track), SOFTENG 2019, TASE 2019, INTUITESTBEDS 2019, TAIC-PART 2019

Invited Reviewer or sub-reviewer of journals and conferences:

• Automated Software Engineering Journal (ASE)
• Information and Software Technology (IST)
• Journal of Computer Science and Technology (JCST)
• IEEE Transaction on Reliability
• Frontiers of Computer Science
• Software Quality Journal (SQJ)
• FSE 2019, TACAS 2019, SANER 2019, ICSE 2019, ASE 2018, ISSTA 2018, TACAS 2018, Internetware 2017, ATVA 2016, PRDC 2015, ATVA 2015, TASE 2015

Teaching

• Master Seminar (Research Topics in Software Engineering)
  Spring 2020, ETH (Lecturer)
• Master Seminar (Research Topics in Software Engineering)
  Fall 2019, ETH
• CZ2006 Software Engineering
  2017/2018 Semester 1, NTU
• Tools of Program Analysis, Testing, and Verification
  2015 Semester 1, ECNU

Other Links

Github: github.com/tingsu

A Bibliography of Papers on Android app testing (Google sheet).

CAUT: a Dynamic Symbolic Execution (DSE)-based test generator for C language

A Bibliography of Papers and Tools on Data Flow Testing, maintained by myself.

A List of Code-based Test Generators, maintained by Zoltán MICSKEI.

A Bibliography of Papers on Symbolic Execution Technique and its Applications, maintained by Saswat Anand, another link

A List of Android research papers and tools, maintained by impillar.

Google Coding Style Guides, Research News, The Morning Paper

researchers: Philip Guo, Koushik Sen

labs: xuanwu_lab, SEAL

material: Introduction to Software Testing, Latex for ECNU PhD Thesis, Advice for researchers and students by Michael Ernst,


Last Modification: 2020.1