Property-based Testing(基于性质的测试)
Property-based Testing的经典论文:
-
Fink, George, and Matt Bishop. “Property-based testing: a new approach to testing for assurance.” ACM SIGSOFT Software Engineering Notes 22.4 (1997): 74-80. https://dl.acm.org/doi/pdf/10.1145/263244.263267
-
Koen Claessen, John Hughes: QuickCheck: a lightweight tool for random testing of Haskell programs. ICFP 2000: 268-279 https://dl.acm.org/doi/10.1145/357766.351266
Property-based Testing的应用论文
[Mobile Apps测试]
-
Lam, Edmund SL, Peilun Zhang, and Bor-Yuh Evan Chang. “ChimpCheck: property-based randomized test generation for interactive apps.” Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. 2017. https://github.com/cuplv/chimpcheck
[Web Apps测试]
-
O‘Connor, Liam, and Oskar Wickström. “Quickstrom: property-based acceptance testing with LTL specifications.” Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation. 2022. https://github.com/quickstrom/quickstrom
[ROS系统测试]
-
André Santos, Alcino Cunha, Nuno Macedo: Property-based testing for the robot operating system. A-TEST@ESEC/SIGSOFT FSE 2018: 56-62 https://dl.acm.org/doi/10.1145/3278186.3278195
[Java SDK Library测试]
-
Rohan Padhye, Caroline Lemieux, Koushik Sen: JQF: coverage-guided property-based testing in Java. ISSTA 2019: 398-401 https://dl.acm.org/doi/pdf/10.1145/3293882.3339002 (工具: https://github.com/rohanpadhye/jqf )
[其他一些软件系统的测试]
-
Andreas Stahlbauer, Marvin Kreis, Gordon Fraser: Testing scratch programs automatically. ESEC/SIGSOFT FSE 2019: 165-175 https://dl.acm.org/doi/10.1145/3338906.3338910
-
Stefan Karlsson, Adnan Causevic, Daniel Sundmark: QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIs. ICST 2020: 131-141 https://ieeexplore.ieee.org/document/9159071/
-
Andreas Löscher, Konstantinos Sagonas: Targeted property-based testing. ISSTA 2017: 46-56 https://dl.acm.org/doi/10.1145/3092703.3092711
可以理解为也属于Property-based Testing的应用论文
- Park M, Jang H, Byun T, et al. Property-based testing for LG home appliances using accelerated software-in-the-loop simulation[C]//2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 2020: 120-129.
- Ruijie Meng, Zhen Dong, Jialin Li, Ivan Beschastnikh, and Abhik Roychoudhury. 2022. Linear-time temporal logic guided greybox fuzzing. In Proceedings of the 44th International Conference on Software Engineering (ICSE '22). Association for Computing Machinery, New York, NY, USA, 1343–1355. https://doi.org/10.1145/3510003.3510082
Property-based Testing的talks:
- Oskar Wickström - Property-Based Testing The Ugly Parts: Case Studies from Komposition https://www.youtube.com/watch?v=z2ete8VZnZY
- Hughes - Don‘t Write Tests. https://www.youtube.com/watch?v=hXnS_Xjwk2Y
- John Hughes - Testing the Hard Stuff and Staying Sane. https://www.youtube.com/watch?v=zi0rHwfiX1Q
Property-based Testing的一些其他参考资料(都讲的很好)
- Overview of Property-Based Testing Functionality,https://github.com/jmid/pbt-frameworks (很不错的一个资源,是一个丹麦的教授维护的,他自己也教授functional programming, 汇集了不少PBT的框架比较和资源)
- Model-based Testing (Stateful and Model-based Properties with Examples in Java): https://johanneslink.net/model-based-testing/ (这篇技术博客把广义上的model-based testing讲的非常清楚)
- Relational Problems to Teach Property-Based Testing (by John Wrenna , Tim Nelsona , and Shriram Krishnamurthia) https://cs.brown.edu/~tbn/publications/wnk-pj20-pbt.pdf
- Property-based Testing and Test Oracles https://swen90006.github.io/notes/Property-based-testing.html#
(澳洲一个大学的测试课教程,Section 6.4.3 - Metamorphic oracles, Section 6.4.7 - Oracle derivation, Section 6.5 - Property-based testing。关于MT和PBT这块讲解的挺清楚,其中谈到Metamorphic Testing(MT)是属于Property-based testing(PBT)的一种测试方法)
与Property-based Testing工具相关的资料:
- MacIver D R, Hatfield-Dodds Z. Hypothesis: A new approach to property-based testing[J]. Journal of Open Source Software, 2019, 4(43): 1891. https://joss.theoj.org/papers/10.21105/joss.01891.pdf
- junit-quickcheck: https://github.com/pholser/junit-quickcheck
- Hedgehog: https://hedgehog.qa/
- hypothesis: https://github.com/HypothesisWorks/hypothesis/