During the International Working Conference on Source Code Analysis & Manipulation (SCAM), a GrammaTech research publication was awarded the Institute of Electrical and Electronics Engineers (IEEE) Computer Society TCSE (Technical Council on Software Engineering) Distinguished Paper Award.
The publication was chosen through further rounds of review and voting among program committee members. The paper, available here for viewing, details Bug-Injector, a system that automatically creates benchmarks for customized evaluation of static analysis tools.
Bug-Injector works by inserting bugs based on bug templates into real-world host programs. It runs tests on the host program to collect dynamic traces, searches the traces for a point where the state satisfies the preconditions for some bug template, then modifies the host program to “inject” a bug based on that template. Injected bugs were used as test cases to build a static analysis tool evaluation benchmark. Bug-Injector pairs every injected bug with the program input that exercises that bug. The team identified a broad range of requirements and desiderata for bug benchmarks; their approach generated on-demand test benchmarks to meet these requirements. It also allowed them to create customized benchmarks suitable for evaluating tools for a specific use case (e.g., a given codebase and class of bug). Their experimental evaluation demonstrates the suitability of the generated benchmark for evaluating static bug-detection tools and for comparing the performance of different tools.
According to their website, the aim of the International Working Conference on Source Code Analysis & Manipulation (SCAM) is to bring together researchers and practitioners working on theory, techniques and applications which concern analysis and/or manipulation of the source code of computer systems. While much attention in the wider software engineering community is properly directed towards other aspects of systems development and evolution, such as specification, design and requirements engineering, it is the source code that contains the only precise description of the behavior of the system. The analysis and manipulation of source code thus remains a pressing concern.
For more information, check out our blog post detailing Bug-Injector.
Visit us at Embedded World 2020 in Nuremberg/Germany from February 25 to February 27! Meet us in Hall 4, booth 4-423.
Bug-Injector research was sponsored by the Defense Advanced Research Projects Agency (DARPA) under Contract No. D17PC00096 and the Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via contract number HHSP233201600062C. The views, opinions, findings, and conclusions or recommendations contained herein are those of the authors and should not be interpreted as necessarily representing the official views policies or endorsements, either expressed or implied, of DARPA or DHS.