Opinions on Internal and External Validity

URL

Change Log

When What
December 14th, 2015 Donated by Janet Siegmund

Reference

Studies who have been using the data (in any form) are required to include the following reference:

@inproceedings{Siegmund:2015:VIE:2818754.2818759,
 author = {Siegmund, Janet and Siegmund, Norbert and Apel, Sven},
 title = {Views on Internal and External Validity in Empirical Software Engineering},
 booktitle = {Proceedings of the 37th International Conference on Software Engineering - Volume 1},
 series = {ICSE '15},
 year = {2015},
 isbn = {978-1-4799-1934-5},
 location = {Florence, Italy},
 pages = {9--19},
 numpages = {11},
 url = {https://dl.acm.org/citation.cfm?id=2818754.2818759},
 acmid = {2818759},
 publisher = {IEEE Press},
 address = {Piscataway, NJ, USA},
}

About the Data

Overview of Data

1) studies.csv : Literature survey of papers from ESEC/FSE, ICSE, and EMSE. Contains data on how they were validated.
2) resultsComplete.csv : Contains the responses of the program-committee members and our categorization of the responses.

Attribute Information

1) studies.csv:
Contains name of the paper, conference and response for the following 5 questions

  • Was an empirical method applied?
  • Were the experimental subjects human or non-human?
  • Were the human experimental subjects professionals or students?
  • Was an internal or external replication reported?
  • How are threats to validity described?

    2) resultsComplete.csv
    Contains the responses of the program-committee members and our categorization of the responses

Paper Abstract

Empirical methods have grown common in software engineering, but there is no consensus on how to apply them properly. Is practical relevance key? Do internally valid studies have any value? Should we replicate more to address the tradeoff between internal and external validity? We asked the community how empirical research should take place in software engineering, with a focus on the tradeoff between internal and external validity and replication, complemented with a literature review about the status of empirical research in software engineering. We found that the opinions differ considerably, and that there is no consensus in the community when to focus on internal or external validity and how to conduct and review replications.