scroll to top
0

EBSCO Auth Banner

Let's find your institution. Click here.

Manipulating the Alpha Level Cannot Cure Significance Testing.

  • Academic Journal
  • Trafimow D; Department of Psychology, New Mexico State University, Las Cruces, NM, United States.
    Amrhein V; Zoological Institute, University of Basel, Basel, Switzerland.; Swiss Ornithological Institute, Sempach, Switzerland.
    Areshenkoff CN; Centre for Neuroscience Studies, Queens University, Kingston, ON, Canada.
    Barrera-Causil CJ; Faculty of Applied and Exact Sciences, Metropolitan Technological Institute, Medellín, Colombia.
    Beh EJ; School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia.
    Bilgiç YK; Department of Mathematics, State University of New York at Geneseo, Geneseo, NY, United States.
    Bono R; Quantitative Psychology Unit, Faculty of Psychology, University of Barcelona, Barcelona, Spain.; Institut de Neurociències, University of Barcelona, Barcelona, Spain.
    Bradley MT; Department of Psychology, Faculty of Arts, University of New Brunswick, Saint John, NB, Canada.
    Briggs WM; Independent Researcher, New York, NY, United States.
    Cepeda-Freyre HA; School of Psychology, Benemérita Universidad Autónoma de Puebla, Puebla, Mexico.
    Chaigneau SE; Center for Social and Cognitive Neuroscience, School of Psychology, Universidad Adolfo Ibáñez, Santiago, Chile.
    Ciocca DR; Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina.
    Correa JC; School of Statistics, Faculty of Sciences, National University of Colombia, Medellín, Colombia.
    Cousineau D; School of Psychology, University of Ottawa, Ottawa, ON, Canada.
    de Boer MR; Department of Health Sciences, Vrije Universiteit Amsterdam and Amsterdam Public Health Research Institute, Amsterdam, Netherlands.
    Dhar SS; Department of Mathematics and Statistics, Indian Institute of Technology, Kanpur, India.
    Dolgov I; Department of Psychology, New Mexico State University, Las Cruces, NM, United States.
    Gómez-Benito J; Quantitative Psychology Unit, Faculty of Psychology, University of Barcelona, Barcelona, Spain.; Institut de Neurociències, University of Barcelona, Barcelona, Spain.
    Grendar M; Biomedical Center Martin, Jessenius Faculty of Medicine, Comenius University, Martin, Slovakia.; Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, Slovakia.
    Grice JW; Department of Psychology, Oklahoma State University, Stillwater, OK, United States.
    Guerrero-Gimenez ME; Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina.
    Gutiérrez A; Faculty of Statistics, Saint Thomas University, Bogotá, Colombia.
    Huedo-Medina TB; Department of Allied Health Sciences, College of Health, Agriculture, and Natural Resources, University of Connecticut, Storrs, CT, United States.
    Jaffe K; Departamento de Biología de Organismos, Universidad Simón Bolívar, Caracas, Venezuela.
    Janyan A; Department of Cognitive Science and Psychology, New Bulgarian University, Sofia, Bulgaria.; National Research Tomsk State University, Tomsk, Russia.
    Karimnezhad A; Department of Biochemistry, Microbiology, and Immunology, University of Ottawa, Ottawa, ON, Canada.
    Korner-Nievergelt F; Swiss Ornithological Institute, Sempach, Switzerland.; Oikostat GmbH, Ettiswil, Switzerland.
    Kosugi K; School of Human Sciences, Senshu University, Kawasaki, Japan.
    Lachmair M; Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany.
    Ledesma RD; Consejo Nacional de Investigaciones Científicas y Técnicas, Mar del Plata, Argentina.; Facultad de Psicología, Universidad Nacional de Mar del Plata, Mar del Plata, Argentina.
    Limongi R; Pontificia Universidad Católica de Valparaíso, Valparaíso, Chile.; Vicerrectoría de Investigación y Desarrollo, Universidad Tecnológica de Chile INACAP, Santiago, Chile.
    Liuzza MT; Department of Medical and Surgical Sciences, 'Magna Graecia' University of Catanzaro, Catanzaro, Italy.
    Lombardo R; Economics Department, University of Campania 'Luigi Vanvitelli', Capua, Italy.
    Marks MJ; Department of Psychology, New Mexico State University, Las Cruces, NM, United States.
    Meinlschmidt G; Department of Psychosomatic Medicine, University Hospital Basel and University of Basel, Basel, Switzerland.; Division of Clinical Psychology and Cognitive Behavioral Therapy, International Psychoanalytic University, Berlin, Germany.; Division of Clinical Psychology and Epidemiology, Department of Psychology, University of Basel, Basel, Switzerland.
    Nalborczyk L; Université Grenoble Alpes, Centre National de la Recherche Scientifique, LPNC, Grenoble, France.; Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium.
    Nguyen HT; Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States.
    Ospina R; Computational Statistics Laboratory (CAST), Department of Statistics, Universidade Federal de Pernambuco, Recife, Brazil.
    Perezgonzalez JD; Business School, Massey University, Albany, New Zealand.
    Pfister R; Department of Psychology III, University of Würzburg, Würzburg, Germany.
    Rahona JJ; Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany.
    Rodríguez-Medina DA; School of Psychology, National Autonomous University of Mexico, Mexico City, Mexico.
    Romão X; CONSTRUCT-LESE, Faculty of Engineering, University of Porto, Porto, Portugal.
    Ruiz-Fernández S; Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany.; FOM Hochschule für Oekonomie und Management, Essen, Germany.; LEAD Graduate School & Research Network, University of Tübingen, Tübingen, Germany.
    Suarez I; Department of Psychology, Universidad del Norte, Barranquilla, Colombia.
    Tegethoff M; Division of Clinical Psychology and Psychiatry, Department of Psychology, University of Basel, Basel, Switzerland.
    Tejo M; Facultad de Ciencias Naturales y Exactas, Universidad de Playa Ancha, Valparaíso, Chile.
    van de Schoot R; Department of Methods and Statistics, Faculty of Social and Behavioural Sciences, Utrecht University, Utrecht, Netherlands.; North-West University, Optentia Research Focus Area, Vanderbijlpark, South Africa.
    Vankov II; Department of Cognitive Science and Psychology, New Bulgarian University, Sofia, Bulgaria.
    Velasco-Forero S; MINES Paristech, PSL Research University, Centre for Mathematical Morphology, Paris, France.
    Wang T; Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States.
    Yamada Y; Faculty of Arts and Science, Kyushu University, Fukuoka, Japan.
    Zoppino FCM; Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina.
    Marmolejo-Ramos F; School of Psychology, The University of Adelaide, Adelaide, SA, Australia.
  • Frontiers in psychology [Front Psychol] 2018 May 15; Vol. 9, pp. 699. Date of Electronic Publication: 2018 May 15 (Print Publication: 2018).
  • English
  • We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p -value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.
Additional Information
Publisher: Frontiers Research Foundation Country of Publication: Switzerland NLM ID: 101550902 Publication Model: eCollection Cited Medium: Print ISSN: 1664-1078 (Print) Linking ISSN: 16641078 NLM ISO Abbreviation: Front Psychol Subsets: PubMed not MEDLINE
Original Publication: Pully, Switzerland : Frontiers Research Foundation
Perspect Psychol Sci. 2016 Sep;11(5):664-691. (PMID: 27694463)
Stat Med. 1992 May;11(7):875-9. (PMID: 1604067)
Nat Rev Neurosci. 2013 May;14 (5):365-76. (PMID: 23571845)
J Pharm Pract. 2010 Aug;23(4):344-51. (PMID: 21507834)
Aust Crit Care. 2010 May;23(2):93-7. (PMID: 20347326)
Soc Cogn Affect Neurosci. 2009 Dec;4(4):423-8. (PMID: 20035017)
Nat Methods. 2015 Mar;12(3):179-85. (PMID: 25719825)
PeerJ. 2017 Jul 7;5:e3544. (PMID: 28698825)
R Soc Open Sci. 2016 Sep 21;3(9):160384. (PMID: 27703703)
Perspect Psychol Sci. 2012 Nov;7(6):615-31. (PMID: 26168121)
Q J Exp Psychol (Hove). 2014 May;67(5):1037-40. (PMID: 24528377)
Psychon Bull Rev. 2017 Jul 25;:null. (PMID: 28744765)
Am J Epidemiol. 2017 Sep 15;186(6):639-645. (PMID: 28938712)
Stat Med. 2002 Aug 30;21(16):2437-44; author reply 2445-7. (PMID: 12210627)
Psychol Rep. 2016 Oct;119(2):487-504. (PMID: 27502529)
Arch Dermatol. 2004 Dec;140(12):1520-3. (PMID: 15611433)
Psychol Bull. 1968 Sep;70(3):151-9. (PMID: 5681305)
Int J Sports Phys Ther. 2014 Oct;9(5):726-36. (PMID: 25328834)
Behav Res Ther. 2017 Nov;98 :19-38. (PMID: 28577757)
Eur J Epidemiol. 2016 Apr;31(4):337-50. (PMID: 27209009)
PLoS One. 2012;7(2):e32734. (PMID: 22389720)
Science. 2015 Aug 28;349(6251):aac4716. (PMID: 26315443)
Keywords: decision making; null hypothesis testing; p-value; significance testing; statistical significance
Date Created: 20180606 Latest Revision: 20201001
20211214
PMC5962803
10.3389/fpsyg.2018.00699
29867666
sponsored