University of Hertfordshire Research Archive

        JavaScript is disabled for your browser. Some features of this site may not work without it.

        Browse

        All of UHRABy Issue DateAuthorsTitlesThis CollectionBy Issue DateAuthorsTitles

        Arkivum Files

        My Downloads
        View Item 
        • UHRA Home
        • University of Hertfordshire
        • Research publications
        • View Item
        • UHRA Home
        • University of Hertfordshire
        • Research publications
        • View Item

        Quality science from quality measurement: The role of measurement type with respect to replication and effect size magnitude in psychological research

        View/Open
        Published_Version.pdf (PDF, 846Kb)
        Author
        Kornbrot, Diana
        Wiseman, Richard
        Georgiou, George
        Attention
        2299/19826
        Abstract
        The quality of psychological studies is currently a major concern. The Many Labs Project (MLP) and the Open-Science-Collaboration (OSC) have collected key data on replicability and statistical effect sizes. We build on this work by investigating the role played by three measurement types: ratings, proportions and unbounded (measures without conceptual upper limits, e.g. time). Both replicability and effect sizes are dependent on the amount of variability due to extraneous factors. We predicted that the role of such extraneous factors might depend on measurement type, and would be greatest for ratings, intermediate for proportions and least for unbounded. Our results support this conjecture. OSC replication rates for unbounded, 43% and proportion 40% combined are reliably higher than those for ratings at 20% (effect size, w = .20). MLP replication rates for the original studies are: pro- portion = .74, ratings = .40 (effect size w = .33). Original effect sizes (Cohen’s d) are highest for: unbounded OSC cognitive = 1.45, OSC social = .90); next for proportions (OSC cogni- tive = 1.01, OSC social = .84, MLP = .82); and lowest for ratings (OSC social = .64, MLP = .31). These findings are of key importance to scientific methodology and design, even if the reasons for their occurrence are still at the level of conjecture.
        Publication date
        2018-02-12
        Published in
        PLoS ONE
        Published version
        https://doi.org/10.1371/journal.pone.0192808
        Other links
        http://hdl.handle.net/2299/19826
        Metadata
        Show full item record

        Related items

        Showing items related by title, author, creator and subject.

        • The Experience of Being a Trainee Clinical Psychologist From a Black and Minority Ethnic Group: A Qualitative Study 

          Shah, Snehal (2010-12-22)
          Aim: The existing evidence-base indicates that the experience of being a Black and Minority Ethnic (BME) trainee clinical psychologist is under-researched. The aim of the current study was to capture the broader training ...
        • The Mother and The Child Clinical Psychologist: a Discursive Analysis of Professional Conversations 

          Puchalska, Sylvia (2012-02-01)
          Aim Motherhood is placed under a lens by society: mothers are expected to fit within narrowly defined characteristics which dictate who mothers should be and how they should act. Although there are numbers of articles ...
        • Should Participants be Left to their Own Devices? Comparing Paper and Smartphone Diaries in Psychological Research 

          Laughland, Andrew; Kvavilashvili, Lia (2018-12-01)
          Growing smartphone ownership creates unprecedented opportunities for using participants’ own smartphones as diaries to record transient phenomena in daily life. In three studies, we assessed the hypothesis that participant-owned ...
        Keep in touch

        © 2019 University of Hertfordshire

        I want to...

        • Apply for a course
        • Download a Prospectus
        • Find a job at the University
        • Make a complaint
        • Contact the Press Office

        Go to...

        • Accommodation booking
        • Your student record
        • Bayfordbury
        • KASPAR
        • UH Arts

        The small print

        • Terms of use
        • Privacy and cookies
        • Criminal Finances Act 2017
        • Modern Slavery Act 2015
        • Sitemap

        Find/Contact us

        • T: +44 (0)1707 284000
        • E: ask@herts.ac.uk
        • Where to find us
        • Parking
        • hr
        • qaa
        • stonewall
        • AMBA
        • ECU Race Charter
        • disability confident
        • AthenaSwan