Scale coarseness is a methodological artifact that tends to attenuate effect sizes. First plot shows both variables as continuous and normal. The second plot shows coarseness (5-point scale) only on X, leaving Y continuous. Last plot shows coarseness on both variables.

Dec 12, 2023 · 11:08 PM UTC

how can i use these to fudge my research so that it catches attention and makes me famous?
Bin your data in different increments until you get a significant effect
Replying to @MatthewBJane
Can you share your code? I feel there is something wrote with your results.
set.seed(343) dat = MASS::mvrnorm(1000,mu=c(0,0), Sigma=cbind(c(1,.50),c(.50,1)), empirical=TRUE) scoreX <- dat[,1] coarse_scoreX <- as.numeric(cut(scoreX,breaks=5)) scoreY <- dat[,2] coarse_scoreY <- as.numeric(cut(scoreY,breaks=5))
Replying to @MatthewBJane
Interesting! I feel like this is an argument for using visual analog scales. Any recommendations when working with e.g. likert scales? Also, thanks for sharing codes in a comment — it appears coarseness does not influence false positive rates? That’s still that I guess..
I think likert items are fine (people tend to bunch up at certain values anyways), and you can administer more items quickly. The bias goes away quickly when multiple likert items are aggregated.
this is interesting but I'm a bit of a data-layperson, can anyone give an example of how this can be avoided? Like what's an example of a continuous variable turned into a coarse one?
A survey may have respondents fill out a multiple choice question like: How stressed have you been in the last week? 1. Not at all 2. Somewhat so 3. Moderately so 4. Very much so even though stress is a continuous variable (presumably), we measure it with 4 points
Replying to @MatthewBJane
we should make it illegal to transform continuous variables into ordinal categorical ones or, god forbid, binary variables
But often this is an issue with the measurement procedure itself so not always post-hoc. For example, using likert scales for something that is theoretically continuous