Success Factors

It may help to ask what causes success in education and see how online education affects it. To a first approximation, ignoring environment, one earns educational success through:

1. IQ/g

   IQ obviously predicts a huge chunk of educational success (leading to the ironic accusation that IQ tests are only academic questions) since the smarter one is, the easier learning anything is, much less one’s schoolwork.

2. Conscientiousness (a personality trait in the Big Five; of the hard work, grit, effort)

   If one is not smart enough that one can simply inhale lessons and pass tests, one still has the option of working hard: doing extra practice problems, asking for help, etc. Success will not come easy, but it will still come. These 2 factors together will correlate somewhere like 0.7 with educational success: someone who is smart and hard-working will go to the top, and someone who is stupid and lazy will not.

3. Miscellaneous

   The rest of the correlation is made up of socioeconomic status, culture (eg. East Asian?) and random other things: random life events or hard-to-measure environmental factors like an extra-inspiring teacher, etc.

(One factor that does not matter much is quality of school buildings, textbooks, etc. These do not substantially predict grades or degrees except as proxies for SES or student body quality or similar things, which is why increasing funding typically makes no difference and natural experiments indicate extremely small returns. Indeed, it’s somewhat absurd to suggest that access is an issue in the Internet age where we are drowning in digitized textbooks & papers, used books, and one cannot give away a 5 year old computer; we are a long way indeed from the world of Ramanujan where the most intelligent & motivated autodidact could have serious trouble laying hands on important resources.)

Online Education’s Factors

How does online education affect them—reducing the need for that factor to reach a certain level of attainment, leaving it alone, or increasing the need for that factor?

1. IQ seems like it could go any way:

   • Any effects could roughly cancel out, perhaps in some sort of compensating mechanism where students only aim at particular levels of mastery or performance and better or worse methods only change how much time they need to invest before they go off to play video games.

   • It could increase the need for IQ, because now all the extraneous time-wasting ‘gunk’ like sharpening pencils or doing roll-call can be cleared away by the technical solutions, leaving more time for pure learning. By eliminating all the environmental hindrances and variation, the only variation left will come from the student’s innate intellectual abilities: IQ. Students will race through courses until they hit their natural limits; even Sal Khan’s videos can’t make a dim bulb calculate solutions to Schrodinger’s equation.

     It has been noted in the psychometric literature that successful attempts to eliminate socio-economic penalties and provide quality environments for all children would necessarily increase the apparent contribution of heredity: if every child is in an environment that lets them develop and flourish to their fullest extent, then any remaining differences in their development will be due to hereditary factors! If variations in IQ are the joint product of variations in heredity and environment, then eliminating all variation in environment, setting environment to 0, means the remaining variation will be just the variation in heredity.

   • It could reduce the need for IQ, since online education will lead to a marketplace of lessons where only the clearest, most insightful, easily understood lessons survive. In ordinary classrooms staffed by ordinary teachers, extemporaneous lectures or explanations are necessarily more opaque and lower-quality compared to a lecture that the world-class presenter has spent months or years honing.

     But it is an utopian thought that perhaps everyone will be successful at education; so the question becomes, what trait or environmental factor would then become the best predictor of attainment? If you reduce the need for brains, then perhaps you still need motivation and appetite for work, which in conjunction with the previous point about joint products leads us to the next observations…

2. Conscientiousness is the joker. There is one clear possible change: online education will increase demand for Conscientiousness compared to offline education.

   This has been suggested on more than one occasion & connected to discipline/time-management problems¹². This tallies with my personal experience with online courses and classes with online assignment components like computer science classes (where class attendance may be optional and programming projects or homework are submitted remotely). I had a good deal of trouble just sitting down to do the course or assignment, even though it was not necessarily that difficult. The distractions on my laptop beckoned: I would go use crufty old Solaris boxes in the computer labs just to avoid the distractions and get something done. Other experiences were more dramatic: one CS exam was done on computers, with a built-in test suite you could run to get your exact grade, so one could spend hours working on it until one had a perfect 100 (which wasn’t terribly hard), which of course I did—so I was shocked when the teacher showed us the grade distribution and it looked like a normal CS exam distribution, with plenty of <100 scores and outright failures!

3. Miscellaneous is too varied and heterogeneous to be predictable, so we won’t discuss it further.

It’s also worth noting that due to socioeconomic economic status being determined in large part due to IQ & Conscientiousness, and both being highly heritable, we might predict in advance that online education will not be a leveling force as far as other traits go; claims that online education or MOOCs will reduce income inequality seem to be based on a standard social science model worldview in which there are no cognitive differences between individuals since everyone is equally intelligent/patient/hard-working/curious and that any observed inequality of outcomes must be due to pre-existing differences of environment or discrimination, that access to educational material is the bottleneck, and that increasing access will perforce reduce inequality. None of which is true, and so we would not be too surprised to find that MOOCs are dominated by the already well-educated or high SES (Hansen & Reich2015), who are intelligent & motivated enough to seek out further education

Factor Changes

Now, we discarded #3 as being impossible to generalize about, and #2 suggests that Conscientiousness will increase in its correlation with success, while to me the more plausible outcome for #1 is that it will reduce the need. But to be conservative, let’s assume the need for IQ remains unchanged. This suggests the following argument:

1. Material presented in an online education format: requires the same amount of IQ to understand⁶

2. Material presented in an online education format: also requires more Conscientiousness than the same material presented in a classroom

3. there are no other factors; then

4. less of the general population will be able to learn it.

To belabor the obvious and dress it up in mathematical garb: for a particular static set/population Z, the number of Z members which satisfy the requirements IQ + C < IQ + (C+1), because the fraction of the population with both the necessary IQ and the necessary Conscientiousness must be equal to or smaller than the fraction with just the necessary IQ; for any properties (a ∧ b) ≤ P(a). See also the conjunction fallacy. When it comes to normally distributed traits like IQ, modest selection pressure can drive down the fraction of eligible people to near-zero rates; for example, far less than 1% of the population will be 2 standard deviations above the mean on both IQ and Conscientiousness and this holds true even if we assume that both traits are highly correlated with each other (they’re not), see the appendix for formulas & calculations.

(A major caveat here is that the premises really do need absolute values of IQ and Conscientiousness. If you only have correlations, I believe it is possible for IQ’s correlation for educational success remain the same and Conscientiousness’s correlation go up while the fraction of the general population succeeding goes up also. For example, if online education reduced the need for Conscientiousness, but reduced the need for IQ even more, more people will pass by the opposite of our conjunctive reasoning, but any attempt to predict success will benefit less from information about IQ than about Conscientiousness.)

Now, to discuss claim #2 in more detail. The first study cited previously on online education stressing Conscientiousness, Elvers 2003_21ya, is particularly interesting (see the quotes in the footnote). Now, given the evidence from this study that online education scores correlate with Conscientiousness, it seems very likely that #2 is true. However, the result that the online students had the same average as the offline students indicates that the conclusion #4 is not true; the obvious candidate to reject via modus tollens is assumption #1. As one would hope! But if #1 is not true, it could be true to a very large degree—as already mentioned, computerized education could make education a lot less correlated with your raw IQ because it’s presented better or whatever (to listen to the most rapturous users of Khan Academy). However, the equality in scores between the online and offline classes indicates that whatever the drop in IQ requirements, it was offset by the increase in Conscientiousness requirements.

What does this tradeoff between loading on Conscientiousness and IQ suggest?

1. First, it suggests that blended learning will be intermediate in results: I’d expect partial online education to be ‘weaker’ than full online education in loading on Conscientiousness.

   You have to force yourself to go to class, but then it’s still easier to learn without burdening your willpower/Conscientiousness. (You can always, say, not bring your laptop to class—difficult or impossible with online education!) I’d expect the effect of non-mandatory to be intermediate, much like I’d expect frequent mandatory deadlines in online education to help only a little.

2. Second, if one lone course shows such a hit from lack of Conscientiousness, what happens as ever more material goes online and students might be expected to do entire semesters just online?

   Will we see the correlation go up, as students expend all their willpower and run completely dry (see eg. Baumeister & Tierney 2010_14ya, Willpower)? (You may be able to lift 1 weight up to your head and do that 10 times in a row, but if given 10 weights simultaneously to lift, you’ll drop them all.) It seems that the tradeoff might extend well beyond a single course to all courses.

Consequences

Is loading outcome more on Conscientiousness a bad thing? I think it is, for a few reasons, some of which follow directly from the tradeoff and some of which are speculation about future consequences:

1. there is no particular reason to favor Conscientiousness as an additional reward for ‘good’ people. Whether we should favor it over IQ depends on the consequences such as what mental traits we need more of in our elites.

   Conscientiousness is not a ‘virtue’ in the sense that the (non-existent) homunculus in your brain is ‘good’ or ‘bad’ for choosing to be Conscientious or not, any more than it is morally laudable to be high IQ than low IQ. Despite folk psychology & moralizing, the Big Five personality traits are stable over lifetimes like IQ, are turning out to be influenced by heredity like IQ, and progress is being made on tracing the traits to the underlying neurological factors like IQ. You can no more ‘try hard to be able to try hard’ (how circular) than you can try hard to be more intelligent.

   Even if we find that Conscientiousness is not affected or Conscientiousness does not correlate with any problematic traits like psychopathy, that doesn’t exclude other personality traits: Varela et al 2012_12ya finds that online performance was also correlated with being low on “gregariousness”, a subfactor of Extraversion matching on “individuals who confine themselves from social settings” (rather than just being quiet and reserved in social settings)—is this a good, bad, or neutral thing, morally? Or practically?

2. As already observed, the school system already rewards Conscientious grinds, and oppresses creativity. Do we need to make the former even more true? Think of how this will penalize bright creative potential-future-great-scientists—but uninterested in forcing themselves to do mandated drudge-work—nerds. We have all heard stories of geniuses like Einstein or Darwin or Jung who despised lower or higher education, or did their best to ignore it while educating themselves—Simonton’s 1994_30ya Greatness: Who makes history and why estimates that this is not a few anecdotes but 60% of his sample. (Conscientiousness is necessary for scientific greatness, but not that much.)

   If we stand idly by and let Conscientiousness shifts happen, saying that it must be a good thing since it is happening, I believe we are guilty of status quo bias. A useful thought here is Bostrom’s reversal test: why do we think that the current demands for Conscientiousness are optimal? Or the double-reversal test: suppose some alien or technology suddenly intervened in our educational system and made it load even more heavily on IQ but someone came up with a simple way to place burden back on Conscientiousness—would we accept their solution? I suspect in both cases, we would be unable to produce any good answer to this important issue.

   It’s worth noting that a little appreciated property of the bell curve or normal distribution is that a very small shift in the average can have unintuitively large consequences at the tails: a shift of 1 IQ point in the general population can result in considerable changes in the population all the way out at, say, 160+ IQs. We can grasp this by looking at the changes in rarity by standard deviation: someone at 2 deviations is 1 in 22, 3 deviations 1 in 370, 4 = 1 in 15,787 (42× fewer than 3), and 5 = 1 in 1,744,277 (110× fewer than 4). A common standard deviation for particular IQ tests is 15 points, so 3 deviations out is ~145 IQ, which is around the observed minimum for great Nobel-winning scientific or mathematical work⁷. This is interesting because some interventions like iodization can have shockingly large effects on IQ in the worst-off environments—such as an average increase of as much as 15 IQ points—which would suggest that if a hypothetical intervention moved a population a standard deviation 85 → 100 IQ, its subpopulation at 145 goes from being 4 deviations away to 3—which increases that subpopulation’s ranks by 42× or 4200%!⁸ (One meta-analysis of iodine effects, Scrimshaw 1998_26ya, apparently did find a shift of the overall bell curve; and further, iodization benefits females more than males, so there may have been nontrivial consequences there…)

   The implications are obvious for any academic system that forced its membership’s average IQ down a few points in exchange for researchers higher on Conscientiousness: it may have outsized effect on how much of its membership are the very smartest researchers around.

3. The tradeoff resulting in online education favoring Conscientiousness was neither designed in nor realized by the designers; it is purely accidental and undesired. Wouldn’t it be extraordinary if an accidental tradeoff turned out to be exactly optimal? How very convenient!

   This is a good time to apply the status-quo reversal test: suppose online education did not result in any such tradeoff but a Khan Academy staffer unilaterally made some changes meant solely to make KA scores reflect Conscientiousness more (perhaps your progress would be deleted if you did not Conscientiously log in every week and do a few problems). Would you approve of this change? Suppose further online education actually reduced the need for Conscientiousness (maybe because the service pings your cellphone with a quick practice problem every so often); would you approve of the staffer’s change then? If you would not approve in the latter scenario where the shift along the tradeoff curve is intentional, why would you approve of a shift caused accidentally?

4. The cheapness of online education may prove irresistible and a case of worse is better: the cost of human teachers is nontrivial and may be increasing (whether this is due to backloaded pension compensation, growth of the education sector & diminishing returns, Baumol’s cost disease etc.), and this has prompted reactions like the death of university tenure & wholesale use of adjuncts, attacks on unions, and interest in automated methods of teaching… like online education. Already cuts have begun. Even if online education is worse, there may be no choice about whether to use it or not—a sort of educational enclosure movement. This shift may or may not be economically efficient (if the public sector is able to force the losses onto the public which is not organized enough to avoid it, perhaps due to ideological divisions).

5. Economic growth is increasingly captured in the US by the most-educated, with income growth going mostly to graduate degree holders. So anything which may lessen the ranks of the most highly educated seems like it would exacerbate the inequality of returns to education. Is some general increases in the net wealth of the economy worth it? People do not eat absolute wealth increases, they eat relative increases—more egalitarian economies are happier populaces. (Note the same question can be asked of other ‘cheaper’ things like globalization and outsourcing, and the answer in those other cases is not trivial. Pareto-efficient does not mean everyone is better off, just that no one is worse off, and this assumes humans do not care about their rankings or place—a patently false approximation.)

What other consequences may there be?

This prodigious event is still on its way, still wandering; it has not yet reached the ears of men. Lightning and thunder require time, the light of the stars requires time, deeds, though done, still require time to be seen and heard. This deed is still more distant from them than the most distant stars—and yet they have done it themselves.

Appendices

# install.packages("fMultivar")
library ("fMultivar")

x <- rnorm2d(10000000, rho=0.5)

xgreater <- length(subset(x, x[,1] > mean(x[,1])+2*sd(x[,1])))
xandygreater <- length(subset(x, x[,1] > mean(x[,1])+2*sd(x[,1]) & x[,2] > mean(x[,2])+2*sd(x[,2])))

c(xgreater, xandygreater); c(xgreater / length(x), xandygreater / length(x), xgreater / xandygreater) * 100

## example results for different values of 'rho='
# 0.1
# [1] 454,664 17,570
# [1] 2.273e+00 8.785e-02 2.588e+03
#
# 0.2
# [1] 458,284 82,552
# [1]   2.2914   0.4128 555.1458
#
# 0.5
# [1] 454,484 80,872
# [1]   2.2724   0.4044 561.9794
#
# 0.9
# [1] 455,242 267,912
# [1]   2.276   1.340 169.922
#
# 0.95
# [1] 455,162 321,024
# [1]   2.276   1.605 141.784
#
# 0.99
# [1] 455,260 394,448
# [1]   2.276   1.972 115.417

fmin <- function (y,mu1,mu2,sigma1,sigma2,rho)
     {t1<-dnorm(y,mean=mu1,sd=sigma1)
     tt<-rho*(y-mu1)/(sigma1*sqrt(1-rho*rho))
     tt<-tt-(y-mu2)/(sigma2*sqrt(1-rho*rho))
     t1<-t1*pnorm(tt)
     t2<-dnorm(y,mean=mu2,sd=sigma2)
     tt<-rho*(y-mu2)/(sigma2*sqrt(1-rho*rho))
     tt<-tt-(y-mu1)/(sigma1*sqrt(1-rho*rho))
     t2<-t2*pnorm(tt)
     return(t1+t2)}

fmin(c(1:200),100,100,15,15,0.1)
#  [1] 1.849e-11 2.864e-11 4.418e-11 6.784e-11 1.037e-10 1.578e-10 2.392e-10 3.608e-10 5.418e-10
#  ...

sum(fmin(c(1:200),100,100,15,15,0.1)[130:200])
[1] 0.001004

sum(dnorm(c(1:200), 100, 15)[130:200])
# [1] 0.02459

## the function blows NaN chunks on 1.0, so we'll try a lot of 9s:
sum(fmin(c(1:200),100,100,15,15,0.9999999999)[130:200])
# [1] 0.02459

llim <- 2
ulim <- Inf
rho <- 0.9

f <- function(x,y) {
    (1 / (2 * pi * sqrt(1 - rho^2))) * exp(-(1 / (2 * (1 - rho^2))) * (x^2 + y^2 - 2 * rho * x * y))
}

# double-integration
integrate(function(y) {
   sapply(y, function(y) {
     integrate(function(x) f(x,y), llim, ulim)$value
   })
 }, llim, ulim)

# 0.01336 with absolute error < 1.6e-05

from numpy import *;
from scipy.integrate import *;

from matplotlib.pyplot import *;

def func(x, y, rho):
        return 1.0/(2.0*pi*sqrt(1.0-rho**2.0)) * exp(-0.5/(1.0-rho**2.0) * (x**2.0 + y**2.0 - 2.0*rho*x*y))

def answer(rho):
        return dblquad(func, 2, 100, lambda x: 2, lambda x: 100, args=(rho,))[0]

x = arange(-0.99, 0.99, 0.1)
y = zeros(len(x))
for i in range(len(x)):
        y[i] = answer(x[i])
        print x[i], y[i]

plot(x, y)
show()

llim <- 3
ulim <- Inf
rho <- 0.6
# ...
# 0.0001397 with absolute error < 3.4e-05

Backlinks