February 2022 Gwern.net newsletter with links on TODO
February 2022’s Gwern.net newsletter is now out; previous, January 2022 (archives). This is a collation of links and summary of major changes, overlapping with my Changelog; brought to you by my donors on Patreon.
Writings
-
Gwern.net: backlinks & similar-links now recursively pop up; experimental Ar5iv use; wrote GPT-3-using
paragraphizer.py
tool & used to add line-breaks to ~1k annotations; link icon system rewrite (faster, better+test-suite, >220 new link icons); live link popup system rewrite (similarly, >850 domains to popup);
Links
AI
-
“Berkeley Crossword Solver: Automated Crossword Solving”, et al 2022
-
Crypko.ai v1.0 (full-body anime generation now)
-
-
multi-task: “Gato: A Generalist Agent”, et al 2022 (small Decision Transformer-esque can learn >500 tasks without failing & while smoothly scaling—the scaling hypothesis finally begins to eat DRL & robotics?); “Multi-Game Decision Transformers”, et al 2022 (near-human offline single-checkpoint ALE agent with scaling & rapid transfer)
-
multi-agent: “Multi-Agent Reinforcement Learning is a Sequence Modeling Problem”, et al 2022; “Offline Pre-trained Multi-Agent Decision Transformer”, et al 2021 (just interleave agent choices)
-
exploration: “Online Decision Transformer”, et al 2022; et al 2022; “Towards Learning Universal Hyperparameter Optimizers with Transformers”
-
-
“SayCan: Do As I Can, Not As I Say: Grounding Language in Robotic Affordances”, et al 2022 (language models powering robots; see also Decision Transformer)/ “Inner Monologue: Embodied Reasoning through Planning with Language Models”, et al 2022 (extending SayCan- PaLM robotics with feedback); “LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action”, et al 2022
-
“A Roadmap for Big Model”, et al 2022 (Chinese multi-organization overview paper of scaling behavior & practices)
-
Anthropic raises $0.58b for DL scaling; now 40 people; Introducing Adept AI Labs (video demo; 9 ex-GB, DM, OAI researchers, $0.065b funding, ‘bespoke’ approach, training models to use existing common software); “We Are Conjecture, A New Alignment Research Startup” (Q&A; ex-EAI Connor Leahy, Sid Black & Gabriel Alfour leave Aleph Alpha to launch scaling startup)
-
“Chinchilla: Training Compute-Optimal Large Language Models”, et al 2022 (current LLMs are undertrained)
-
“Flamingo: a Visual Language Model for Few-Shot Learning”, et al 2022 (plugging Chinchilla into vision model for instant multimodal model)
-
-
“ULM2: Unifying Language Learning Paradigms”, et al 2022 (bi+unidirectional training of an open-sourced T5-20b model beating GPT-3)
-
“OPT: Open Pre-trained Transformer Language Models”, et al 2022 (FB is releasing reimplemented GPT-3 checkpoints publicly)
-
“Pathways: Asynchronous Distributed Dataflow for ML”, et al 2022 (training T5-136b on 2×1024 TPUv3-pods at 97% utilization); “Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads”, et al 2022
Genetics
Everything Is Heritable:
-
“Polygenic prediction of educational attainment within and between families from genome-wide association analyses in 3 million individuals”, et al 2022 (EA3; 12-16% EDU)
-
“Genetics, leadership position, and well-being: An investigation with a large-scale GWAS”, et al 2022 (another curious bipolar genetic correlation)
Engineering:
-
“Whole-genome risk prediction of common diseases in human preimplantation embryos”, et al 2022 ( media)
-
“An extra-uterine system to physiologically support the extreme premature lamb”, et al 2017
-
“Evolutionary Biology and Gene Editing of Cat Allergen, Fel d 1”, et al 2022 ( media); “The Next Weird Way We’re Changing Cats: What if you could make your cat hypoallergenic with biotechnology?” (more on cat allergies & anti-allergen antibodies)
-
“Selection for long and short sleep duration in Drosophila melanogaster reveals the complex genetic network underlying natural variation in sleep”, et al 2017 (no life expectancy difference between flies bred to 10-hour daily sleep differences)
Statistics/Meta-Science
-
“Inside the Virus-Hunting Nonprofit at the Center of the Lab-Leak Controversy” (moral mazes, self-regulation, normalization-of-deviance & gain-of-function research)
-
“A Systematic Review of Human Challenge Trials, Designs, and Safety”, Adams-et al 2022 (few or no fatalities in >187 challenge trials historically; we are being too conservative with challenge trials & killing too few volunteers)
-
“Book Review: Making Nature, 2015” (how did prestige journals like ‘CNS’ become prestigious?)
-
“Squares in Squares” (most optimal known packing for 1–89 squares; also ugly; see 2009)
Politics/Religion
-
“Large-scale cooperation in small-scale foraging societies”, 2022
-
“Contrasting Concepts of Harmony in Architecture”: Peter Eisenman vs Christopher Alexander (saying the quiet part out loud about why ‘Modern architecture’ is so bad); “Why are buildings today drab and simple, while buildings of the past were ornate and elaborately ornamented? The answer is not the cost of labor”
Psychology/Biology
-
“Safety, tolerability, pharmacokinetics, and pharmacodynamics of concomitant administration of multiple doses of cagrilintide with semaglutide 2.4 mg for weight management: a randomized, controlled, phase 1b trial”, et al 2021 (cagrilintide + semaglutide = 17% weight loss at 20 weeks, and still going!); “Lilly’s tirzepatide delivered up to 22.5% weight loss in adults with obesity or overweight in SURMOUNT-1” (et al 2022 )/“FDA approves Lilly’s Mounjaro™ (tirzepatide) injection”
-
“Weight regain and cardiometabolic effects after withdrawal of semaglutide: the STEP 1 trial extension”, et al 2022 (but appetite-suppressants require chronic treatment to keep all of the lost weight off)
-
“Unimolecular Polypharmacy for Treatment of Diabetes and Obesity”, et al 2022 (triple-peptide much more effective for weight loss than merely semaglutide+tirzepatide)
-
-
“Deep Phenotyping and Lifetime Trajectories Reveal Limited Effects of Longevity Regulators on the Aging Process in C57BL/6J Mice”, et al 2022 (fasting/mTOR/hormone interventions only change baseline mortality/health, not aging?)
-
“Effect of long-term caloric restriction on DNA methylation measures of biological aging in healthy adults”, et al 2023 (current epigenetic clocks do not consistently measure the benefits of CR)
Technology
Economics
-
“The Soviet Problem with Two ‘Unknowns’: How an American Architect and a Soviet Negotiator Jump-Started the Industrialization of Russia, Part I: Albert Kahn”/“Part 2: Saul Bron”, Melnikova-Raich 201014ya/201113ya (like MITI in Japan or the Communist Party in China, Soviet industrialization owed little to Stalin or Communism, and much to the Russian & American peoples)
-
“From Stroke to Stoke: The Multiple Sporting Legacies of the Southern California Home Swimming Pool”, 2021 (how repairing the Chicago World’s Fair led to skateboarding)
Fiction
Miscellaneous
-
1811–1812212ya New Madrid earthquakes (known unknowns)
-
Wheel-well stowaway (still happens)
Film/TV
Live-action:
-
Everything Everywhere All at Once (sekaikei Buddhist modal realism by way of Hong Kong live-action comic martial arts movies)
-
The Green Knight (2021)
I nearly burst out laughing when I saw a headline that the 1300s medieval romance Sir Gawain and the Green Knight (which I’d read in J. R. R. Tolkien’s translation) was getting a Hollywood movie adaptation. Hadn’t this trend of remakes, sequels, and adaptations of existing intellectual trend gone a little too far? Unlike Beowulf, whose 3-part structure & monster-fighting is naturally adapted, The Green Knight’s structure and ending hardly seems fit for a dramatic, glossy, save-that-cat Hollywood-style adaptation; how does one end it, with an explanation of the prank and belly-laughs all around? So when it came out, and early reviews were respectful (even allowing for the COVID-19 movie drought making reviewers desperate for some art-house material to review), I had to go see.
I forgot that, unlike the Met HD operas, there was no captioning, and made the mistake of trying to tough it out rather than miss part of the beginning to go get a captioning device from the movie theater; unfortunately, The Green Knight is also up-to-the-minute Hollywood in that every scene is so dark you squint & everyone mumbles in as heavily-accented English as possible. So I sort of understood it, but I was dissatisfied—particularly with the Green Knight’s wife’s speech, which was clearly pivotal but I couldn’t quite follow if it was a standard anti-humanist environmentalist rant saying the quiet part out loud or what. A plane ride gave me time to rewatch (most of) it with captions, and now I feel I understand what director David Lowery was getting at (and that Robin Hanson took away entirely too negative a take).
In TGK, sorceress Morgan le Fay’s failson not-yet-Sir Gawain has ‘failed to launch’ (as Lowery says of himself), spending his days drinking and whoring around a grubby medieval city, choked with dirt and smoke and animals and peasants. He has no stories to tell or deeds to boast of, despite being nephew to the storied King Arthur himself, and no self-esteem. Morgan, seeing this, writes a spell telling a story which conjures up the Green Knight, who, one Christmas Day, breaks in on the Round Table feasts, and to their fearful faces issues a challenge to a ‘game’.
None take it, except the just-shamed Gawain, who steps forward, leaps over the table, and cuts off the Knight’s head with its own axe—only for it to get up and ride away, ordering Gawain to come to it the next Christmas Day to receive the corresponding cut. Gawain becomes a marked man, famous throughout the land for the uncanny story he has abruptly become the protagonist of; drunkards at bars know, and even small peasant children watch his story told as a Punch-and-Judy-esque puppet play, which ends with the Green Knight cutting off his head. (Gawain does not find this comforting, despite being assured by King Arthur that “it is only a game”, and is convinced he is doomed.)
All too soon, it is time to leave; he is tempted by his whore to stay and live out an obscure life, but leaves. He is waylaid by bandits, and some nifty SFX explore the alternate story where he lacks the determination to cut himself loose, and dies tied up in the forest. Bereft of horse and most gear, he stumbles into the abandoned house of the martyr Saint Winifred, whose ghost asks him to retrieve her head from the pond. (Winifred is briefly alluded to in Sir Gawain and the Green Knight, but nothing like this incident happens.) She is appalled when the neophyte actor doesn’t know his lines and tentatively ventures a question about reward: “Why would you ask that? Why would you ever ask that?” He retrieves it successfully. (And she, in return, gifts him back the axe lost to the bandits.) Still lost and starving, Gawain continues on, battered by the elements, possibly high on suspicious mushrooms eaten out of desperation, and seeing strange phenomenon like processions of titanic giants (which the talking fox saves him from). Finally, he stumbles his way to a castle.
The lord (the Green Knight), lady, and an old woman (both apparently Morgan) offer him hospitality, as they too have heard all about the adventure of Gawain and the Green Knight, and their castle turns out to be near the Green Knight’s abode. They are not concerned about it, noting that “it is just a game” (and they should know), and offer to let him stay until Christmas Day, but warning him that they will be gone afterwards and he will have to go home. Meanwhile, the lord has Gawain play another game, by exchanging ‘winnings’ from whatever the lord hunts, and whatever Gawain does. The lady turns out to be a writer of stories herself, sometimes editing the stories she copies. She spends her time with Gawain, doing things like a crude photographic portrait of him. (Gawain finds looking at his portrait to be disconcerting.) She seduces him by holding out a magic protective charm (previously woven by Morgan), which Gawain doesn’t resist, mocking him afterwards as not a knight. In a discussion with the lord about honor, Gawain acknowledges that the game is indeed silly, but that he needs to do it. The lady asks why is the Green Knight green, of all colors? She notes that green is the color of nature, and it is the color that flesh like human bodies or books turn into when they rot, that castles are overgrown and overthrown by green things, that all begins and ends in green. (“This thing all things devours: / Birds, beasts, trees, flowers; / Gnaws iron; bites steel; / Grinds hard stones to meal; / Slays king, ruins town, / And beats high mountain down.”) Gawain is not sure what to make of that.
Gawain leaves for his destiny, with a final encounter with the lord, who tries to make another exchange in their game, only to be repelled by an angry and exhausted Gawain. The talking fox tries to talk him out of it, calling him a fool headed to doom for nothing; likewise repelled. Gawain arrives at the Green Knight’s chapel and another alternate history is explored: if Gawain flees the Knight, returns home, tells some lie about it, succeeds the aged King Arthur, takes his whore as mistress, makes a political marriage, as everything crumbles, leading to war, defeat, and King Gawain alone in his throne room as the enemy invade. He rejects this temptation too, and prepares to receive the blow. The Green Knight arises with the axe, jovially says “Now, off with your head”, and swings. Cut. The End.
This leaves a viewer with questions.
First, the director thinks it should be obvious whether Gawain dies; and it is: Gawain does not die, but gets a nick for flinching and resorting to magic charms, marring the symmetry of the game and undermining the bravery he displayed. He does not die in the original and nothing strongly undermines that. He is repeatedly told by the architects of the game that it is only a game—and it is a game his mother contrived for his benefit, to give him a story to tell. Finally, it does not make a good story for us if Gawain’s head rolls in the dirt and the maggots eat his body with none the wiser.
Second, what is the point of all this? Gawain went to a lot of trouble for all this. And what is the point for us viewers? What is the point of truncating the ending, if it’s so obvious that Gawain lives? Why invent an entire interlude with Winifred? Why the talking fox mocking him? Is the director misguidedly following the writing advice attributed to New Yorker editors that every short story should have the last page automatically removed, to make the viewer work? Maybe he’s just a jerk?
The real subject is Hollywood’s favorite subject: Hollywood, and the virtues of entertainment and telling stories. Does Gawain do anything useful for his world by playing this silly game with the Green Knight? Yes—Gawain creates culture. The lady enumerated many things which succumb to the green, but left out the most salient: not the books, nor the men, but what the books contained, about what the men did. That never dies, and even as every building from the 1300s crumble, the words of the poet (whose name is lost) live on; as another poet put it, “Cattle die, kinsmen die, / the self must also die; / but glory never dies, / for the man who is able to achieve it.” This is what makes a Knight of the Round Table: creating a story that never dies. Honor exists as a plot device for quests, constraints, vows, and beefs. Gawain becomes Sir Gawain by providing a tale of bandits, daring escapes, of ghosts and inexplicable phenomena, of a vanished castle, of the mysterious Green Knight: a story to be told at the Round Table and then throughout King Arthur’s kingdom at the fireplace of the most miserable peasant, who would never take the Knight’s challenge, but who will listen to the singer to warm their cold bones and distract the dirty child. This is more than most people ever do with their lives. There is nobility in that. (What do you expect him to do, singlehandedly launch an Industrial Revolution? I’m pretty sure that was not in any of the Arthurian poems. Only Mark Twain.)
And for us? The Green Knight serves a similar function as The Ring, in reconciling ourselves to our existence: we are all Gawain, because no one knows how their story ends, whether it is a comedy or tragedy. As Tolkien appreciated (“‘It’s a dangerous business, Frodo, going out of your door,’ he used to say. ‘You step into the Road, and if you don’t keep your feet, there is no telling where you might be swept off to.”), once you leave home and enter the wide world, you enter an abyss of ignorance. Countless stories shoot off in every direction, lives you could’ve lived, for better or for worse. If we shun the call and fail to launch a quest, we will never go anywhere or do anything. How easy it is to not rise to the challenge, to remain at home, to try to merely ’be good’.
Gawain was such, until into his tepid daily life erupted a mystery. Gawain can’t know how the story ends, because he is inside mystery: no matter who tells him “it is just a game”, he doesn’t know it is a game. We must bare our necks to the Green Knight, and wince as the axe whistles down and the screen cuts to black. Perhaps we will be a corpse in the woods, or perhaps we will become a famous knight.
Nature, and the world, are not kind (even if our mothers are); they simply are. Sooner or later, the green reclaims its own. A happy ending is never guaranteed. But how tragic to never try.