- See Also
- Gwern
-
Links
- “RAG vs Fine-Tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture”, Balaguer et al 2024
- “Inside the Chaos at OpenAI: Sam Altman’s Weekend of Shock and Drama Began a Year Ago, With the Release of ChatGPT”, Hao & Warzel 2023
- “Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation”, Ding et al 2023
- “Does GPT-4 Pass the Turing Test?”, Jones & Bergen 2023
- “PAIR: Jailbreaking Black Box Large Language Models in 20 Queries”, Chao et al 2023
- “Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”, Qi et al 2023
- “Non-Determinism in GPT-4 Is Caused by Sparse MoE”, 152334H 2023
- “Large Language Models As Superpositions of Cultural Perspectives”, Kovač et al 2023
- “AI Is a Lot of Work: As the Technology Becomes Ubiquitous, a Vast Tasker Underclass Is Emerging—And Not Going Anywhere”, Dzieza 2023
- “I’m Afraid I Can’t Do That: Predicting Prompt Refusal in Black-Box Generative Language Models”, Reuter & Schulze 2023
- “Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4”, Chang et al 2023
- “GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models”, Eloundou et al 2023
- “Why Didn’t DeepMind Build GPT-3?”, Godwin 2023
- “OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’”, Konrad & Cai 2023
- “GPT-3 As Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities”, Bommarito et al 2023
- “Language Models Are Better Than Humans at Next-Token Prediction”, Shlegeris et al 2022
- “HALIE: Evaluating Human-Language Model Interaction”, Lee et al 2022
- “TruthfulQA: Measuring How Models Mimic Human Falsehoods”, Lin et al 2021
- “‘How GPT-3 Is Shaping Our AI Future’ With Sam Altman/Azeem Azhar (The Exponential View), Wednesday 7 October 2020”
- “Scaling Laws for Neural Language Models: Figure 15: Far beyond the Model Sizes We Study Empirically, We Find a Contradiction between Our Equations § Pg17”, Kaplan 2020 (page 17 org openai)
- “Towards Synthesizing Complex Programs from Input-Output Examples”, Chen et al 2017
- “Genetics of Caffeine Consumption and Responses to Caffeine”, Yang et al 2010
- “Why GPT-3 Matters”, Gao 2024
- “Greg Brockman: OpenAI and AGI”, Brockman 2024
- M74108556
- sharifshameem
- Miscellaneous
- Bibliography
See Also
Gwern
“The Scaling Hypothesis”, Gwern 2020
Links
“RAG vs Fine-Tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture”, Balaguer et al 2024
RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture
“Inside the Chaos at OpenAI: Sam Altman’s Weekend of Shock and Drama Began a Year Ago, With the Release of ChatGPT”, Hao & Warzel 2023
“Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation”, Ding et al 2023
Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation
“Does GPT-4 Pass the Turing Test?”, Jones & Bergen 2023
“PAIR: Jailbreaking Black Box Large Language Models in 20 Queries”, Chao et al 2023
PAIR: Jailbreaking Black Box Large Language Models in 20 Queries
“Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”, Qi et al 2023
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
“Non-Determinism in GPT-4 Is Caused by Sparse MoE”, 152334H 2023
“Large Language Models As Superpositions of Cultural Perspectives”, Kovač et al 2023
Large Language Models as Superpositions of Cultural Perspectives
“AI Is a Lot of Work: As the Technology Becomes Ubiquitous, a Vast Tasker Underclass Is Emerging—And Not Going Anywhere”, Dzieza 2023
“I’m Afraid I Can’t Do That: Predicting Prompt Refusal in Black-Box Generative Language Models”, Reuter & Schulze 2023
I’m Afraid I Can’t Do That: Predicting Prompt Refusal in Black-Box Generative Language Models
“Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4”, Chang et al 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4
“GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models”, Eloundou et al 2023
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
“Why Didn’t DeepMind Build GPT-3?”, Godwin 2023
“OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’”, Konrad & Cai 2023
OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’
“GPT-3 As Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities”, Bommarito et al 2023
GPT-3 as Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities
“Language Models Are Better Than Humans at Next-Token Prediction”, Shlegeris et al 2022
Language models are better than humans at next-token prediction
“HALIE: Evaluating Human-Language Model Interaction”, Lee et al 2022
“TruthfulQA: Measuring How Models Mimic Human Falsehoods”, Lin et al 2021
“‘How GPT-3 Is Shaping Our AI Future’ With Sam Altman/Azeem Azhar (The Exponential View), Wednesday 7 October 2020”
“Scaling Laws for Neural Language Models: Figure 15: Far beyond the Model Sizes We Study Empirically, We Find a Contradiction between Our Equations § Pg17”, Kaplan 2020 (page 17 org openai)
“Towards Synthesizing Complex Programs from Input-Output Examples”, Chen et al 2017
Towards Synthesizing Complex Programs from Input-Output Examples
“Genetics of Caffeine Consumption and Responses to Caffeine”, Yang et al 2010
“Why GPT-3 Matters”, Gao 2024
“Greg Brockman: OpenAI and AGI”, Brockman 2024
M74108556
sharifshameem
Miscellaneous
-
/doc/ai/nn/transformer/gpt/3/2019-11-07-amodei-aiandcompute-twodistincteras-gpt3modified.jpg: -
https://andrewmayne.com/2023/11/14/is-the-reversal-curse-real/:View External Link:
https://andrewmayne.com/2023/11/14/is-the-reversal-curse-real/ -
https://barryzhang.substack.com/p/our-humble-attempt-at-fine-tuning -
https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates -
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4594466:View External Link:
-
https://www.cerebras.net/blog/introducing-gigagpt-gpt-3-sized-models-in-565-lines-of-code: -
https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse#pfHTedu4GKaWoxD5K -
https://www.reddit.com/r/mlscaling/comments/146rgq2/chatgpt_is_running_quantized/:
Bibliography
-
https://arxiv.org/abs/2401.08406#microsoft: “RAG vs Fine-Tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture”, -
https://www.theatlantic.com/technology/archive/2023/11/sam-altman-open-ai-chatgpt-chaos/676050/: “Inside the Chaos at OpenAI: Sam Altman’s Weekend of Shock and Drama Began a Year Ago, With the Release of ChatGPT”, -
https://arxiv.org/abs/2310.08419: “PAIR: Jailbreaking Black Box Large Language Models in 20 Queries”, -
https://arxiv.org/abs/2310.03693: “Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”, -
https://152334h.github.io/blog/non-determinism-in-gpt-4/: “Non-Determinism in GPT-4 Is Caused by Sparse MoE”, -
https://arxiv.org/abs/2307.07870: “Large Language Models As Superpositions of Cultural Perspectives”, -
https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots: “AI Is a Lot of Work: As the Technology Becomes Ubiquitous, a Vast Tasker Underclass Is Emerging—And Not Going Anywhere”, -
https://arxiv.org/abs/2306.03423: “I’m Afraid I Can’t Do That: Predicting Prompt Refusal in Black-Box Generative Language Models”, -
https://arxiv.org/abs/2303.10130: “GPTs Are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models”, -
https://www.forbes.com/sites/alexkonrad/2023/02/03/exclusive-openai-sam-altman-chatgpt-agi-google-search/: “OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’”, -
https://arxiv.org/abs/2301.04408: “GPT-3 As Knowledge Worker: A Zero-Shot Evaluation of AI CPA Capabilities”, -
https://arxiv.org/abs/2109.07958: “TruthfulQA: Measuring How Models Mimic Human Falsehoods”,