corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a0878e-c03c-708e-9d5a-cbbba8648d8c

Can we safely automate alignment research?

Home - Joe Carlsmith · published 30 April 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at joecarlsmith.com Wayback capture
3blogs citing
First cited
24 Sept 2025
Most recent
13 Jul 2026
Rank this month
not ranked
Links from this post
59
Words captured
17,177

Cited by

Blog
In the post
Date
EleutherAI
A Dynamical Model of AI Governability Read at the source ↗
13 Jul 2026
benjamintodd.substack.com
What needs to be done about AI risk Read at the source ↗
7 Jul 2026
jacquesthibodeau.com
Automated alignment research needs a better plan than 'Stop if we catch them scheming' Read at the source ↗
24 Sept 2025

Links from this post

Link
Host
https://joecarlsmith.substack.com/p/how-do-we-solve-the-alignment-problem
joecarlsmith.substack.com
Video and transcript of talk on automating alignment research Read at the source ↗
joecarlsmith.com
https://docs.google.com/presentation/d/1ow3mrRAgje8WdzXTxhJMT1SDEItYeWWEO9ZgCD9LmQ4/edit
docs.google.com
https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/17069901-can-we-safely-automate-alignment-research
joecarlsmithaudio.buzzsprout.com
https://joecarlsmith.substack.com/p/ai-for-ai-safety
joecarlsmith.substack.com
https://arxiv.org/abs/2311.08379
arxiv.org
A minimal viable product for alignment Read at the source ↗
aligned.substack.com
https://joecarlsmith.substack.com/i/158809605/waystations-on-the-path
joecarlsmith.substack.com
https://joecarlsmith.substack.com/i/158809605/sources-of-labor
joecarlsmith.substack.com
https://arxiv.org/abs/1805.00899
arxiv.org
Weak-to-strong generalization Read at the source ↗
openai.com
https://arxiv.org/pdf/2311.08379
arxiv.org
Detecting misbehavior in frontier reasoning models Read at the source ↗
openai.com
https://www.lesswrong.com/posts/zi6SsECs5CCEyhAop/gpt-4o-is-an-absurd-sycophant
lesswrong.com
Crisp and fuzzy tasks Read at the source ↗
aligned.substack.com
https://en.wikipedia.org/wiki/Cross-entropy
en.wikipedia.org
https://arxiv.org/abs/2204.05862
arxiv.org
https://www.anthropic.com/research/alignment-faking
anthropic.com
https://arxiv.org/abs/2310.13548
arxiv.org
https://www.anthropic.com/research/reasoning-models-dont-say-think
anthropic.com
https://www.openphilanthropy.org/research/how-much-computational-power-does-it-take-to-match-the-human-brain/
openphilanthropy.org
https://alignment.anthropic.com/2025/recommended-directions/
alignment.anthropic.com
https://www.openphilanthropy.org/request-for-proposals-technical-ai-safety-research/
openphilanthropy.org
https://arxiv.org/abs/2412.14093
arxiv.org
https://arxiv.org/abs/2502.17424
arxiv.org
On the Biology of a Large Language Model Read at the source ↗
transformer-circuits.pub
https://arxiv.org/pdf/2503.11926
arxiv.org
https://arxiv.org/abs/2312.09390
arxiv.org
https://arxiv.org/pdf/2501.13011
arxiv.org
https://assets.anthropic.com/m/317564659027fb33/original/Auditing-Language-Models-for-Hidden-Objectives.pdf
assets.anthropic.com
https://arxiv.org/pdf/1810.08575
arxiv.org
https://arxiv.org/abs/1811.07871
arxiv.org
https://arxiv.org/pdf/2208.08345
arxiv.org
https://arxiv.org/pdf/2109.10996
arxiv.org
https://arxiv.org/pdf/2409.03077
arxiv.org
https://arxiv.org/abs/1609.03543
arxiv.org
https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834
amazon.com
https://arxiv.org/abs/1906.01820
arxiv.org
https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit?tab=t.0
docs.google.com
https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/evaluating-potential-cybersecurity-threats-of-advanced-ai/An_Approach_to_Technical_AGI_Safety_Apr_2025.pdf
storage.googleapis.com
https://arxiv.org/abs/2304.03442
arxiv.org
https://arxiv.org/pdf/2211.03540
arxiv.org
https://arxiv.org/pdf/2311.08702
arxiv.org
https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e2a8514-75e5-4f68-8216-99547d267d91_682x285.png
substackcdn.com
https://arxiv.org/pdf/2310.01405
arxiv.org
Interpretability Dreams Read at the source ↗
transformer-circuits.pub
https://arxiv.org/abs/2307.13702
arxiv.org
https://arxiv.org/abs/2503.11926
arxiv.org
https://www.lesswrong.com/posts/5hApNw5f7uG8RXxGS/the-open-agency-model
lesswrong.com
https://www.lesswrong.com/posts/h7QETH7GMk9HcMnHH/the-no-sandbagging-on-checkable-tasks-hypothesis
lesswrong.com
https://www.lesswrong.com/posts/abmzgwfJA9acBoFEX/notes-on-countermeasures-for-exploration-hacking-aka
lesswrong.com
https://joecarlsmith.substack.com/i/157445591/agency-prerequisites
joecarlsmith.substack.com
https://joecarlsmith.substack.com/i/157445591/motivation-prerequisites
joecarlsmith.substack.com
Supervise Process, not Outcomes Read at the source ↗
ought.org
https://www.lesswrong.com/posts/CKkPLoiwZ9LsBmzAb/how-useful-for-alignment-relevant-work-are-ais-with-short
lesswrong.com
https://www.lesswrong.com/posts/jpGHShgevmmTqXHy5/decomposing-agency-capabilities-without-desires
lesswrong.com
https://joecarlsmith.substack.com/i/159044941/the-ai-for-ai-safety-sweet-spot
joecarlsmith.substack.com
https://www.lesswrong.com/posts/W3KfxjbqBAnifBQoi/we-should-try-to-automate-ai-safety-work-asap
lesswrong.com
https://www.lesswrong.com/posts/CXYf7kGBecZMajrXC/validating-against-a-misalignment-detector-is-very-different
lesswrong.com