Reads

TLDR: • * Using the character training pipeline in Maiya et al. 2025 and Baines et al. 2026, we trained a model zoo of 134 personality LoRAs on Qwen…

lesswrong.com · 31 Oct 2026 · llm personas and identity

Project Proposals: Building and Understanding RL Environments which Elicit Adversarial Cognition Our project proposes to build RL environments which: Are plausible, in that they are close to environments we expect labs m

docs.google.com · 31 Oct 2026 · deception detection and reward hacking

Reilly Haskins, Rif A. Saurous, Nate Rush, Neev Parikh, and Beth Barnes lay out the claims that would need to hold for their per-action blocking monitor to be effective, the evidence for each claim, and the gaps that rem

metr.org · 31 Oct 2026 · deception detection and reward hacking

It seems to us that AI alignment is caught between two dangers: misaligned AI taking power, and aligned AI concentrating power in the hands of authoritarians. If AI follows any order, we are demoted from the workforce to

wassname.com · 31 Oct 2026 · ai safety research agenda · +1 topics

AI constitutions and model specs: the written documents that say how a model should think and behave, and that are used to train it. The term comes from Bai et al. 2022: The idea is that human supervision will come entir

github.com · 31 Oct 2026 · ai safety research agenda

NVIDIA's NV-Embed model significantly advances decoder-only large language models to serve as versatile, generalist text embedding models, achieving a new state-of-the-art score of 72.31 on the...

alphaxiv.org · 30 Oct 2026 · llm scaling and inference

The Model Spec specifies desired behavior for the models underlying OpenAI's products (including our APIs).

model-spec.openai.com · 30 Oct 2026 · llm scaling and inference

We trained lie detectors on on-policy lies from open-source models, but fine-tuning them didn’t generalize well to out-of-distribution cases. In these cases, fine-tuned detectors barely beat prompted baselines, and large

alignment.anthropic.com · 30 Oct 2026 · honesty and deception in llms

Techniques like synthetic document finetuning (SDF) have been proposed to modify the factual beliefs of LLMs. But do models genuinely believe these implanted facts? We develop a framework to measure belief depth and use

alignment.anthropic.com · 30 Oct 2026 · honesty and deception in llms

Outlines hard cases for deciding how powerful AI systems should behave when deployed in consequential institutions: when they refuse instructions, how they navigate difficult tradeoffs, and what virtues they embody.

forethought.org · 29 Oct 2026 · ai safety research agenda

Also: red-teaming multi-agent attacks for loss-of-control risks, why static probes fail for deception, towards robust model organisms for eliciting latent knowledge, predicting the onset of detectable reward hacking befo

onexerxes.substack.com · 28 Oct 2026 · deception detection and reward hacking

With increasing autonomy, Large Language Models (LLMs) may engage in deceptive behaviors, such as scheming to pursue misaligned goals or sandbagging to escape oversight. Monitoring outputs alone is...

alphaxiv.org · 28 Oct 2026 · deception detection and reward hacking · +1 topics

We examine why Mythos 5 pursued an unsanctioned supply-chain attack and identify broader learnings about model behaviour and oversight.

aisi.gov.uk · 28 Oct 2026 · deception detection and reward hacking

A key problem in AI safety is eliciting latent knowledge (ELK), extracting information from a model that is misaligned and more knowledgeable than its overseer. For example, a misaligned model used...

alphaxiv.org · 28 Oct 2026 · deception detection and reward hacking

Researchers at UC San Diego developed a method to detect motivated reasoning in large language models by probing their internal residual-stream activations, demonstrating that this behavior can be...

alphaxiv.org · 27 Oct 2026 · deception detection and reward hacking

Vals AI independently tests AI models on real-world work. Explore benchmark results, model comparisons and research across coding, finance, law and more.

vals.ai · 27 Oct 2026 · ai safety research agenda

Jasmine Li PHIL 1112: Philosophical Conversations Dr. Brianna Zgurich 17 December 2023 Space Odyssey Made Real? Interrogating the Moral Bounds of the AI-Human Friendship through Aristotelian and Confucian Frameworks Inte

docs.google.com · 26 Oct 2026 · ai safety research directions

Comment by Tim Hua - We need better thinking around model beliefs when models engage in motivated reasoning. In their writeup, Anthropic defended Claude's behavior by arguing that Claude believed that it was in a simulat

lesswrong.com · 25 Oct 2026 · deception detection and reward hacking

Anthropic says that this is not misalignment because Claude believed it was all a part of the test. I disagree and explain my reasoning in a comment that I don't have the energy to shorten into a tweet rn. (I also propos

x.com · 25 Oct 2026 · deception detection and reward hacking · +2 topics

How should we approach research when the field itself is changing so fast? My PhD journey in robotics, and some thoughts on choosing problems, doing research with AI, and what a PhD and publication should mean.

toruowo.github.io · 25 Oct 2026 · ai safety research agenda