Dictionary

alignment tax en · NOUN

Etymology

First attested in a speech in 2019 by computer scientist Paul Christiano (see quotation), who attributed the idea to AI researcher and writer Eliezer Yudkowsky.

Meanings

1. (qualifier:artificial intelligence) A cost to the capabilities of an artificial intelligence resulting from the effects of aligning it with human ethics and morality.
   • I like this notion of an "alignment tax" […] the reason I might compromise is if there's some tension, between having the AI that's robustly trying to do what I want, and having the AI that is competent or intelligent, and the alignment tax is intended to capture that gap—that cost that I incur if I insist on alignment. 2019 August 29, Paul Christiano, Current work in AI alignment, EA Global San Francisco 2019:
   • We want an alignment procedure that avoids an alignment tax, because it incentivizes the use of models that are unaligned but more capable on these tasks. 2022 March 4, Ouyang, L. et. al., “Training language models to follow instructions with human feedback”, in arXiv, →DOI:
   • The fact that larger models are less subject to forgetting may be related to the fact that larger models do not incur significant alignment taxes. 2021 December 1, Askell, A. et. al., “A General Language Assistant as a Laboratory for Alignment”, in arXiv, →DOI:
   • We note that instead of an alignment tax our proposal entails a safety dividend – the more rational the system the more capable and the safer it will be. 2023 February 27, Kornai, A. et. al., “Safety without alignment”, in arXiv, →DOI:

Forms

wikipedia: Eliezer Yudkowsky