🖥 Amanda Askell: a Philosopher Who Teaches AI to be Good

Amanda Askell is an ethicist at the startup Anthropic, developer of the chatbot Claude. Askell is responsible for fine-tuning and AI alignment, making the models have good character traits.

⚪️ The Askell team created the first "AI with character"—Claude 3. Amanda tried to teach the model human-like traits such as honesty, curiosity, thoughtfulness, and open-mindedness. She also engineered Claude to tell people it doesn't have feelings, memory, or self-awareness.

⚪️ Askell studied philosophy at Oxford and the University of Dundee and received her PhD from New York University. Before joining Anthropic in 2021, she spent 2.5 years working on AI safety at OpenAI.

⚪️ In 2024, Amanda made Time magazine's 100 Most Influential People in AI list.

Amanda Askell on the possibilities of AI:

▶️ People sometimes think robots can do everything. Claude isn't this grand source of authority on everything, and you shouldn't believe its outputs blindly, Askell says.

▶️ AI didn't evolve. It might not have the equivalent of a nervous system, she explains in a conversation with Lex Fridman. At the same time, it has all of the language and intelligence components that we normally associate probably with consciousness,

▶️ Relationships with AI have to be handled with extreme care for many reasons. One is you probably don't want people performing long-term attachments to something that might change with the next iteration. 

▶️ The emergence of AGI is likely to be a continuous process. Perhaps it happens when we see the models coming up with novel solutions all the time, especially to easier problems. 

🐦 Askell actively posts in her X account and, as an ethicist, sometimes puts out philosophical challenges. For example:

Bob is convicted of killing Pete, and is sentenced to 30 years in prison for the crime. Pete actually faked his own death and has been living in Hawaii ever since. Bob serves his 30 years. As soon as he is released, Bob finds Pete and kills him. Should Bob go to prison for this?

What is your answer?

👍 — yes, he should
🎃 — no, he shouldn't
🤔 — this is a very ambiguous situation

#Claude #Anthropic @hiaimediaen