ChatAnthropic
This will help you getting started with ChatAnthropic chat models. For detailed documentation of all ChatAnthropic features and configurations head to the API reference.
Overview
Integration details
Class | Package | Local | Serializable | PY support | Package downloads | Package latest |
---|---|---|---|---|---|---|
ChatAnthropic | @langchain/anthropic | ❌ | ✅ | ✅ |
Model features
Tool calling | Structured output | JSON mode | Image input | Audio input | Video input | Token-level streaming | Token usage | Logprobs |
---|---|---|---|---|---|---|---|---|
✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
Setup
You’ll need to sign up and obtain an Anthropic API
key, and install the @langchain/anthropic
integration package.
Credentials
Head to Anthropic’s website to sign up to
Anthropic and generate an API key. Once you’ve done this set the
ANTHROPIC_API_KEY
environment variable:
export ANTHROPIC_API_KEY="your-api-key"
If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:
# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"
Installation
The LangChain ChatAnthropic integration lives in the
@langchain/anthropic
package:
- npm
- yarn
- pnpm
npm i @langchain/anthropic
yarn add @langchain/anthropic
pnpm add @langchain/anthropic
Instantiation
Now we can instantiate our model object and generate chat completions:
import { ChatAnthropic } from "@langchain/anthropic";
const llm = new ChatAnthropic({
model: "claude-3-haiku-20240307",
temperature: 0,
maxTokens: undefined,
maxRetries: 2,
// other params...
});
Invocation
const aiMsg = await llm.invoke([
[
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
],
["human", "I love programming."],
]);
aiMsg;
AIMessage {
"id": "msg_01M9yt3aSqKJKM1RnZF4f44Q",
"content": "Voici la traduction en français :\n\nJ'adore la programmation.",
"additional_kwargs": {
"id": "msg_01M9yt3aSqKJKM1RnZF4f44Q",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 29,
"output_tokens": 20
}
},
"response_metadata": {
"id": "msg_01M9yt3aSqKJKM1RnZF4f44Q",
"model": "claude-3-haiku-20240307",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 29,
"output_tokens": 20
},
"type": "message",
"role": "assistant"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 29,
"output_tokens": 20,
"total_tokens": 49
}
}
console.log(aiMsg.content);
Voici la traduction en français :
J'adore la programmation.
Chaining
We can chain our model with a prompt template like so:
import { ChatPromptTemplate } from "@langchain/core/prompts";
const prompt = ChatPromptTemplate.fromMessages([
[
"system",
"You are a helpful assistant that translates {input_language} to {output_language}.",
],
["human", "{input}"],
]);
const chain = prompt.pipe(llm);
await chain.invoke({
input_language: "English",
output_language: "German",
input: "I love programming.",
});
AIMessage {
"id": "msg_012gUKUG65teaois31W3bfGF",
"content": "Ich liebe das Programmieren.",
"additional_kwargs": {
"id": "msg_012gUKUG65teaois31W3bfGF",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 23,
"output_tokens": 11
}
},
"response_metadata": {
"id": "msg_012gUKUG65teaois31W3bfGF",
"model": "claude-3-haiku-20240307",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 23,
"output_tokens": 11
},
"type": "message",
"role": "assistant"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 23,
"output_tokens": 11,
"total_tokens": 34
}
}
Multimodal inputs
Claude-3 models support image multimodal inputs. The passed input must
be a base64 encoded image with the filetype as a prefix
(e.g. data:image/png;base64,{YOUR_BASE64_ENCODED_DATA}
). Here’s an
example:
import fs from "fs/promises";
import { ChatAnthropic } from "@langchain/anthropic";
import { HumanMessage } from "@langchain/core/messages";
const imageData2 = await fs.readFile("../../../../../examples/hotdog.jpg");
const llm2 = new ChatAnthropic({
model: "claude-3-sonnet-20240229",
});
const message2 = new HumanMessage({
content: [
{
type: "text",
text: "What's in this image?",
},
{
type: "image_url",
image_url: {
url: `data:image/jpeg;base64,${imageData2.toString("base64")}`,
},
},
],
});
await llm2.invoke([message2]);
AIMessage {
"id": "msg_01AuGpm6xbacTwoUFdNiCnzu",
"content": "The image shows a hot dog. It consists of a cylindrical bread roll or bun that has been sliced lengthwise, revealing the bright red hot dog sausage filling inside. The hot dog sausage appears to be made from seasoned and smoked meat. This classic fast food item is a popular snack or meal, commonly enjoyed at sporting events, cookouts, and casual eateries.",
"additional_kwargs": {
"id": "msg_01AuGpm6xbacTwoUFdNiCnzu",
"type": "message",
"role": "assistant",
"model": "claude-3-sonnet-20240229",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 276,
"output_tokens": 88
}
},
"response_metadata": {
"id": "msg_01AuGpm6xbacTwoUFdNiCnzu",
"model": "claude-3-sonnet-20240229",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 276,
"output_tokens": 88
},
"type": "message",
"role": "assistant"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 276,
"output_tokens": 88,
"total_tokens": 364
}
}
See the official docs for a complete list of supported file types.
Agents
Anthropic models that support tool calling can be used in the Tool Calling agent. Here’s an example:
import { z } from "zod";
import { ChatAnthropic } from "@langchain/anthropic";
import { tool } from "@langchain/core/tools";
import { AgentExecutor, createToolCallingAgent } from "langchain/agents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
const llm3 = new ChatAnthropic({
model: "claude-3-sonnet-20240229",
temperature: 0,
});
// Prompt template must have "input" and "agent_scratchpad input variables"
const prompt3 = ChatPromptTemplate.fromMessages([
["system", "You are a helpful assistant"],
["placeholder", "{chat_history}"],
["human", "{input}"],
["placeholder", "{agent_scratchpad}"],
]);
const currentWeatherTool3 = tool(async () => "28 °C", {
name: "get_current_weather",
description: "Get the current weather in a given location",
schema: z.object({
location: z.string().describe("The city and state, e.g. San Francisco, CA"),
}),
});
const agent3 = createToolCallingAgent({
llm: llm3,
tools: [currentWeatherTool3],
prompt: prompt3,
});
const agentExecutor3 = new AgentExecutor({
agent: agent3,
tools: [currentWeatherTool3],
});
const input3 = "What's the weather like in SF?";
const result3 = await agentExecutor3.invoke({ input: input3 });
console.log(result3.output);
[
{
index: 0,
type: 'text',
text: '\n\nThe current weather in San Francisco, CA is 28°C.'
}
]
Custom headers
You can pass custom headers in your requests like this:
import { ChatAnthropic } from "@langchain/anthropic";
const llm4 = new ChatAnthropic({
model: "claude-3-sonnet-20240229",
maxTokens: 1024,
clientOptions: {
defaultHeaders: {
"X-Api-Key": process.env.ANTHROPIC_API_KEY,
},
},
});
const res4 = await llm4.invoke("Why is the sky blue?");
console.log(res4);
AIMessage {
"id": "msg_013Ft3kN62gNtiMWRqg6xxt8",
"content": "The sky appears blue due to a phenomenon called Rayleigh scattering. Here's a brief explanation:\n\n1) Sunlight is made up of different wavelengths of light, including the visible spectrum that we see as colors.\n\n2) As sunlight passes through the Earth's atmosphere, the different wavelengths of light interact with the gas molecules in the air.\n\n3) The shorter wavelengths of light, such as the blue and violet colors, get scattered more easily by the tiny gas molecules. This is because the wavelengths are similar in size to the molecules.\n\n4) The longer wavelengths of light, such as red and orange, get scattered much less by the gas molecules and travel more directly through the atmosphere.\n\n5) The blue wavelengths that are scattered in different directions become scattered across the entire sky, making the sky appear blue to our eyes.\n\n6) During sunrise and sunset, the sun's rays travel through more atmosphere before reaching our eyes, causing the blue light to get scattered away and allowing more of the red/orange wavelengths to pass through, giving those colors in the sky.\n\nSo in essence, the abundant scattering of blue light by the gas molecules in the atmosphere is what causes the sky to appear blue during the daytime.",
"additional_kwargs": {
"id": "msg_013Ft3kN62gNtiMWRqg6xxt8",
"type": "message",
"role": "assistant",
"model": "claude-3-sonnet-20240229",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 13,
"output_tokens": 272
}
},
"response_metadata": {
"id": "msg_013Ft3kN62gNtiMWRqg6xxt8",
"model": "claude-3-sonnet-20240229",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 13,
"output_tokens": 272
},
"type": "message",
"role": "assistant"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 13,
"output_tokens": 272,
"total_tokens": 285
}
}
Tools
The Anthropic API supports tool calling, along with multi-tool calling. The following examples demonstrate how to call tools:
Single Tool
import { ChatAnthropic } from "@langchain/anthropic";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const calculatorSchema5 = z.object({
operation: z
.enum(["add", "subtract", "multiply", "divide"])
.describe("The type of operation to execute."),
number1: z.number().describe("The first number to operate on."),
number2: z.number().describe("The second number to operate on."),
});
const tool5 = {
name: "calculator",
description: "A simple calculator tool",
input_schema: zodToJsonSchema(calculatorSchema5),
};
const llm5 = new ChatAnthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
model: "claude-3-haiku-20240307",
}).bind({
tools: [tool5],
});
const prompt5 = ChatPromptTemplate.fromMessages([
[
"system",
"You are a helpful assistant who always needs to use a calculator.",
],
["human", "{input}"],
]);
// Chain your prompt and model together
const chain5 = prompt5.pipe(llm5);
const response5 = await chain5.invoke({
input: "What is 2 + 2?",
});
console.log(response5);
AIMessage {
"id": "msg_01XPUHrR4sNCqPr1i9zcsAsg",
"content": [
{
"type": "text",
"text": "Okay, let me use the calculator tool to find the answer:"
},
{
"type": "tool_use",
"id": "toolu_01MhUVuUedc1drBKLarhedFZ",
"name": "calculator",
"input": {
"number1": 2,
"number2": 2,
"operation": "add"
}
}
],
"additional_kwargs": {
"id": "msg_01XPUHrR4sNCqPr1i9zcsAsg",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 449,
"output_tokens": 101
}
},
"response_metadata": {
"id": "msg_01XPUHrR4sNCqPr1i9zcsAsg",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 449,
"output_tokens": 101
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "calculator",
"args": {
"number1": 2,
"number2": 2,
"operation": "add"
},
"id": "toolu_01MhUVuUedc1drBKLarhedFZ",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 449,
"output_tokens": 101,
"total_tokens": 550
}
}
Forced tool calling
In this example we’ll provide the model with two tools:
calculator
get_weather
Then, when we call bindTools
, we’ll force the model to use the
get_weather
tool by passing the tool_choice
arg like this:
.bindTools({
tools,
tool_choice: {
type: "tool",
name: "get_weather",
}
});
Finally, we’ll invoke the model, but instead of asking about the
weather, we’ll ask it to do some math. Since we explicitly forced the
model to use the get_weather
tool, it will ignore the input and return
the weather information (in this case it returned <UNKNOWN>
, which is
expected.)
import { ChatAnthropic } from "@langchain/anthropic";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const calculatorSchema6 = z.object({
operation: z
.enum(["add", "subtract", "multiply", "divide"])
.describe("The type of operation to execute."),
number1: z.number().describe("The first number to operate on."),
number2: z.number().describe("The second number to operate on."),
});
const weatherSchema6 = z.object({
city: z.string().describe("The city to get the weather from"),
state: z.string().optional().describe("The state to get the weather from"),
});
const tools6 = [
{
name: "calculator",
description: "A simple calculator tool",
input_schema: zodToJsonSchema(calculatorSchema6),
},
{
name: "get_weather",
description:
"Get the weather of a specific location and return the temperature in Celsius.",
input_schema: zodToJsonSchema(weatherSchema6),
},
];
const llm6 = new ChatAnthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
model: "claude-3-haiku-20240307",
}).bind({
tools: tools6,
tool_choice: {
type: "tool",
name: "get_weather",
},
});
const prompt6 = ChatPromptTemplate.fromMessages([
[
"system",
"You are a helpful assistant who always needs to use a calculator.",
],
["human", "{input}"],
]);
// Chain your prompt and model together
const chain6 = prompt6.pipe(llm6);
const response6 = await chain6.invoke({
input: "What is the sum of 2725 and 273639",
});
console.log(response6);
AIMessage {
"id": "msg_018G4mEZu8KNKtaQxZQ3o8YB",
"content": [
{
"type": "tool_use",
"id": "toolu_01DS9RwsFKdhHNYmhwPJHdHa",
"name": "get_weather",
"input": {
"city": "<UNKNOWN>",
"state": "<UNKNOWN>"
}
}
],
"additional_kwargs": {
"id": "msg_018G4mEZu8KNKtaQxZQ3o8YB",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 672,
"output_tokens": 51
}
},
"response_metadata": {
"id": "msg_018G4mEZu8KNKtaQxZQ3o8YB",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 672,
"output_tokens": 51
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "get_weather",
"args": {
"city": "<UNKNOWN>",
"state": "<UNKNOWN>"
},
"id": "toolu_01DS9RwsFKdhHNYmhwPJHdHa",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 672,
"output_tokens": 51,
"total_tokens": 723
}
}
The tool_choice
argument has three possible values:
{ type: "tool", name: "tool_name" }
|string
- Forces the model to use the specified tool. If passing a single string, it will be treated as the tool name."any"
- Allows the model to choose the tool, but still forcing it to choose at least one."auto"
- The default value. Allows the model to select any tool, or none.
withStructuredOutput
import { ChatAnthropic } from "@langchain/anthropic";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { z } from "zod";
const calculatorSchema7 = z
.object({
operation: z
.enum(["add", "subtract", "multiply", "divide"])
.describe("The type of operation to execute."),
number1: z.number().describe("The first number to operate on."),
number2: z.number().describe("The second number to operate on."),
})
.describe("A simple calculator tool");
const llm7 = new ChatAnthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
model: "claude-3-haiku-20240307",
});
// Pass the schema and tool name to the withStructuredOutput method
const modelWithTool7 = llm7.withStructuredOutput(calculatorSchema7);
const prompt7 = ChatPromptTemplate.fromMessages([
[
"system",
"You are a helpful assistant who always needs to use a calculator.",
],
["human", "{input}"],
]);
// Chain your prompt and model together
const chain7 = prompt7.pipe(modelWithTool7);
const response7 = await chain7.invoke({
input: "What is 2 + 2?",
});
console.log(response7);
{ operation: 'add', number1: 2, number2: 2 }
You can supply a “name” field to give the LLM additional context around
what you are trying to generate. You can also pass includeRaw
to get
the raw message back from the model too.
const includeRawModel7 = llm7.withStructuredOutput(calculatorSchema7, {
name: "calculator",
includeRaw: true,
});
const includeRawChain7 = prompt7.pipe(includeRawModel7);
const includeRawResponse7 = await includeRawChain7.invoke({
input: "What is 2 + 2?",
});
console.log(includeRawResponse7);
{
raw: AIMessage {
"id": "msg_01TrkHbEkioCYNHQhqxw5unu",
"content": [
{
"type": "tool_use",
"id": "toolu_01XMrGHXeSVTfSw1oKFZokzG",
"name": "calculator",
"input": {
"number1": 2,
"number2": 2,
"operation": "add"
}
}
],
"additional_kwargs": {
"id": "msg_01TrkHbEkioCYNHQhqxw5unu",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 552,
"output_tokens": 69
}
},
"response_metadata": {
"id": "msg_01TrkHbEkioCYNHQhqxw5unu",
"model": "claude-3-haiku-20240307",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 552,
"output_tokens": 69
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "calculator",
"args": {
"number1": 2,
"number2": 2,
"operation": "add"
},
"id": "toolu_01XMrGHXeSVTfSw1oKFZokzG",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 552,
"output_tokens": 69,
"total_tokens": 621
}
},
parsed: { operation: 'add', number1: 2, number2: 2 }
}
API reference
For detailed documentation of all ChatAnthropic features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_anthropic.ChatAnthropic.html