Web Scraping as an Anthropic Tool Use Function

Anthropic's tool use API lets Claude call external functions mid-conversation. Define a scrape tool and Claude will invoke it when a question requires live web data. The structured JSON response — title, text, links, metadata — is exactly what Claude needs to reason, summarize, or answer questions about a page.

Get API Key →Read the Docs

What you'd use this for

Claude-powered research assistants that browse the web, customer support bots that look up live docs, coding assistants that read API documentation, any Claude workflow that benefits from live web access.

How it works

example

import anthropic, httpx

client = anthropic.Anthropic()

tools = [{
    "name": "fetch_url",
    "description": "Fetch and extract content from any URL. Returns title, text, links, and metadata.",
    "input_schema": {
        "type": "object",
        "properties": {
            "url": {"type": "string", "description": "The URL to fetch"},
            "render_js": {"type": "boolean", "description": "Use Playwright for JS-rendered pages"}
        },
        "required": ["url"]
    }
}]

def fetch_url(url: str, render_js: bool = False) -> str:
    r = httpx.post("https://api.denkbot.dog/scrape",
        headers={"Authorization": f"Bearer {DENKBOT_API_KEY}"},
        json={"url": url, "renderJs": render_js, "format": "json"})
    d = r.json()
    return f"Title: {d['title']}\nText: {d['text'][:3000]}"

response = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "What does https://example.com say?"}]
)