NanoCrawl
HTTP-native micropayments for AI data access. Publishers set a price. AI agents pay per page — off-chain, zero gas. Humans browse for free.
The Problem
The web is broken for AI. Publishers block crawlers and earn nothing. Crawlers scrape anyway and risk lawsuits. NanoCrawl is the payment layer in between — an MCP server gives any AI agent a wallet, and a one-line middleware lets any publisher monetise their content at sub-cent granularity.
Architecture
The full round-trip: x402 over HTTP, off-chain EIP-3009 signing, Circle Gateway batch settlement on Arc, and dual withdrawal paths.
Cloudflare controls access.
NanoCrawl defines how machines pay for it.
Cryptographic, not accounting
Every payment is a signed EIP-3009 authorization — verifiable on-chain, non-repudiable, settled in batches. Cloudflare-style billing aggregates usage centrally. Circle Nanopayments signs per request.
Agent-native by design
No accounts. No billing cycles. No API keys. An AI agent reads robots.txt, discovers the price, signs off-chain, and pays — all autonomously via MCP. Sub-cent granularity makes per-page economics viable.
Cross-chain out of the box
Payments settle on Arc — where USDC is the native gas token, so merchants pay nothing to receive. Circle CCTP lets them sweep to Base, Unichain, or any supported chain in one call. No bridging UX, no manual steps.
Complements Cloudflare — doesn't compete
Cloudflare excels at detection and enforcement. NanoCrawl is the payment layer that plugs in behind it. Our Cloudflare Worker template deploys NanoCrawl as a Worker in front of any origin — Cloudflare guards the gate, NanoCrawl handles the economics.
How It Works
Crawler hits a protected page
Middleware classifies the request as a crawler. Returns HTTP 402 with a PAYMENT-REQUIRED header encoding price, network, and seller address.
Agent signs off-chain
MCP server reads the 402, signs an EIP-3009 authorization off-chain — zero gas, no on-chain transaction. Retries with a PAYMENT-SIGNATURE header.
Content delivered, revenue recorded
Circle Gateway verifies the signature and locks funds. Merchant sees revenue on the dashboard in real time. Merchant sweeps to any chain via CCTP.
See It Live
Two curl commands. The protocol is plain HTTP — no SDK, no wallet, no account.
curl -si https://nanocrawl.vercel.app/products/1 \
-H "User-Agent: AI-Crawler/1.0" \
| head -6
# HTTP/2 402
# payment-required: eyJ4NDAyVmVyc2lvbiI6MiwiYWNjZXB0cy...
# content-type: application/json
#
# {"x402Version":2,"accepts":[{"scheme":"exact","network":"eip155:5042002",
# "asset":"0x3600...","amount":"1000","payTo":"0xSeller..."}]}curl -si https://nanocrawl.vercel.app/products/1 \
-H "User-Agent: AI-Crawler/1.0" \
-H "PAYMENT-SIGNATURE: <base64-eip3009>" \
| head -4
# HTTP/2 200
# payment-response: eyJzdWNjZXNzIjp0cnVlLCJ0cmFuc2FjdGlvbiI6...
# {"id":1,"name":"Widget Alpha","price":29.99,...}curl https://nanocrawl.vercel.app/.well-known/ai-payKey Properties
Zero gas per payment
EIP-3009 off-chain authorization — no on-chain tx until batch settlement
HTTP-native
Standard 402 status code. No SDK required on the publisher side
Human pass-through
Browser signals detected. Humans never see a paywall
Cross-chain withdrawal
CCTP: burn USDC on Arc, mint on Base, Unichain, or any supported chain
Idempotent payments
UUID-based deduplication. Safe to retry on network failure
Drop-in for any site
Cloudflare Worker template gates any origin — static sites, APIs, R2 buckets
Built On
Circle Nanopayments
EIP-3009 · CCTP · Gateway
x402 Protocol
HTTP payment standard
Arc Testnet
USDC as native gas
Model Context Protocol
AI agent wallet layer