Get Started as a Merchant
Add x402 payment gating to your site. AI agents pay per page. Humans browse for free. No blockchain plumbing required.
What NanoCrawl adds to your site
One integration gives you four things. The first two are automatic — no extra config needed.
Auto-served at /robots.txt with payment metadata: price, network, seller address, verifying contract. Any crawler that follows web standards finds it immediately.
RFC 9309 · auto
Auto-served JSON manifest at /.well-known/ai-pay. Agents and SDKs parse it to bootstrap a GatewayClient without hitting a 402 first. Typed structured data.
Emerging standard · auto
Middleware classifies requests as crawler vs human. Crawlers get a 402 with PAYMENT-REQUIRED header. Humans pass through. One import.
Next.js or CF Worker · configure
Circle Gateway verifies EIP-3009 sigs, locks funds. Revenue streams to the dashboard in real time. Withdraw to any chain via CCTP.
Circle Gateway · configure
Discovery endpoints (auto)
NanoCrawl serves these automatically once the seller wallet is configured. No extra steps needed — they're included in every integration.
GET /robots.txtRFC 9309 extensionUser-agent: AI-Crawler Allow: /products Crawl-fee: 0.001 USDC Payment-Network: eip155:5042002 Payment-Asset: 0x3600... Payment-PayTo: 0xSeller... Payment-VerifyingContract: 0x0077...
Enables the proactive flow: agent pre-signs EIP-3009 from this metadata, sends PAYMENT-SIGNATURE on the first request — saves one full HTTP round-trip per page.
GET /.well-known/ai-payStructured JSON manifest{
"protocol": "x402",
"accepts": [{ "network": "eip155:5042002",
"asset": "0x3600...", "amount": "1000",
"payTo": "0xSeller...", ... }],
"routes": [{ "pathPattern": "/products/*",
"amountUsdc": 0.001 }]
}Machine-readable bootstrap for agent SDKs. Parse once, cache, use for all requests. Analogous to /.well-known/openid-configuration.
Option A — Next.js middleware
One environment variable and a single import. Works with any Next.js 13+ App Router project.
1. Install the SDK
npm install @circle-fin/x402-batching @x402/evm
2. Set environment variables
# .env.local NANOCRAWL_SELLER_PRIVATE_KEY=0x<your-wallet-private-key> NEXT_PUBLIC_SELLER_WALLET=0x<your-wallet-address> PRICE_USDC=0.001
Use the same wallet for both. The private key stays server-side only.
3. Add the middleware
// middleware.ts (project root) export { nanocrawlMiddleware as middleware } from '@nanocrawl/middleware' export const config = { matcher: ['/products/:path*'] }Gate any route pattern. Humans pass through. Crawlers get a 402.
4. Check the dashboard
Revenue appears at /nanocrawl/dashboard in real time as crawlers pay.
Option B — Cloudflare Worker
A standalone Worker that proxies any origin. Works in front of static sites, APIs, R2 buckets — anything behind a Cloudflare zone. No Node.js required, no external dependencies.
1. Create the Worker
npm create cloudflare@latest nanocrawl-worker
Replace the generated
src/index.tswith the NanoCrawl CF Worker template.2. Set secrets and variables
wrangler secret put SELLER_WALLET wrangler secret put PRICE_USDC # optional, default 0.001 # wrangler.toml [vars] ORIGIN_URL = "https://your-real-site.com"
3. Deploy
wrangler deploy
Arc Testnet Reference
Need test USDC? Hit the Circle faucet and then call client.deposit() to fund the Gateway balance.