Detailed guide

This guide explains what idlidu is, how the www site is organized, how data moves from the open web to your API key, and how billing, indices, and cross-dataset joins fit together. It is written for a careful human reader — not as legal advice, and not as a claim that every host is paid general availability.

1. The big picture in one page

idlidu is one brand with many small data businesses underneath it. Each business is called a track (or asset / dataset). Tracks do not sell consulting or custom research memos. They sell self-serve machine-readable history (and a few computed checks) that AI agents and scripts can call with an API key.

Think of three layers:

Behind every track is roughly the same factory line:

  watch public sources → save raw bytes (+ hash + time)
       → notice change → extract structured fields → publish an index
                              ↓
         bill keys / usage ← serve HTTP + MCP ← document honestly

Raw history is the moat. Extraction can be imperfect; the product still keeps yesterday’s bytes so agents can cite evidence later.

2. How to use this guide

Shorter stepwise checklist: Handbook. Machine bootstrap: /handbook.json.

3. The www website — every major page

The brand front door is https://www.idlidu.com. It is not a dashboard of twenty-one equal tiles. It is a router: brand first, then honest product status, then docs.

3.1 Home (/)

• States what idlidu is in plain language.
• Points to featured / soft-launch products and the handbook / guide.
• Soft-launch honesty: not every slug is “paid GA.”

3.2 Products (/products/)

• Featured: usually finance (the usual onboarding host).
• Soft-launch cohort: other tracks with live docs/API hosts but waitlist-style checkout until card PSP is named.
• Each card links to the track’s docs site ({name}.idlidu.com), not into a fake mall cart.

3.3 Indices (/indices/ + /indices.json)

Citeable time series (baskets, drift counts, adoption shares, join rates).
Free sample = last point only on each track’s GET /v1/public/sample/index.
Full history needs Growth+ on that track. These are not risk scores, credit ratings, or malware verdicts.

3.4 Handbook (/handbook/ + /handbook.json)

Stepwise operating manual: hosts, auth, OpenAPI, MCP, buyer path, agent playbook, portfolio map, pricing summary, troubleshooting.

3.5 This guide (/guide/)

Longer narrative of website + factory + workflows (you are here). Markdown source also ships as /guide.md.

3.6 Developers (/developers/)

Short path: get a key idea → call HTTP → optional MCP. Links OpenAPI examples and finance agents pages.

3.7 Pricing (/pricing/)

Shared list-price ladder (Starter → Institution), bulk SKUs, multi-dataset packs, specialty join packs. Until Cashfree/PayPal (or crypto) secrets are live, checkout is waitlist / webhook adapter, not “buy now with card.”

3.8 Legal (/legal/, Terms, Privacy, AUP)

Platform terms. Owner-approved drafts may still be marked not counsel-reviewed. Track sites add their own disclaimer tokens.

3.9 Bot (/bot/)

How idlidu’s own fetchers identify themselves (idlidu-bot/1.0 and track-specific bots) and how to contact us about crawling.

Navigation mental model

www  →  pick a product  →  track docs (methodology / pricing / agents)
                       →  track API  (OpenAPI + /v1/…)
                       →  track MCP  (tools/list + tools/call)

4. What a “track” is (one product unit)

Every track is a small vertical: its own seed universe, fixtures, Workers, D1/R2 where provisioned, billing key prefix, OpenAPI, and disclaimer.

4.1 Three public hosts

Examples: finance.idlidu.com, apibreak-api.idlidu.com, uaweb-mcp.idlidu.com.

4.2 Three product shapes

4.3 Inside the repo (for orientation)

A typical track folder looks like:

{track}/
  seed/           curated allowlist (YAML/JSON)
  fixtures/       offline responses for CI / soft-launch demos
  packages/       config, billing, core/db, extracts
  apps/
    api/          Hono Worker — /v1 routes
    mcp/          MCP Worker
    ingest/       cron / queue stub or live fetch (ToS-gated)
    web/          Astro docs site
  openapi/        contract
  ACTION-PLAN.md  build plan + kill criteria

You do not need the repo to use the product — OpenAPI and docs are enough — but this map explains why three hosts exist and why “fixture mode” appears in health responses.

5. The factory workflow (end-to-end)

5.1 Watch

On a schedule (cron), or on demand for Shape C, the ingest Worker looks at a capped allowlist of public URLs or APIs. Universes start tiny (tens of entities) and grow only when pull and ToS allow.

5.2 Save raw

Original bytes (or a faithful snapshot) go to object storage when provisioned, with:

• content hash
• fetch timestamp
• source URI

This is the unforgeable “we saw this” layer.

5.3 Notice change

If today’s hash ≠ yesterday’s, write a change / drift / break event with a stable id and methodology version.

5.4 Extract

Turn messy raw into queryable fields (ratios, owners, schema widen flags, …). Bad extracts must not delete raw.

5.5 Index

Publish at least one methodology-versioned series buyers can cite. Examples:

• credit conditions percentiles (finance)
• silent API break / mismatch rate (apibreak ± S1 join)
• llms.txt adoption share
• managed K8s list-price basket

5.6 Serve

OpenAPI wins when docs and code disagree.

5.7 Bill

Plans: Starter / Growth / Archive / Scale / Institution (Shape C has a higher ladder).
Metering: monthly call quotas; soft 429 + upgrade URL.
Webhook: POST /v1/billing/webhook with HMAC; mints idl_{track}_… keys.
Optional specialty pack sku (e.g. pack_api_twin) binds Archive + join features on the home track.

5.8 Document & hands-off

Each track keeps methodology + legal pages. CI / hands-off scripts prove the fixture path still works without live hammering of third parties.

What stays off the line

• Human research memo SKUs
• Support desk / “book a demo” consulting
• Opaque “risk scores” without transparent counting
• Running the customer’s full HPC job on Cloudflare as the main SKU

6. How a human buyer actually works the site

1. Land on www — read the promise; open Products.
2. Pick one track — prefer finance for first success; otherwise a soft-launch host you care about.
3. Read methodology on {name}.idlidu.com/methodology/ — what is measured, what is not.
4. Skim pricing — plan caps, history depth, bulk.
5. Get a key — waitlist / manual / webhook until card checkout is named; never commit keys.
6. Call health + OpenAPI + a public sample before spending quota.
7. Authenticate with Authorization: Bearer idl_….
8. Wire your agent — cache entity ids; prefer as-of queries; watch usage.
9. Escalate only when docs fail — mail [email protected]; expect multi-day replies.

Example (finance-shaped):

curl -sS "https://finance-api.idlidu.com/v1/health"
curl -sS "https://finance-api.idlidu.com/v1/openapi.json" | head
curl -sS -H "Authorization: Bearer $IDLI_KEY" \
  "https://finance-api.idlidu.com/v1/banks?q=…"

Exact paths differ — always check that track’s OpenAPI.

7. How an agent / automation should work

1. Fetch https://www.idlidu.com/handbook.json for the portfolio map.
2. Fetch https://{name}-api.idlidu.com/v1/openapi.json.
3. Prefer JSON fields over scraping HTML.
4. Loop: health → openapi → (public sample) → auth search → as-of / events / index → usage → stop.
5. On MCP runtimes, point at {name}-mcp and list tools — do not invent names.
6. When answering a human: cite dataset, methodology version, as-of, and the track disclaimer.
7. Never claim idlidu endorsed a bank, vendor, package, model, or crawl policy.

Customer agents calling our APIs need keys. That is not a license to scrape third-party sites.

8. Pricing, packs, and checkout workflow

Canonical list: Pricing · source common/PRICING.md.

8.1 Plan families (Shape A/B default)

Shape C (compute checks) uses a higher ladder (199 → 6,999).

8.2 Packs

Generic: pack_3_growth, pack_5_archive, cohort / scale / institution portfolio packs.
Specialty (join stories): pack_geo_observe, pack_api_twin, pack_agent_surface, pack_k8s_twin — bound via webhook sku on home tracks; card UI still waitlist until PSP wires.

8.3 Checkout reality (soft-launch)

Buyer interest → waitlist / [email protected]
       or HMAC webhook checkout.completed { plan | sku }
       → mint track API key
       → buyer calls API / MCP

Cashfree/PayPal card and crypto Path A/B are designed but may still be deferred (secrets / ready flags). x402 stays off.

9. Indices and cross-dataset joins (value layer)

Research map: common/value/ · machine index common/value/opportunities.json.

9.1 Why joins exist

Competitors often archive one surface (specs only, robots only, prices only). idlidu’s wedge is paired observations:

Join event feeds are typically Archive+ (or the specialty pack). Indices may still be Growth+.

9.2 Soft-launch honesty for joins

Early join tables use entity maps + fixture / denormalized citations. They prove the product shape. Live cross-Worker sync and larger seed overlap come later. Kill rule: if fewer than ~3 unique paid keys use join routes in 90 days, demote to a soft report.

10. Portfolio map (what each dataset is for)

Status language on www Products is authoritative for marketing. Hosts may exist while “paid GA” is gated.

Soft-launch cohort (1–10)

Expansion set (11–21)

11. Ops, spend, and soft-launch posture

• Spend soft-cap this stage: about USD 500/mo across Workers / D1 / R2 / Pages / fetch.
• Crons may be on and staggered; some tracks stay fixture-only when ToS or cost gates apply.
• gpucap live cloud allocations stay gated (cost blowup).
• Monitor probes /v1/health across API / MCP / ingest / web.
• No support desk — founders mail only.
• Marketing must not invent GA, counsel review, or brokerage claims the product does not have.

12. Legal and ethics (human summary)

• Read Terms, Privacy, AUP.
• Outputs are not credit ratings, legal advice, malware verdicts, CPI, security certifications, chemical offers, or recruiting services.
• Allowed: lawful use of keys, caching within plan, agents that respect rate limits.
• Forbidden: key resale, credential stuffing, using our bots to bypass third-party paywalls, illegal content, abuse of shared infra.
• idlidu fetchers respect source robots.txt; see /bot/.

13. Troubleshooting ladder

14. Quick reference

Brand guide:   https://www.idlidu.com/guide/
Handbook:      https://www.idlidu.com/handbook/
handbook.json: https://www.idlidu.com/handbook.json
Products:      https://www.idlidu.com/products/
Indices:       https://www.idlidu.com/indices/
Pricing:       https://www.idlidu.com/pricing/
Developers:    https://www.idlidu.com/developers/
Legal:         https://www.idlidu.com/legal/
Bot:           https://www.idlidu.com/bot/
Mail:          [email protected]

Per track:     {name}.idlidu.com
               {name}-api.idlidu.com/v1/…
               {name}-mcp.idlidu.com

Auth:          Authorization: Bearer idl_…
Discover:      GET /v1/health
               GET /v1/openapi.json
               GET /v1/public/sample/…  (where offered)

15. Related deeper docs (repo)

Not a launch claim. Not legal advice. Not a substitute for per-track methodology. Soft-launch hosts may exist before paid card checkout is live.