AEO Scorecard Connector

A public "get your free scorecard" product that grades a company's AI search readiness (Answer Engine Optimization, or AEO) across roughly 30 data points, from Moz and SEMrush authority data to industry queries generated by fine-tuned prompts. It generated $700K in pipeline in a few weeks. Behind it sits a heavy, expensive analysis engine; this page covers the plumbing I built so thousands of signups couldn't crash it, double-bill it, or expose it.

DEEP DIVE · LEAD MAGNET INFRASTRUCTURE

production · Vercel Next.js Neon Postgres Vercel Cron HubSpot Forms Slack magic-link auth Calendly webhooks

Heavy engine, public edge

The analysis engine takes 15 to 30 minutes per company and costs real money every time it runs. Marketing wanted it as a public "get your free scorecard" lead magnet. Putting a slow, expensive engine directly on the public internet is how you get abuse, double-billing, and timeouts.

The answer was a deliberately small connector that owns intake, queueing, storage, and projection: and explicitly does not own the analysis pipeline, the landing page design, or the data dictionary. Clear ownership boundaries meant the frontend designer, the backend owner, and I could all ship independently.

flowchart TD
  FORM[Public lead form] --> HSF[Submit to HubSpot Forms]
  FORM --> RUN[Create run record in Neon]
  API[On-demand API
SDR / operator use] --> RUN
  HSIN[Request from HubSpot] --> RUN
  RUN --> DEDUPE{In-flight run for
same domain?}
  DEDUPE -->|Yes| REUSE[Reuse existing run
no double billing]
  DEDUPE -->|No| POST[POST engine queue
server-side credentials only]
  POST --> POLL[Automatic check-in
every 10 minutes]
  REUSE --> POLL
  POLL --> STORE[(Store canonical payload)]
  STORE --> PUB[Short-form projection
public scorecard page]
  STORE --> ADMIN[Admin dashboard
full raw payload + status]
  ADMIN --> EMAIL[HubSpot transactional email
with scorecard link]

The check-in loop that keeps it healthy

In plain English: every few minutes, the system checks on every running analysis, revives ones that got stuck, gives up on lost causes, and feeds new requests to the engine at a pace it can handle. A burst of signups can never overwhelm it. The details:

Never bites off too much: each check-in looks at a capped number of running analyses, with a time limit per lookup, so one slow response can't stall the whole round.
Revives the stuck: an analysis still "processing" after a few hours is assumed stuck inside the engine and automatically restarted.
Gives up on lost causes: anything that never finishes after a longer window gets marked failed instead of clogging the queue forever.
Meters the flow: only a fixed number of analyses run in the engine at once; the rest wait in line locally, so a spike in signups can't overwhelm the expensive part.
Never pays twice for the same company: if an analysis for that domain is already running or finished, the request reuses it. Forcing a fresh run requires an explicit override.

stateDiagram-v2
  [*] --> pending: run created
  pending --> queued: POST to engine
  queued --> processing: engine picks up
  processing --> ready: payload fetched + stored
  processing --> requeued: hung > N hours → re-POST
  requeued --> processing
  pending --> failed: stale > max hours
  processing --> failed: stale > max hours
  ready --> [*]
  note right of queued
    Only a fixed number run
    at once. The rest wait in
    line, so bursts never
    overwhelm the engine.
  end note

What the connector owns

Route	Purpose
/api/lead-form	Public landing-page intake: submits to HubSpot Forms and queues the analysis
/api/runs/on-demand	One-domain run generation for SDR/operator use, with dedupe-by-default
/api/runs/poll	The scheduled check-in loop described above
/api/strategy360-callback	Lets the engine announce "done" directly, as a backup to the check-in loop
/api/scorecards/[slug]	The trimmed public version of a finished scorecard
/api/scorecards/[slug]?format=raw	Exact raw engine response for debugging
/dashboard	Operator surface: run statuses, field counts, payload and page links
/admin/login	Company-only sign-in via short-lived magic links delivered by Slack DM
/api/inbound/calendly	Signed booking webhooks, closes the loop from scorecard to booked demo
/api/health	Deployment and integration readiness check

Route

Purpose

/api/lead-form

Public landing-page intake: submits to HubSpot Forms and queues the analysis

/api/runs/on-demand

One-domain run generation for SDR/operator use, with dedupe-by-default

/api/runs/poll

The scheduled check-in loop described above

/api/strategy360-callback

Lets the engine announce "done" directly, as a backup to the check-in loop

/api/scorecards/[slug]

The trimmed public version of a finished scorecard

/api/scorecards/[slug]?format=raw

Exact raw engine response for debugging

/dashboard

Operator surface: run statuses, field counts, payload and page links

/admin/login

Company-only sign-in via short-lived magic links delivered by Slack DM

/api/inbound/calendly

Signed booking webhooks, closes the loop from scorecard to booked demo

/api/health

Deployment and integration readiness check

One source of truth, two views

The analysis is stored once. The public landing page gets a trimmed version built for prospects; the internal dashboard keeps the complete record for the team. The frontend designer builds against a stable, connector-owned format, so the engine can change internally without ever breaking the public page.

Locked down by default, spends money deliberately

The keys never leave the building: the public site talks to the connector; only the connector holds the credentials to the expensive engine.
Development can't spend money: on a developer's machine, real analysis runs are disabled by default. Everything works for testing, but nothing hits the paid engine unless deliberately switched on for a paid test.
Missing security config blocks deployment: if the secret that protects the public data endpoints isn't set, production refuses to serve rather than serving unprotected.
No passwords to steal: the internal dashboard signs people in with short-lived login links sent by Slack, restricted to company email addresses. There is no password database to breach.
Abandoned bookings become follow-ups: if someone clicks "Book a Demo" but never finishes booking, the team gets a Slack alert after a grace period. An almost-customer is a sales signal, not silence.

"The connector's whole job is being boring: never pay twice, check on the slow thing, lock the public thing, and give the team one dashboard where nothing is mysterious."