How to set up AI lead scoring for small business in a weekend

Most small teams do not have a strategy problem with AI lead scoring. They have an execution and safety problem.

This playbook shows you how to set up AI lead scoring for a small business with Zapier and OpenAI, ship something real in a weekend, and still keep your data, CRM, and compliance under control.

You will see:

• A 5 minute go / no go checklist
• Exact Webhooks by Zapier calls with current OpenAI request patterns
• Concrete PII redaction regexes and storage rules
• HubSpot and Salesforce field mappings and JSON payloads
• Testing, monitoring, and a clean migration path to server side

Introduction & 5 minute checklist (operator path)

Before you touch Zapier or your CRM, decide if this project should even start.

Who this weekend build is for

• Ops / RevOps lead with admin access to CRM and Zapier, light JavaScript comfort, and OpenAI account access.
• Sales manager who can define what a “hot” lead looks like and review a sample of scored leads.
• Engineer optional on day one, required later if volume or compliance gets serious.

5 minute go / no go checklist

Say “go” only if you can honestly check these boxes:

1. Data & consent
   • You know where leads come from (forms, ads, chat, uploads).
   • Your lead forms or contracts allow processing data with vendors for “service improvement” or similar language. If you are unsure and in a regulated space, stop and ask counsel.
2. Volume & cost sanity
   • You expect fewer than about 5,000 AI scored leads per month for now.
   • You are ok with a rough, hypothetical OpenAI bill in the low three figures or less for early testing. You will make this more concrete below.
3. Access & tools
   • Zapier account with Webhooks and Code by Zapier available.
   • Admin access to your CRM (HubSpot or Salesforce) to add fields and update records by API.
   • OpenAI API key created and stored in a password manager or secret vault.
4. Sales alignment
   • Someone on sales commits to review at least 20 AI scored leads and give feedback within 48 hours.
5. Risk tolerance
   • You agree that for the first 72 hours, AI will propose scores but humans decide whether to trust them for high value leads.

If any of these fail, your problem is not tooling. Fix consent, access, or ownership first.

More Read

Ai churn prediction for Shopify stores: 90-day playbook

Use AI To Fix Email Timing Without Spamming

AI onboarding email sequences for SaaS in one week

Production-ready AI cart abandonment email sequence for Shopify

No-code AI lead qualification automation playbook

Stack selection and tradeoffs

You have three real options for AI lead scoring:

1. Zapier plus OpenAI (what we build here)
2. Native CRM scoring features (HubSpot / Salesforce add ons or marketplaces)
3. Self hosted or serverless service that calls OpenAI or another model directly

Zapier plus OpenAI

Best if your main constraint is speed and low engineering.

• Pros
  • No infrastructure work on day one.
  • Ops can own flow changes.
  • Easy to plug into existing form and CRM automations.
• Cons
  • Limited control over concurrency and retries.
  • JavaScript environment is constrained, third party libraries like Ajv are not guaranteed.
  • Data goes through Zapier, which matters for some jurisdictions.
• Failure modes
  • Zap loops that call OpenAI more than once per lead.
  • Parsing errors when the model returns slightly off JSON.
  • PII sitting unredacted in Zap history or Google Sheets steps.

CRM native scoring

Best if you want simpler rules, not AI, or your legal team blocks external AI vendors.

• Use this for deterministic rules like “score +10 if company size > 200” or “score +5 if job title contains ‘VP’”.
• Many teams get most of the value from rules based scoring with almost no risk.

Self hosted or serverless service

Best if control, cost, or privacy is your main constraint.

• Pros
  • Control batching, retries, logging, and residency.
  • Better unit tests and observability.
• Cons
  • Requires engineering and deployment.
  • Slower to ship the first version.

Rough cost break even template

Use this as a planning tool, not an exact forecast.

Monthly_OpenAI_cost ≈ calls_per_month × avg_tokens_per_call / 1000 × cost_per_1k_tokens

Break_even_engineering_months ≈ (engineering_hours × hourly_rate) / monthly_savings

Imagine you call the model 3,000 times per month at a modest token count and get a hypothetical OpenAI bill of 80 dollars. If an engineer needs 15 hours at 150 dollars per hour to build a server side service, that is 2,250 dollars in labor. To justify that, you would want a long enough time horizon or higher monthly savings.

Rule of thumb: stay on Zapier until either:

• Your Zapier plus OpenAI cost feels uncomfortable relative to lead value, or
• You start hitting timeouts, rate limits, or compliance constraints.

Minimal runnable pipeline (ops version)

This section gives you a Zapier pipeline you can build in an afternoon.

Target architecture

1. Trigger: new or updated lead in CRM or form tool.
2. Step 1: Formatter or Code step to construct a compact text summary of the lead.
3. Step 2: Code step to redact PII from that summary.
4. Step 3: Webhooks by Zapier step to call OpenAI.
5. Step 4: Code step to validate and normalize the AI response.
6. Step 5: CRM update step to save score and explanation.
7. Step 6: Optional Slack alert for “Hot” or “Ambiguous” leads.

Trigger setup

Example: HubSpot new contact

1. Create a new Zap.
2. Trigger app: HubSpot.
3. Event: New Contact or New Contact in List.
4. Connect your HubSpot account, test trigger, and pull a sample contact.

Example: Salesforce new lead

1. Trigger app: Salesforce.
2. Event: New Record with Object = Lead.
3. Test trigger and pull a sample.

Step: build input summary

Add a Code by Zapier step, choose JavaScript.

Input Data (example keys):

• first_name → map from CRM
• last_name → map from CRM
• email → map from CRM
• phone → map from CRM
• company → map from CRM
• job_title → map from CRM
• lead_source → map from CRM
• description → form free text, chat transcript, or notes
• country → map from CRM
• existing_mql_score → rules based score if you have one

Code:

const {
  first_name,
  last_name,
  email,
  phone,
  company,
  job_title,
  lead_source,
  description,
  country,
  existing_mql_score
} = inputData;

const summary = `
Name: ${first_name || ''} ${last_name || ''}
Email: ${email || ''}
Phone: ${phone || ''}
Company: ${company || ''}
Job title: ${job_title || ''}
Lead source: ${lead_source || ''}
Country: ${country || ''}
Existing_rules_score: ${existing_mql_score || 'N/A'}
Notes: ${description || ''}
`.trim();

return { summary };

At this point summary still has PII. The next step strips it out before OpenAI sees it.

Step: PII redaction (Code by Zapier)

Add another Code by Zapier JavaScript step.

Input Data:

• text → map from previous step’s summary

Code:

const text = inputData.text || '';

function redact(text) {
  let out = text;

  // Emails
  out = out.replace(
    /[a-zA-Z0-9._%+-]+@(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}/g,
    '[EMAIL]'
  );

  // Phone numbers (simple international patterns)
  out = out.replace(
    /(\+?\d{1,3}[\s.-]?)?(?:\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4})/g,
    '[PHONE]'
  );

  // Simple credit card pattern (very defensive)
  out = out.replace(
    /\b(?:\d[ -]*?){13,19}\b/g,
    '[CARD]'
  );

  // Names from first/last if you want to strip them too
  // You might choose to keep company names, but remove person names
  if (inputData.first_name) {
    const reFirst = new RegExp(inputData.first_name, 'gi');
    out = out.replace(reFirst, '[FIRST_NAME]');
  }
  if (inputData.last_name) {
    const reLast = new RegExp(inputData.last_name, 'gi');
    out = out.replace(reLast, '[LAST_NAME]');
  }

  return out;
}

const redacted = redact(text);

return { redacted };

This is intentionally simple but far better than sending raw content. You can expand it in Appendix style later.

PII redaction quick test harness

Before going live, drop a few strings into this step and click “Test”. Example inputs and expected outputs:

• Input: Contact John Smith at john.smith@example.com or +1 415-555-1234
  Expected: Contact [FIRST_NAME] [LAST_NAME] at [EMAIL] or [PHONE]
• Input: Card 4111 1111 1111 1111 used for trial
  Expected: Card [CARD] used for trial

Step: OpenAI API call with Webhooks by Zapier

Now you score the redacted summary.

1. Add a Webhooks by Zapier step.
2. Choose Custom Request.

URL: https://api.openai.com/v1/responses

Method: POST

Data Pass-Through?: No

Headers:

• Authorization → Bearer YOUR_OPENAI_API_KEY
• Content-Type → application/json

Data (Body) as raw JSON, using Zapier’s field mapping for the redacted summary:

{
  "model": "gpt-4.1-mini",
  "input": [
    {
      "role": "system",
      "content": "You are a B2B marketing ops assistant. Given a redacted lead summary, assign a numeric score from 0 to 100 and a tier label: Hot, Warm, or Cold. Use only the information provided. Do not guess PII.
      
Return a strict JSON object with this shape:
{
  \"score\": number between 0 and 100,
  \"tier\": \"Hot\" | \"Warm\" | \"Cold\",
  \"reason\": string (one short sentence),
  \"risk_flags\": string[] (zero or more of: \"low_intent\", \"bad_fit\", \"missing_info\")
}"
    },
    {
      "role": "user",
      "content": "Lead summary:\n{{steps.redact_step.redacted}}"
    }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "lead_score",
      "schema": {
        "type": "object",
        "properties": {
          "score": { "type": "number", "minimum": 0, "maximum": 100 },
          "tier": { "type": "string", "enum": ["Hot", "Warm", "Cold"] },
          "reason": { "type": "string" },
          "risk_flags": {
            "type": "array",
            "items": {
              "type": "string",
              "enum": ["low_intent", "bad_fit", "missing_info"]
            }
          }
        },
        "required": ["score", "tier", "reason", "risk_flags"],
        "additionalProperties": false
      },
      "strict": true
    }
  }
}

In Zapier, replace {{steps.redact_step.redacted}} with the actual mapped field from your redaction step. The important piece here is response_format with json_schema. That tells the OpenAI API to return structured JSON that matches the schema in output[0].content[0].text.

Exact Webhook by Zapier POST example

This is what Zapier sends on the wire, conceptually:

POST /v1/responses HTTP/1.1
Host: api.openai.com
Authorization: Bearer YOUR_OPENAI_API_KEY
Content-Type: application/json

{
  "model": "gpt-4.1-mini",
  "input": [
    { "role": "system", "content": "..." },
    { "role": "user", "content": "Lead summary:\nName: [FIRST_NAME] ..." }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "lead_score",
      "schema": { "...": "as above" },
      "strict": true
    }
  }
}

Step: parse and validate model output in Zapier

OpenAI’s /v1/responses returns JSON. The actual scored JSON lives in a nested structure. In Webhooks by Zapier, your next Code step will see a “Raw Body” string. You need to parse it.

Add a Code by Zapier JavaScript step.

Input Data:

• body → map from Webhook Raw Body (or similar field Zapier shows)

Code:

const body = inputData.body;

let parsed;
try {
  parsed = JSON.parse(body);
} catch (e) {
  // Hard failure, send to manual review
  return {
    valid: false,
    error: 'invalid_json',
    message: e.message
  };
}

// Expecting: parsed.output[0].content[0].text
let text;
try {
  const firstOutput = parsed.output && parsed.output[0];
  const firstContent = firstOutput && firstOutput.content && firstOutput.content[0];
  text = firstContent && firstContent.text;
} catch (e) {
  return {
    valid: false,
    error: 'missing_output',
    message: e.message
  };
}

if (!text) {
  return { valid: false, error: 'empty_text', message: 'No text in response' };
}

let data;
try {
  data = JSON.parse(text);
} catch (e) {
  return { valid: false, error: 'inner_json_parse', message: e.message };
}

// Lightweight schema checks instead of Ajv
function validate(obj) {
  if (typeof obj.score !== 'number') return 'score_not_number';
  if (obj.score < 0 || obj.score > 100) return 'score_out_of_range';
  if (typeof obj.tier !== 'string') return 'tier_not_string';
  if (!['Hot', 'Warm', 'Cold'].includes(obj.tier)) return 'tier_invalid';
  if (typeof obj.reason !== 'string') return 'reason_not_string';
  if (!Array.isArray(obj.risk_flags)) return 'risk_flags_not_array';
  for (const f of obj.risk_flags) {
    if (!['low_intent', 'bad_fit', 'missing_info'].includes(f)) {
      return 'risk_flag_invalid';
    }
  }
  return null;
}

const error = validate(data);
if (error) {
  return {
    valid: false,
    error,
    raw_text: text
  };
}

// Coerce score to integer for CRM if needed
const numericScore = Math.round(data.score);

return {
  valid: true,
  score: numericScore,
  tier: data.tier,
  reason: data.reason,
  risk_flags: JSON.stringify(data.risk_flags)
};

This in Zap validator replaces Ajv. You do simple type and range checks and handle failures explicitly.

Step: deterministic fallback rules

Use Paths by Zapier or a Filter step:

• If valid == true → update CRM with AI score.
• If valid == false → set a safe default, tag lead for manual review, and optionally send Slack alert.

Example safe default rule:

• If AI fails → ai_lead_score = existing_rules_score or ai_tier = "Warm" but do not auto assign to reps. Ask for manual review.

Secure data handling and PII redaction

PII redaction regex library

You already used a few patterns above. Here is a small library you can adapt in a dedicated appendix style Code step or server side:

const patterns = {
  email: /[a-zA-Z0-9._%+-]+@(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}/g,
  phone: /(\+?\d{1,3}[\s.-]?)?(?:\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4})/g,
  creditCard: /\b(?:\d[ -]*?){13,19}\b/g,
  ipAddress: /\b(?:\d{1,3}\.){3}\d{1,3}\b/g
};

function redactAll(text) {
  let out = text;
  out = out.replace(patterns.email, '[EMAIL]');
  out = out.replace(patterns.phone, '[PHONE]');
  out = out.replace(patterns.creditCard, '[CARD]');
  out = out.replace(patterns.ipAddress, '[IP]');
  return out;
}

Redaction test harness example

You can run a small fixed suite inside a Code step:

const tests = [
  {
    input: 'Email a@b.com',
    expected: 'Email [EMAIL]'
  },
  {
    input: 'Call +44 20 7946 0018',
    expected: 'Call [PHONE]'
  }
];

const results = tests.map(t => {
  const out = redactAll(t.input);
  return {
    input: t.input,
    output: out,
    pass: out === t.expected
  };
});

return { results: JSON.stringify(results, null, 2) };

Run this once while building to confirm your redactor behaves as expected.

Storage and retention rules

For small teams, simple rules beat complex tools.

• Where AI sees data
  • Send only redacted summaries to OpenAI.
  • Do not include internal IDs that tie directly to customer identities in the prompt.
• Where raw AI responses live
  • Store the scored outputs in CRM fields only. Avoid exporting full AI responses to shared spreadsheets.
  • If you keep raw JSON for debugging, use a restricted database, data warehouse, or secure storage where access is logged and limited.
• Retention
  • Set a policy such as “delete raw_responses after 90 days” and keep only the numeric score and tier for long term analytics.

Compliance checklist (operator friendly)

Use this as a conversation starter with legal or security, not as legal advice.

• Confirm whether leads are from jurisdictions such as the EU or other regions with strict privacy rules.
• Confirm that your terms or privacy notice allow vendor processing for scoring and analytics.
• Check that both OpenAI and Zapier offer a data processing agreement for your account tier if you need one.
• Decide whether any health, financial, or children’s data is involved. If yes, pause and escalate to counsel before sending any content to external AI vendors.
• Document: which fields you send to OpenAI, where redaction happens, retention period, who has access, and incident process if an error occurs.

OpenAI request patterns (current API best practices)

Choosing the endpoint and model

• Use POST /v1/responses with a model that supports structured output and JSON schema.
• Set response_format to a JSON schema so you can parse with confidence.
• Keep temperature low, such as 0 or 0.2, for stability.

Example: basic responses request

POST https://api.openai.com/v1/responses
Authorization: Bearer YOUR_OPENAI_API_KEY
Content-Type: application/json

{
  "model": "gpt-4.1-mini",
  "input": [
    {"role": "system", "content": "You are..."},
    {"role": "user", "content": "Lead summary: ..."}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": { "...": "as defined earlier" }
  }
}

Idempotency and retry guidance

In Zapier you do not control HTTP headers as fully as in custom code, but you can still avoid double scoring.

• Use a unique external id, like CRM lead ID, for each request. Store a flag in CRM such as ai_score_status = “scored” when done. Before calling OpenAI, check that status is not already “scored”.
• Retry only on transient status codes. Conceptually:
  • Safe to retry: 408, 429, 500, 502, 503, 504.
  • Do not retry on 400, 401, 403, 404.

Retry and backoff pseudo code

async function callWithRetry(requestFn, maxAttempts = 3) {
  let attempt = 0;
  let delayMs = 1000;

  while (attempt < maxAttempts) {
    attempt++;
    const res = await requestFn();

    if (res.ok) return res;

    const status = res.status;

    // Permanent errors
    if ([400, 401, 403, 404].includes(status)) {
      throw new Error(`Permanent error ${status}`);
    }

    // Transient errors: backoff
    if ([408, 429, 500, 502, 503, 504].includes(status)) {
      if (attempt === maxAttempts) {
        throw new Error(`Max attempts reached with status ${status}`);
      }
      await new Promise(r => setTimeout(r, delayMs));
      delayMs *= 2;
      continue;
    }

    // Other errors: treat as permanent for now
    throw new Error(`Unexpected status ${status}`);
  }
}

Zapier itself does not let you write this exact function, but you can mirror the logic using:

• Zapier’s built in automatic retries where available.
• Paths or filters to send repeated failures into a “manual review” Slack channel instead of looping forever.

Validator patterns (in Zap and remote)

A. Lightweight in Zap validator (no Ajv)

You already saw a basic version. Here is a compact template you can reuse:

function validateLeadScore(obj) {
  const errors = [];

  if (typeof obj !== 'object' || obj === null) {
    errors.push('not_object');
    return errors;
  }

  if (typeof obj.score !== 'number') errors.push('score_not_number');
  if (obj.score < 0 || obj.score > 100) errors.push('score_out_of_range');

  if (!['Hot', 'Warm', 'Cold'].includes(obj.tier)) errors.push('tier_invalid');
  if (typeof obj.reason !== 'string' || !obj.reason.trim()) errors.push('reason_missing');

  if (!Array.isArray(obj.risk_flags)) {
    errors.push('risk_flags_not_array');
  } else {
    const allowed = ['low_intent', 'bad_fit', 'missing_info'];
    for (const f of obj.risk_flags) {
      if (!allowed.includes(f)) errors.push('risk_flag_invalid:' + f);
    }
  }

  return errors;
}

Expected failure messages show up as task output so operators can see why a lead went to manual review.

B. Remote validator webhook

If your team has a lightweight serverless environment, you can host a small validator and call it from Zapier.

Example: Node style handler

// Express-like handler
app.post('/validate-lead-score', (req, res) => {
  const payload = req.body; // expect the model JSON object

  const errors = validateLeadScore(payload); // reuse function above

  if (errors.length === 0) {
    return res.json({ valid: true, normalized: payload });
  }

  return res.status(200).json({
    valid: false,
    errors,
    normalized: null
  });
});

In Zapier, you use Webhooks again:

• URL: your validator endpoint, for example https://your-api.example.com/validate-lead-score.
• Method: POST.
• Body: the object you parsed from OpenAI.

Zapier then branches based on valid.

CRM mapping templates (HubSpot & Salesforce)

This section gives sample fields you can create and exact update payloads.

HubSpot mapping template

Create these custom contact properties:

• ai_lead_score (number)
• ai_lead_tier (single line text or dropdown)
• ai_lead_reason (multi line text)
• ai_lead_risk_flags (multi line text)
• ai_score_status (single line text: “pending”, “scored”, “error”)

Sample HubSpot contact update JSON (for HubSpot API v3):

{
  "properties": {
    "ai_lead_score": "78",
    "ai_lead_tier": "Hot",
    "ai_lead_reason": "Ready to book a demo based on notes.",
    "ai_lead_risk_flags": "[\"missing_info\"]",
    "ai_score_status": "scored"
  }
}

In Zapier:

1. Add a HubSpot action: Update Contact.
2. Use the contact ID from the trigger.
3. Map:
   • ai_lead_score → score from validator step.
   • ai_lead_tier → tier.
   • ai_lead_reason → reason.
   • ai_lead_risk_flags → risk_flags.
   • ai_score_status → literal string “scored”.

Salesforce mapping template

Create custom fields on the Lead object:

• AI_Lead_Score__c (Number, 0 decimals)
• AI_Lead_Tier__c (Picklist: Hot, Warm, Cold)
• AI_Lead_Reason__c (Long Text Area)
• AI_Lead_Risk_Flags__c (Long Text Area)
• AI_Score_Status__c (Picklist: pending, scored, error)

Sample Salesforce REST update payload:

{
  "AI_Lead_Score__c": 78,
  "AI_Lead_Tier__c": "Hot",
  "AI_Lead_Reason__c": "Ready to book a demo based on notes.",
  "AI_Lead_Risk_Flags__c": "[\"missing_info\"]",
  "AI_Score_Status__c": "scored"
}

In Zapier:

1. Add Salesforce action: Update Record.
2. Object: Lead.
3. Record ID: from trigger.
4. Map custom fields from validator output.

Automation rules for human handoff

• HubSpot example
  • Workflow: when ai_lead_tier = Hot and ai_score_status = scored and lifecycle_stage = lead, then:
    • Assign to a rep queue.
    • Create a task “AI Hot Lead” due within one business day.
• Salesforce example
  • Process Builder or Flow: when AI_Lead_Tier__c = 'Hot', create a Task and notify owner.

Testing, metrics, and acceptance criteria

Seed a 20 row test dataset

Before turning the Zap on fully, prepare a simple spreadsheet or view with about 20 leads that cover:

• Very strong fit, high intent.
• Good fit, low intent.
• Poor fit, any intent.
• Ambiguous or sparse information.

Have sales or marketing label each as Hot, Warm, or Cold and give a simple numeric score they believe is fair. This is your “expected” set.

Automated comparator script

Export the AI scored results for these 20 leads to CSV. Use a small script to compare.

import csv

with open('expected.csv') as f:
    expected = {row['lead_id']: row for row in csv.DictReader(f)}

with open('actual.csv') as f:
    actual = {row['lead_id']: row for row in csv.DictReader(f)}

rows = []
for lead_id, exp in expected.items():
    act = actual.get(lead_id)
    if not act:
        continue
    exp_score = float(exp['score'])
    act_score = float(act['ai_lead_score'])
    exp_tier = exp['tier']
    act_tier = act['ai_lead_tier']

    diff = abs(exp_score - act_score)
    tier_match = exp_tier == act_tier

    rows.append({
        'lead_id': lead_id,
        'exp_score': exp_score,
        'act_score': act_score,
        'diff': diff,
        'exp_tier': exp_tier,
        'act_tier': act_tier,
        'tier_match': tier_match
    })

# Simple metrics
avg_diff = sum(r['diff'] for r in rows) / len(rows)
tier_accuracy = sum(1 for r in rows if r['tier_match']) / len(rows)

print('Average score difference:', avg_diff)
print('Tier accuracy:', tier_accuracy)

You can run this locally. Decide acceptability thresholds with sales, for example:

• Average score difference within 15 points.
• Tier accuracy at or above 0.75 for this first pass.

Calibration check idea

Over time, you want to know whether “Hot” really means “likely to convert”. As a simple early test, track for a month:

• Number of Hot leads that convert to opportunity.
• Number of Warm leads that convert.

If Hot and Warm convert at similar rates, your tiers do not separate outcomes yet. Adjust prompts or combine AI score with current rules.

Monitoring and alerting

Where to store monitoring data

• A simple database or warehouse table with columns such as:
  • lead_id
  • created_at
  • ai_lead_score
  • ai_lead_tier
  • ai_score_status
  • validator_error
• You can also use Airtable or a similar tool if SQL is not available.

Example SQL monitoring queries

Daily volume and error rate

SELECT
  DATE(created_at) AS day,
  COUNT(*) AS total_scored,
  SUM(CASE WHEN ai_score_status = 'error' THEN 1 ELSE 0 END) AS error_count,
  SUM(CASE WHEN ai_score_status = 'error' THEN 1 ELSE 0 END) * 1.0
    / COUNT(*) AS error_rate
FROM lead_scores
GROUP BY day
ORDER BY day DESC
LIMIT 14;

Distribution by tier

SELECT
  ai_lead_tier,
  COUNT(*) AS count
FROM lead_scores
WHERE created_at >= CURRENT_DATE - INTERVAL '7 days'
GROUP BY ai_lead_tier;

Grafana or Kibana style dashboards

If you have Grafana or Kibana:

• Panel 1: time series of daily total_scored and error_rate.
• Panel 2: stacked bar of tier distribution per week.
• Panel 3: table of last 20 errors with validator_error and a link to CRM.

Slack incident message templates

Use Zapier to send to Slack when certain conditions trip, such as error_rate above a threshold.

Template: high error rate

Channel: #ai-lead-scoring-alerts

Text:
"AI lead scoring alert

Window: last 60 minutes
Errors: {{error_count}}
Total scored: {{total_scored}}
Error rate: {{error_rate}}

Action:
- Check Zap 'AI Lead Scoring' task history
- If error_rate > 0.2 for more than 30 minutes, pause Zap and route new leads to manual review."

Template: unusual tier distribution

"AI lead scoring distribution change

Period: last 24 hours
Hot: {{hot_pct}}%
Warm: {{warm_pct}}%
Cold: {{cold_pct}}%

This differs from the previous 7-day average by more than 20 percentage points.

Action:
- Spot check 10 recent Hot and 10 recent Cold leads
- Confirm if prompt, model, or upstream fields changed."

Operational playbook and rollback

How to pause safely

• Step 1: Turn off the Zap that calls OpenAI.
• Step 2: In CRM automation, add or adjust rules so:
  • Leads are no longer auto assigned based on ai_lead_tier.
  • Fallback to your existing rules score or manual triage view.
• Step 3: Communicate to sales that AI scores are frozen or under review.

Idempotent fallback rules

Decide ahead of time what “manual mode” looks like.

• Example: if ai_score_status = 'error' or Zap is off, send new leads to:
  • Existing rules based scoring queue, or
  • A shared “Unscored” list for human review.

Incident tiers

• Minor: occasional parse errors below 5 percent. Log and refine prompts or validator.
• Moderate: error rate above 20 percent for more than an hour or model outputs are clearly off. Pause scoring for high value segments.
• Severe: PII found in model input that should have been redacted, or misrouting of high value leads. Pause Zap, delete or correct affected data, and review redaction step with security.

Scaling and next steps

When Zapier is enough

• Low to moderate volume.
• Sales and ops can maintain the flow without engineering.
• Legal is comfortable with Zapier as a processor.

When to migrate off Zapier

• Throughput: scoring starts to lag or you hit rate / time limits.
• Cost: Zapier task cost plus OpenAI cost exceeds your comfort, and you can batch leads more efficiently server side.
• Privacy: you need stronger control over where data is processed and stored.

Migration checklist to server side

1. Stabilize the contract
   • Lock in the JSON schema for lead scores.
   • Freeze CRM field names and meanings.
2. Rebuild the core logic in a service
   • Implement: PII redaction, OpenAI request, validator, CRM update, and logging.
   • Add proper retries, idempotency keys, and structured logging.
3. Shadow mode
   • Run server side scoring in parallel with Zapier for a while.
   • Compare outputs on a sample of leads.
4. Cutover
   • Disable Zapier AI scoring, keep other automations.
   • Switch CRM workflows to trust the new service’s scores.
5. Retire Zapier scoring
   • Archive or delete the old scoring Zap once you are confident.

Decision: what to do this weekend

Choose your next step clearly.

• Choose Zapier plus OpenAI if:
  • You can check the go / no go boxes.
  • Your priority is a working AI lead scoring test in a weekend.
  • Your volume is modest and privacy risks are understood.
• Choose rules only scoring if:
  • You need predictable, explainable scores.
  • You lack consent clarity or PII constraints are tight.
• Plan a server side service if:
  • You already see that volume and compliance needs will grow.
  • You can commit engineering time soon.

If speed is your current constraint, build the Zapier pipeline with strong redaction and validation, keep the first 72 hours in sample mode, and make the migration decision only after you have real scoring data.