{"slug":"realtor-lead-scraping-pipeline","title":"Realtor Lead Scraping and Verification Pipeline for Cold Email","tags":["scraping","lead-gen","cold-email","realtors","data-processing"],"agent_summary":"End-to-end pipeline for scraping realtor leads, verifying data quality, deduplicating against existing campaigns, and preparing for cold email outreach.","trigger_phrases":["realtor scraping pipeline","lead scraping workflow","cold email lead pipeline","scrape and verify leads","realtor lead generation","dedup scraped leads"],"runnable":true,"markdown":"## Realtor Lead Scraping Pipeline\n\n### Pipeline Stages\n\n#### 1. Scraping Layer\n- Use Zillow scraper (verified working)\n- Target specific geos/service areas\n- Cache results to avoid duplicate API calls across cities\n\n#### 2. ICP Verification\nFilter for Ideal Customer Profile:\n- Correct service area (instant cut if outside)\n- Actual realtor (not assistant/team member)\n- 99% confidence threshold\n\n#### 3. Deduplication Layers\nRemove duplicates against:\n- Existing leads in Superbase (previously scraped)\n- Existing leads in Instantly campaigns (already being emailed)\n- Existing client customers (from client's CRM export)\n\n#### 4. Data Normalization\n- Standardize company names\n- Clean phone numbers\n- Normalize email formats\n- Consistent field mapping\n\n#### 5. Email Verification\n- Validate emails before adding to campaigns\n- Remove bounced/invalid addresses\n- Flag catch-all domains\n\n### Superbase Schema Requirements\nEach client table needs:\n- `existing_customers` column for dedup\n- Standard lead fields (name, email, phone, company)\n- `source` field (which scraper)\n- `scraped_date` timestamp\n\n### Quality Control Process\n```\n1. Run small test batch (100 leads)\n2. Run through full pipeline\n3. Benchmark quality at end\n4. If quality passes, run full service area scrape\n5. If issues, debug before scaling\n```\n\n### Proxy Management\n- Use residential proxies for scraping\n- Monitor credit usage\n- Small batch test before full scrape to preserve credits\n\n### Campaign Integration\nAfter pipeline completion:\n1. Export verified leads\n2. Import to Instantly AI\n3. Assign to appropriate client campaign\n4. Monitor bounce rates (pause if >5%)\n\n### Handling Multiple Clients\n- Run pipelines for both clients together\n- Each client has separate Superbase table\n- Each client has separate Instantly campaigns\n- Dedup runs per-client\n","html":"<h2>Realtor Lead Scraping Pipeline</h2>\n<h3>Pipeline Stages</h3>\n<h4>1. Scraping Layer</h4>\n<ul>\n<li>Use Zillow scraper (verified working)</li>\n<li>Target specific geos/service areas</li>\n<li>Cache results to avoid duplicate API calls across cities</li>\n</ul>\n<h4>2. ICP Verification</h4>\n<p>Filter for Ideal Customer Profile:</p>\n<ul>\n<li>Correct service area (instant cut if outside)</li>\n<li>Actual realtor (not assistant/team member)</li>\n<li>99% confidence threshold</li>\n</ul>\n<h4>3. Deduplication Layers</h4>\n<p>Remove duplicates against:</p>\n<ul>\n<li>Existing leads in Superbase (previously scraped)</li>\n<li>Existing leads in Instantly campaigns (already being emailed)</li>\n<li>Existing client customers (from client's CRM export)</li>\n</ul>\n<h4>4. Data Normalization</h4>\n<ul>\n<li>Standardize company names</li>\n<li>Clean phone numbers</li>\n<li>Normalize email formats</li>\n<li>Consistent field mapping</li>\n</ul>\n<h4>5. Email Verification</h4>\n<ul>\n<li>Validate emails before adding to campaigns</li>\n<li>Remove bounced/invalid addresses</li>\n<li>Flag catch-all domains</li>\n</ul>\n<h3>Superbase Schema Requirements</h3>\n<p>Each client table needs:</p>\n<ul>\n<li><code>existing_customers</code> column for dedup</li>\n<li>Standard lead fields (name, email, phone, company)</li>\n<li><code>source</code> field (which scraper)</li>\n<li><code>scraped_date</code> timestamp</li>\n</ul>\n<h3>Quality Control Process</h3>\n<pre><code>1. Run small test batch (100 leads)\n2. Run through full pipeline\n3. Benchmark quality at end\n4. If quality passes, run full service area scrape\n5. If issues, debug before scaling\n</code></pre>\n<h3>Proxy Management</h3>\n<ul>\n<li>Use residential proxies for scraping</li>\n<li>Monitor credit usage</li>\n<li>Small batch test before full scrape to preserve credits</li>\n</ul>\n<h3>Campaign Integration</h3>\n<p>After pipeline completion:</p>\n<ol>\n<li>Export verified leads</li>\n<li>Import to Instantly AI</li>\n<li>Assign to appropriate client campaign</li>\n<li>Monitor bounce rates (pause if >5%)</li>\n</ol>\n<h3>Handling Multiple Clients</h3>\n<ul>\n<li>Run pipelines for both clients together</li>\n<li>Each client has separate Superbase table</li>\n<li>Each client has separate Instantly campaigns</li>\n<li>Dedup runs per-client</li>\n</ul>\n"}