Car Sniper

Car Sniper is the reconstructed monorepo for a multi-service car-search and alerting stack that originally lived across separate repositories. This open-source release is intended as a historical record of the system: what it did, how the services fit together, and how the code evolved over time.

At its peak, the stack was built to watch fresh private-party vehicle listings and live auctions, store them, let users search them on the website, and push updates over websockets. Later, a Go rewrite was started for some of the higher-volume by-owner sources.

Repository layout

• car-site/: the customer-facing Express and EJS website
• car-socket/: the Socket.IO and MySQL event hub
• car-scraper/: the original JavaScript scraper suite
• car-image-server/: the small image ingest and hosting service
• go-car-scraper/: the later Go rewrite for selected inventory sources

What the system did

The preserved code shows a system with four major capabilities:

1. Marketplace ingestion from many sources, especially private-seller inventory and enthusiast auction sites.
2. Real-time fan-out over websockets, with a central hub that stored listings and rebroadcast new results.
3. End-user search and workflow features such as filters, saved cars, hidden cars, login, and subscription pages.
4. Operational scaling work, including a dedicated image service and a later Go ingestion pipeline with normalized make, model, and trim extraction.

Marketplace coverage

The JavaScript scraper suite contains integrations for these marketplaces:

• Inventory feeds: Facebook Marketplace, AutoTrader, CarGurus, Cars.com, Craigslist, eBay Motors, and OfferUp
• Auction feeds: Bring a Trailer, Cars & Bids, Copart, and IAAI

Historical note: there is also a dupont/ folder, and the preserved dupont.js documents an AutoTrader-style scraper that lived under that historical folder name.

How the services fit together

System relationships in plain text:

• Users interacted with car-site.
• car-site queried MySQL for accounts, filters, cars, saved cars, and hidden cars.
• car-site also connected to car-socket over Socket.IO for live events and user actions like save and hide.
• car-scraper sent new inventory and auction events into car-socket.
• car-socket deduplicated those events, wrote them into MySQL, and rebroadcast them to the website.
• car-socket also sent some listing images to car-image-server, which compressed and rehosted them.
• car-socket evaluated stored filters and sent Discord webhook alerts when a new listing matched.
• go-car-scraper was a later parallel ingestion path that skipped the socket hub and wrote normalized data directly into PostgreSQL.

Main runtime flow

1. Individual scrapers watched marketplaces and emitted either inventory events (ou-scrape) or auction events (auction-scrape, auction-update, auction-delete-*) into car-socket.
2. car-socket cleaned the payload, applied source-specific dedupe logic, optionally mirrored images through car-image-server, inserted records into MySQL, and rebroadcast new listings to connected clients.
3. The website stored user filters and queried the cars table for paged search results. It also kept track of saved and hidden cars.
4. When filters had webhook metadata attached, car-socket compared each new listing against those filter rules and pushed Discord webhook alerts.
5. The later Go service bypassed the websocket layer for three specific sources and wrote normalized rows directly into PostgreSQL.

Why this code matters historically

A reasonable inference from the codebase is that Car Sniper moved well beyond the toy stage. The strongest signals are:

• A dedicated websocket hub existed because a single scraper process was no longer enough.
• A separate image service existed to normalize or mirror listing images instead of hotlinking them directly.
• The site had user accounts, saved cars, hidden cars, paged filtering, admin pages, and Stripe subscription plumbing.
• The JavaScript suite grew to cover at least 11 named marketplace integrations.
• A second-generation Go scraper was created for structured, large-area scanning across 662 ZIP codes.

None of that proves user counts or revenue, but it does show a system that was used hard enough to justify real operational architecture.

What each project represents

car-site

The customer-facing site. It handled login, registration, sessions, filter-based searching, and the dashboard experience around live listings and saved cars. It also carries surrounding business and operations code from the same company, including subscriptions, downloads, admin pages, and server-management screens.

car-socket

The event switchboard. It accepted scraper traffic, wrote rows into MySQL, rebroadcast live events, tracked saved and hidden cars, and sent Discord webhook alerts for matching filters.

car-scraper

The original sprawling JavaScript scraper lab. Some modules use direct HTTP requests, some use Playwright, some reuse stored session state, and several keep sentHooks.json files to avoid re-emitting the same listing.

car-image-server

A very small service that accepted an external image URL, fetched it, resized and recompressed it, stored it locally, and returned a hosted URL. The archive captures the compression and hosting layer that supported the rest of the stack.

go-car-scraper

The beginning of a faster, more structured rewrite. It focused on Cars.com, CarGurus, and AutoTrader, scanned a broad ZIP-code list, normalized the titles into make, model, and trim fields, and inserted into PostgreSQL with confidence scores.

Archival notes

This repository is best read as a historical system snapshot that documents the architecture, workflows, and service boundaries of the original product.

• The public archive omits disposable runtime artifacts such as proxy credentials, HTML captures, and temp uploads, while preserving the application and service logic.
• The JavaScript scraper suite is documented most accurately as a set of individual long-running workers rather than a single central launcher.
• The repository centers on the application code and service interactions that defined how the system operated.
• Reuse is easiest when approached as reference material for the system design and implementation patterns captured here.

Reading order

If you want the clearest picture of the system, read the subproject READMEs in this order:

1. car-socket
2. car-scraper
3. car-site
4. car-image-server
5. go-car-scraper