Intro

Overview

Empathy

Observations

Friction

Synthesis

Scope

Personas

Strategy

5 Ws

Interfaces

Systems

Database

API Reference

AI System DesignAutonomous Job Intelligence⏱️12 Min Read

Scout Engine

An autonomous recruiting agent that crawls ATS platforms, runs dual-stage LLM evaluation pipelines on Groq Cloud, and morphs resumes dynamically to eliminate job application friction.

Explore Scout Repository Launch Live Preview

Role:Design & Development

Timeline:12 Weeks

Industry:HR Tech (Developer Tool)

Scroll to start▼

01. Executive Summary

Project Overview

Modern job boards force candidates to parse through bloated formatting and irrelevant listings just to find basic details like salary caps, tech requirements, or visa sponsorship. Scout is a self-hosted command center designed to automate this entire research and preparation phase. By combining scheduler-driven scrapers, a dual-stage Groq LLM filter, and a dynamic PDF resume generator, Scout transforms the application process from manual grinding into surgical precision.

ghost_sweep_daemon.sh

STATUS: ACTIVE

Click 'Trigger Ghost Sweep' below to simulate the AI parser daemon...

Next.js 16 Cron Daemon

02. Empathy & Context

The Job Hunting Grind

Job hunting is notoriously exhausting. Engineers spend hours navigating multiple job boards (Google Jobs, LinkedIn, RemoteOK), filtering out roles that don't fit their salary bands or visa requirements, and tracking everything in massive, manually maintained spreadsheets.

Worse yet, to satisfy Applicant Tracking Systems (ATS), candidates must meticulously alter their resume bullet points for every single application. High-intent outreach also requires writing custom cold emails and LinkedIn messages to hiring managers.

I set out to build an autonomous agent that could handle the heavy lifting of research and preparation. It sweeps job boards, classifies matches against user preferences, extracts skill gaps, and prepares outreach hooks and customized PDF resumes on demand.

03. Field Observations

Observations

Manual parsing is a major time sink

Candidates spend upwards of 75% of their search time parsing wall-of-text descriptions simply checking for blockers like missing visa sponsorship, remote location mismatch, or low salary ranges.

Resume tailoring is tedious and slow

Tailoring bullet points to match the target stack is vital for ATS passing, but doing it manually per role is highly repetitive and slows down application rates to just 2–3 submittals per day.

04. Friction & Blockers

Friction & Blockers

Cognitive Fatigue

Visual clutter and repetitiveness across popular job boards dilute high-quality leads, leading to application burnout and reduced output quality.

Lack of Preparation Context

Candidates apply blindly without knowing where their skills gap lies relative to the job requirements, creating interview friction and high rejection rates.

05. Research Synthesis

Research Synthesis

Automate the filtration pipeline

By running scheduled classifiers on raw text datasets in the background, we can drop the visual tax of search entirely. Only matched entries pass to the candidate.

Optimistic, non-blocking UI queueing

Background scraping shouldn't cause layout shifting. Real-time updates must queue incoming results in a notification drawer, letting the user merge them on command.

06. Project Scope

Scope of Work

Scout aims to create a highly focused, self-hosted job hunting dashboard. It abstracts all parsing, scraping, scoring, and customization logistics away from the candidate.

Rather than being a social network or generic spreadsheet tracker, Scout operates as a background recruitment agent. It monitors ATS feeds, classifies suitability on Groq Cloud, dynamically tweaks PDF layout engines, and provides tailored outreach.

07. Target Personas

Research & User Personas

To structure the user experience, I mapped out the primary user segments and their critical needs:

👨‍💻

Aarav Mehta

Senior Front-End Dev

Focus: Hybrid Next.js / React

“I want a central dashboard that instantly filters out companies that don't sponsor visas or meet my salary requirements so I can focus only on relevant senior roles.”

Goals

Filter out non-visa sponsoring boards immediately and generate personalized cold emails for hiring managers.

Frustrations

Spending hours parsing job descriptions to check basic salary parameters. Writing custom pitch letters repeatedly.

👩‍💻

Rahul Jain

Junior Full-Stack Developer

Focus: Entry-Level Postings

“I need to know exactly which requirements I miss for a job description, and get advice on how to address those missing pieces during interviews.”

Goals

Find entry-level roles, track skill gaps, and get objection-handling cards prepared for upcoming calls.

Frustrations

Applying to positions and getting filtered out by ATS algorithms because the resume bullet points weren't customized.

08. Design Strategy

Core Objectives & Strategy

To make the job application process seamless, Scout is designed around four key product objectives:

1. Dual-Pipeline Staging Dashboard

Separates incoming scraped jobs into a low-intent “Casual Hunt” queue for review, and a high-intent “Serious Mode” pipeline for active applications.

2. Realtime Staging Queue Pill

Filters incoming background scrapes through a floating indicator, buffering additions via Supabase Realtime to prevent jarring layout jumps.

3. Sniper Resume Morpher

Tailors a custom PDF resume dynamically using `@react-pdf/renderer` by re-ordering project experience bullet points based on the target job stack.

4. Obsidian Mint Surface Ramp

Built on a premium monochromatic dark container system with gradual surface levels, using an Emerald Mint signature accent strictly for match scores.

09. Strategic Alignment

The 5 Ws Alignment

What?

An autonomous, self-hosted job intelligence dashboard that crawls job boards, runs dual-stage LLM classifiers, and prepares custom outreach templates and resumes.

Why?

To bypass the hours spent parsing irrelevant descriptions, checking sponsorships, writing outreach scripts, and tailoring resumes by hand.

When?

Runs automatically on a daily scheduler (CRON trigger) in the background, or dynamically on demand when pasting a manual listing URL.

Who?

Software developers and system designers who want to streamline their job application process and avoid burnout.

Where?

Self-hosted by the user, running client-side on Next.js 16 and syncing data over Supabase.

How?

Leveraging Next.js app routes, Firecrawl scraping APIs, Groq Cloud inference triggers (Llama 8B and 70B models), and `@react-pdf/renderer` layouts.

10. System Interfaces

Walkthrough: The Interface Command

Screen 1

Obsidian Pipeline

Kanban-style application tracker with four stages — Serious Pipeline, Applied, Interviewing, and Archived. Drag cards across columns to track live application progress.

Stage-Based Kanban Layout

Separates applications into four stages — Serious, Applied, Interviewing, and Archived — so you always know where each role stands without scanning spreadsheets.

Groq Match Score Badges

Every card shows a Groq-computed suitability score (e.g. 92%, 70%) so you prioritize high-intent applications and skip low-confidence listings at a glance.

Screen 2

Command Center — Scout Configuration

Configure your target roles, experience level, and locations. The Sweep Activity Log on the right shows every autonomous crawl run — detections found and listings saved.

Scout Command Center — Ghost Scouter Configuration and Sweep Activity Log

Preference-Synced Targeting

Set target roles, experience bands, and location preferences once. Scout syncs these parameters as the filter blueprint for every automated ghost sweep.

Live Sweep Activity Log

A timestamped activity panel records every background cron run — detections found and net listings added — giving full transparency into the crawler's output.

Screen 3

Command Center — Identity Profile

Your professional identity card: salary band, work type, tech arsenal by category, and a full career timeline — all used as the source of truth for resume morphing and scoring.

Tech Arsenal Mapping

Categorizes your skills across AI/ML, tools, UI frameworks, databases, and languages. These tags feed directly into the LLM scoring and resume bullet-point reordering engine.

Search Logic Anchors

Salary band, work type, and company size preferences are persisted as Supabase user profile fields — queried by every sweep to eliminate mismatched listings before they surface.

Screen 4

Casual Hunt — Job Grid

Auto-swept listings displayed in a filterable card grid. Role chips, location, and ATS source are shown on every card alongside a Groq match percentage so you can triage quickly.

Scout Casual Hunt — auto-swept job listing grid with match scores

Filter Rail + ATS Source Tags

A persistent filter chip row lets you narrow by role type, location, and work format without page reloads. ATS source (e.g. Lever, Greenhouse) is labeled on every card.

One-Click Promotion to Pipeline

Interesting listings can be promoted from Casual Hunt to the Obsidian Pipeline in a single action, triggering full 70B distillation and outreach generation.

Screen 5

Live ATS Preview — Sniper Resume Morpher

Split-pane view: job details, skill gap analysis, and AI-generated outreach hook on the left — a live PDF preview of the dynamically tailored resume on the right, ready to download.

Scout Live ATS Preview — tailored resume PDF alongside job detail and skill gap analysis

Dynamic Bullet-Point Reordering

The 70B model re-ranks resume bullet points to surface the most relevant experience first. The PDF preview updates live so you see the exact ATS-ready output before downloading.

Skill Gap + Hook Generation

Identifies missing requirements (e.g. Senior UX/UI experience) and generates an AI outreach hook that reframes adjacent skills — turning gaps into confident talking points.

11. Technical Infrastructure

Engineering for Blistering Speed

Dual-Stage Groq Cloud Pipeline

Executing heavy 70B parameter LLM distillation directly on scraped job postings is cost-prohibitive. Scout implements a two-stage classification strategy.

A lightweight model (Llama-3.1-8B-instant) filters out mismatches in milliseconds. Only passing roles are sent to the larger model (Llama-3.3-70B-versatile) for full context mapping and outreach generation.

scoutRouter.ts

AI Endpoint

const classifyJob = async(text) => {
  const isMatch = await groq.run("llama3-8b", filterRules, text);
  if (!isMatch) return null;
  return await groq.run("llama3-70b", distillTemplate, text);
};

Scheduled Ghost Sweeps

To keep the application queues filled, background workers operate on Cron schedules. The worker utilizes Serper.dev and Firecrawl to scan directories and ATS boards based on user profile preferences.

Cryptographic deduplication runs locally on database ingestion. The system logs sweep diagnostics inside `ghost_sweeps` tables to prevent duplicate API cycles and limit rates.

Cron Sweeper Scriptcron_sweep.ts

export async function GET(req) {
  verifyCronSecret(req);
  const rawJobs = await triggerSweeper(userProfile);
  return syncWithSupabase(rawJobs);
}

12. Database Schema

Relational Architecture

Table: `jobs`

Column	Type	Description
id	uuid	PK, auto-generated
status	text	casual or serious
match_score	int	0-100 score rating
match_explanation	text	one-sentence AI rationale
outreach_hooks	jsonb	cached copy by channel
objection_strategies	text[]	cached objection handling

Table: `user_profile`

Column	Type	Description
id	uuid	PK, auto-generated
salary_min	int	minimum salary threshold
skills	jsonb	list of technologies & level
preferred_roles	text[]	preferred roles mappings

13. API Endpoints

API Reference

Endpoint	Method	Engine	Description
/api/scout	POST	Groq + Firecrawl	Scrape URL → AI distill → save job
/api/scout/distill	POST	Groq + Firecrawl	Re-evaluate existing job by ID
/api/job/analyze-gaps	POST	Groq Llama 8B	Perform skill gap assessment
/api/job/generate-hook	POST	Groq Llama 8B	Create channel-specific pitch outreach copy
/api/cron/sweep	GET/POST	Groq + Serper + Resend	Trigger scheduled crawler checks

Thanks for reading!

Let's collaborate to design and engineer premium interface systems.

UX Case Study

BreezePOS

A touchscreen-optimized countertop register and real-time inventory engine designed for convenience store operators.

Read Case Study →

Portfolio Index

All Case Studies

Browse the full gallery gallery of user experience research prototypes, dashboard engines, and design tools.

View Selected Work →