Google Jobs anti-bot wall fired on every query from Korea, flooding the
summary with errors. Now an engine that raises EngineBlockedError is
skipped for the rest of the run (one attempt total), and blocks are
counted separately instead of polluting the error list.
ATSU school pages were classified ACTIVE because 'APPLY' nav buttons
matched apply keywords. Now:
- job_terms (position/hiring/vacancy/...) must be present for ACTIVE
unless an official ATS platform is detected
- program_terms (residency/admissions/curriculum/tuition/...) classify
school pages as FILTERED (not a job posting)
- added tests for both cases; cleaned the two false-positive leads
- sites/proactive.yaml: search plan (50 states, terms, OCONUS, budget,
denylist for known false positives, verification rules)
- plan: query matrix generator with per-day rotation and budget caps
- engines: DuckDuckGo HTML primary + Google Jobs secondary with anti-bot
fallback
- verifier: pure-heuristic ATS detection and ACTIVE/VERIFY/CLOSED/STALE
classification (no AI at runtime)
- ledger: ProactiveLead/ProactiveRun with lifecycle transitions
(NEW → STILL_OPEN, inactive → REOPENED, VERIFY → ACTIVE = NEW)
- CLI: gimme-job proactive [--dry-run --limit --max-verify]
- BrowserConfig gains proxies list (+ legacy single proxy kept for compat)
- BrowserManager accepts proxy URLs (http/socks5 with optional auth)
- orchestrator retries scrape with next proxy on navigation error or
0-result when site has known data
- ihs: route through US proxy pool (geo-blocked from Korea)
- add README with setup and usage
Tribal Health (tribalhealth.com) is a clinician staffing agency placing providers at tribal/IHS sites — the one non-subscribed source carrying net-new tribal dental roles not on ihs.gov/usajobs. Elementor WordPress loop; collect-all + keyword-filter (mirrors nativehealth). collect_cards walks all numbered board pages internally so sparse roles on later pages aren't dropped by the orchestrator's empty-page break; paginate() returns False.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tracing was accumulating 45GB+ of data (traces) + 5.6GB (dom) + 1.3GB
(screenshots) on every run. These artifacts are only useful during
development; remove them from the production run pipeline entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Apply links (Indeed, Glassdoor, Monster, etc.) are pre-rendered inside
a <template> tag within each card's share_el. <template> content is
inert in the rendered DOM — a regular querySelectorAll from outside
returns nothing, which is why the previous fix produced empty URLs.
Iterate share_el's <template> elements and search their .content for
links matching utm_campaign=google_jobs_apply. Verified on the live
Google Jobs page that each card's template contains the correct
external apply URL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add exclude_company_keywords post-filter (case-insensitive substring
match against company field). Configure googlejobs.yaml to exclude
"U.S. Navy" / "US Navy" — these recurring military recruitment ads
aren't relevant orthodontist openings.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The htidocid-based Google Jobs deep-links require the job to be in the
current search context (location, session) to open the detail panel.
When clicked from Telegram in a different context, the link only opens
the search list page without the specific job detail.
Each card's share_el ancestor has external apply URLs (Indeed,
Glassdoor, Monster, BeBee, AAO Career Center, etc.) pre-loaded in the
DOM. These are direct, stable links to the source job posting that work
regardless of session or location.
Switch jobUrl to use the first external (non-google.com) apply URL
within the card's share_el.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous shareUrl-based URLs included session-tied tokens (shmd, shmds,
shem) that can expire over time, causing the job detail panel to not
open when the link is clicked later from Telegram.
Switch to a minimal Google Jobs URL using only the htidocid and the new
SPA fragment format:
https://www.google.com/search?q=<q>&udm=8#vhid=vt%3D20/docid%3D<id>&vssid=jobs-detail-viewer
Verified via Playwright: this format reliably opens the specific job
detail panel without any session-tied parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use page.click(selector) instead of ElementHandle.click() so Playwright
re-queries the element at click time after AJAX page refresh.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- gilariver: parse employment type from subtitle "Location | Category | Active - Full Time"
- usajobs: add salary_text and employment_type selectors to YAML
- hrsa: filter <br> elements from td children to fix company/employment_type index offset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ihs: Indian Health Service Dentistry (table, keyword filter)
- gilariver: Gila River Health Care via Infor CloudSuite (Angular, click pagination)
- nativehealth: NATIVE HEALTH via SmartRecruiters (AJAX Show More, keyword filter)
- srpmic: Salt River Pima-Maricopa via GovernmentJobs company page (URL keyword search)
- bfrench: Consulting BFrench via JazzHR (simple table, keyword filter)
- govtjobs: GovernmentJobs.com main search (URL keyword search, URL pagination)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New site adapters:
- usajobs: multi-keyword sweep (dentist/orthodontics/orthodontist), URL pagination
- docshealth, southernortho: Paylocity platform, keyword in URL
- pdshealth, saltdental: iCIMS Angular platform, URL pagination
- hospitaljobsonline: Load More button, relative URL fix
- aaoinfo: AAO Career Center, click-based AJAX pagination
- aroragroup: Load More + post-scrape keyword filter
Core improvements:
- Early stop pagination: stops when all fingerprints on a page are already in DB
- multi_keyword_mode: separate — one scrape per keyword, cross-sweep dedup
- gimme-job list command to view collected postings
- Stored column in status command
- Telegram notification (replacing KakaoTalk)
- DB repo: count_by_site(), get_existing_fingerprints()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
I Luk Kim