Tracing was accumulating 45GB+ of data (traces) + 5.6GB (dom) + 1.3GB
(screenshots) on every run. These artifacts are only useful during
development; remove them from the production run pipeline entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Apply links (Indeed, Glassdoor, Monster, etc.) are pre-rendered inside
a <template> tag within each card's share_el. <template> content is
inert in the rendered DOM — a regular querySelectorAll from outside
returns nothing, which is why the previous fix produced empty URLs.
Iterate share_el's <template> elements and search their .content for
links matching utm_campaign=google_jobs_apply. Verified on the live
Google Jobs page that each card's template contains the correct
external apply URL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add exclude_company_keywords post-filter (case-insensitive substring
match against company field). Configure googlejobs.yaml to exclude
"U.S. Navy" / "US Navy" — these recurring military recruitment ads
aren't relevant orthodontist openings.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The htidocid-based Google Jobs deep-links require the job to be in the
current search context (location, session) to open the detail panel.
When clicked from Telegram in a different context, the link only opens
the search list page without the specific job detail.
Each card's share_el ancestor has external apply URLs (Indeed,
Glassdoor, Monster, BeBee, AAO Career Center, etc.) pre-loaded in the
DOM. These are direct, stable links to the source job posting that work
regardless of session or location.
Switch jobUrl to use the first external (non-google.com) apply URL
within the card's share_el.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous shareUrl-based URLs included session-tied tokens (shmd, shmds,
shem) that can expire over time, causing the job detail panel to not
open when the link is clicked later from Telegram.
Switch to a minimal Google Jobs URL using only the htidocid and the new
SPA fragment format:
https://www.google.com/search?q=<q>&udm=8#vhid=vt%3D20/docid%3D<id>&vssid=jobs-detail-viewer
Verified via Playwright: this format reliably opens the specific job
detail panel without any session-tied parameters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use page.click(selector) instead of ElementHandle.click() so Playwright
re-queries the element at click time after AJAX page refresh.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- gilariver: parse employment type from subtitle "Location | Category | Active - Full Time"
- usajobs: add salary_text and employment_type selectors to YAML
- hrsa: filter <br> elements from td children to fix company/employment_type index offset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ihs: Indian Health Service Dentistry (table, keyword filter)
- gilariver: Gila River Health Care via Infor CloudSuite (Angular, click pagination)
- nativehealth: NATIVE HEALTH via SmartRecruiters (AJAX Show More, keyword filter)
- srpmic: Salt River Pima-Maricopa via GovernmentJobs company page (URL keyword search)
- bfrench: Consulting BFrench via JazzHR (simple table, keyword filter)
- govtjobs: GovernmentJobs.com main search (URL keyword search, URL pagination)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New site adapters:
- usajobs: multi-keyword sweep (dentist/orthodontics/orthodontist), URL pagination
- docshealth, southernortho: Paylocity platform, keyword in URL
- pdshealth, saltdental: iCIMS Angular platform, URL pagination
- hospitaljobsonline: Load More button, relative URL fix
- aaoinfo: AAO Career Center, click-based AJAX pagination
- aroragroup: Load More + post-scrape keyword filter
Core improvements:
- Early stop pagination: stops when all fingerprints on a page are already in DB
- multi_keyword_mode: separate — one scrape per keyword, cross-sweep dedup
- gimme-job list command to view collected postings
- Stored column in status command
- Telegram notification (replacing KakaoTalk)
- DB repo: count_by_site(), get_existing_fingerprints()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
I Luk Kim