yirugi
/
stock-oracle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
110 Commits
1 Branch
0 Tags
2.4 MiB
main
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'd513f64922'
${ noResults }
Commit Graph

64 Commits (d513f6492223a22484668cd7f77011d44562a0d5)

Author SHA1 Message Date
I Luk Kim e2c242c7cc fix(sec): 대형 exhibit 파일로 인한 서버 메모리 고갈 방지 
- fetch_text에 max_bytes 파라미터 추가: Content-Length 헤더로 다운로드 전 사전 reject,
  헤더 없으면 content.read(max_bytes+1)로 제한적 읽기
- ValueError는 즉시 raise (retry 없음 — 크기는 재시도해도 안 줄어듦)
- in-memory 캐시(_text_cache) 1MB 가드: 대형 응답은 디스크 캐시에만 저장
- MAX_EXHIBIT_SIZE 1MB → 5MB, fetch_text(max_bytes=...) 호출로 다운로드 전 체크
- exhibit deadline 30초 → 15초 (서비스 + 엔드포인트 + bulk)
- 신규 테스트 5개: Content-Length 사전 거부, body 제한 읽기, 메모리 캐시 가드,
  소형 캐시 유지, ValueError no-retry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 89113d45ab fix(sec): _deadline 레이스 컨디션 수정, bulk exhibit 동시성 제한 추가 
- SECHttpClient._deadline(인스턴스 변수) → contextvars.ContextVar로 교체
  asyncio task별 독립 데드라인으로 싱글턴 공유로 인한 레이스 컨디션 해결
- bulk exhibit에 Semaphore(4) + 전체 300s 타임아웃 추가
  동시 50개 코루틴이 Semaphore(2)를 무제한 점유하던 문제 해결

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim af88f89f69 docs: 앱 내부 문서(루트 HTML, OpenAPI)에 index 엔드포인트 추가 
- main.py 루트 HTML: Stock Market Data 섹션에 /stocks/index/{index_name} 항목 및 예시 추가
- stocks 태그 설명 업데이트 (index constituents 포함)
- get_index_constituents: summary/response_description/docstring 상세화

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim cbc0f93123 feat(stocks): Wikipedia 인덱스 구성 종목 조회 API 추가 
GET /stocks/index/{index_name} 엔드포인트 추가.
sp500/nasdaq100 구성 종목을 Wikipedia에서 실시간 파싱하여 반환.
24시간 Redis 캐시 및 30초 타임아웃 적용.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 1e9432a6e3 fix: exhibit/bulk에서 concurrent DB 세션 충돌 수정 
asyncio.gather로 동시에 실행되는 exhibit 항목들이 단일 DB 세션을
공유하면서 "concurrent operations are not permitted" 에러 발생.

각 _fetch_one 태스크가 AsyncSessionLocal()로 독립 세션 사용하도록 수정.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 45d832ba5c perf: Phase 1-3 Stock Oracle API 성능 개선 
Phase 1A - yfinance hang 제거
- _run_with_timeout() 헬퍼 추가 (asyncio.wait_for 래퍼)
- run_in_executor 6곳에 timeout 적용: history(30s), .info(20s), bulk download(60s)

Phase 1B - SEC Filing deadline 설정
- index_filings: 60s deadline + try/finally
- get_filing_documents: 30s deadline (중첩 호출 시 기존 deadline 유지)
- get_exhibit_content: 30s deadline + try/finally

Phase 1C - _get_ticker_max_range async 전환
- sync → async def + run_in_executor + wait_for(20s)
- get_or_create_company_data에서 period=="max" 사전 체크 → await 직접 호출

Phase 1D - Endpoint 레벨 timeout
- POST /price/data/bulk: 300s → 504
- POST /financial/data/bulk: 300s → 504
- GET /filings/search/{ticker}: 120s → 504
- GET /filings/documents/{accession}: 30s → 504
- GET /filings/exhibit/{accession}: 30s → 504

Phase 2 - Filing 캐시 추가
- GET /filings/documents: @with_cache(ttl=86400)
- GET /filings/exhibit: @with_cache(ttl=86400)

Phase 3A - POST /filings/search/bulk 추가
- BulkFilingSearchRequest/Item/Response 스키마
- search_filings_bulk(): 배치 DB 조회 → 미인덱싱 ticker 병렬 인덱싱(Semaphore 4)
- @with_cache(ttl=3600), 600s endpoint timeout

Phase 3B - POST /filings/exhibit/bulk 추가
- BulkExhibitRequest/Item/Response 스키마
- asyncio.gather + 개별 30s timeout, 최대 50건

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim b88835cded feat(screener): add stock screener API with yfinance EquityQuery 
Implements GET /api/v1/screener/stocks and GET /api/v1/screener/fields
for condition-based stock filtering without manual web searches.

- app/schemas/screener.py: ScreenerStockItem + ScreenerResponse Pydantic models
- app/services/screener_service.py: ScreenerService wrapping yfinance screen()
  via run_in_executor; exchange mapping (NYSE→NYQ, NASDAQ→NMS/NGM/NCM, etc.);
  btwn/gt/lt/is-in/eq EquityQuery builder; post-filter for ETF/FUND exclusion
- app/api/v1/endpoints/screener.py: /stocks (with_cache TTL=300) + /fields metadata
- app/api/v1/api.py: register screener router at prefix /screener
- app/main.py: add screener OpenAPI tag and HTML doc section with examples

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim d82ba624e2 chore: remove SQLite entirely, PostgreSQL only 
SQLite was never used in production (Docker Compose always sets
DATABASE_URL to postgresql+asyncpg://...) but the fallback kept
creating confusing stock_oracle.db files locally.

- config.py: default DATABASE_URL fallback → PostgreSQL (localhost:15433)
- database.py: remove _is_sqlite conditional branch and NullPool import
- error_logs.py: replace strftime/dialect-check with pg to_char()
- request_logs.py: replace strftime with pg to_char()
- feature_builder.py: remove _ensure_utc() helper and all call sites
- overlay_pipeline.py: remove naive-datetime workaround for is_stale()
- requirements-api.txt / requirements-test.txt: drop aiosqlite
- stock_oracle.db: deleted

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 76e154eccd feat(filings): add accepted_at field to filing search response 
Expose SEC ACCEPTANCE-DATETIME so downstream consumers (filing_poller)
can populate Document.accepted_at_utc and Event.filed_at_utc.

- SECFiling model: add nullable accepted_at TIMESTAMP column
- main.py startup: ALTER TABLE migration for existing SQLite DBs
- sec_filings_service: extract acceptanceDateTime from SEC JSON, store on
  INSERT and force_refresh UPDATE
- FilingSummary schema: add Optional[str] accepted_at field
- filings endpoint: map accepted_at as ISO 8601 string in response
- CHANGELOG: v3.0.1 entry

Existing rows have accepted_at=NULL; backfill with force_refresh=true.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 29fc870b37 Add Phase 5 Attention Overlay API and fix Redis port mismatch 
- 12 REST endpoints: overlay score, bulk, top-movers, headlines, youtube, wiki, crowding, trends, history, admin (health/trigger/job-log)
- 10 services: entity_resolver, yahoo_rss_adapter, wikimedia_adapter, youtube_adapter, google_trends_adapter, finra_overlay_loader, feature_builder, overlay_scorer, overlay_pipeline, scheduler
- 10 DB tables across overlay_registry, overlay_raw_event, overlay_feature models
- APScheduler: collect @ 23:30 UTC + feature build @ 01:30 UTC weekdays
- Fix Redis port mismatch: config default 16379 → 16380 to match docker-compose external port
- 64 overlay tests covering cache utils, Redis config, all 12 endpoints, route ordering

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 9e2a8aba47 Codebase improvements: caching decorator, Pydantic v2, DB indexes, connection pooling, Alembic 
- Add @with_cache() decorator to eliminate ~15-line caching boilerplate per endpoint
- Apply decorator to 8 existing endpoints (stocks, alpaca, finra) and add caching
  to 6 previously uncached endpoints (news, etf, filings) with appropriate TTLs
- Migrate all @validator to @field_validator (Pydantic v2), deduplicate validation
  logic into shared functions in validators.py
- Fix datetime.utcnow() → datetime.now(timezone.utc), remove unused uuid import
- Convert ErrorLogResponse class Config → model_config = ConfigDict(...)
- Add health check exception logging instead of silent pass
- Add data_source indexes to PriceData and FinancialData tables
- Initialize Alembic with async engine configuration
- Add persistent HTTP sessions for SEC client (aiohttp) and FRED proxy (httpx)
- Add response_model schemas for Alpaca bars/intraday and news-only/social-only

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim bf6932ca2f Add Alpaca-specific DB table and Redis caching for Alpaca/FINRA endpoints 
Separate Alpaca price data into dedicated AlpacaPriceData table to avoid
UniqueConstraint('ticker', 'date') conflicts with Yahoo Finance PriceData.
Add Redis caching (build_cache_key/get_cached_response/set_cached_response)
to 3 Alpaca endpoints and 2 FINRA query endpoints with appropriate TTLs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim d0f1a9a7d0 Add SEC filings indexing, search, and exhibit extraction (8-K, 6-K, 20-F, 40-F) 
Extract shared SECHttpClient from ETF fetcher (retry, backoff, cache, throttle)
and apply it to both ETF and core SEC services, fixing missing rate limiting.
Add SECFiling DB model, Pydantic schemas, SECFilingsService with auto-indexing,
and REST endpoints at /filings/search, /filings/documents, /filings/exhibit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago
I Luk Kim 355aeb19d3 Initial commit with full project improvements 
Security: config-based CORS, default secret warnings, sort_by validation
Error handling: debug logging in cache silent failures
Architecture: shared resolve_time_parameters, deduplicated logger init, unified route structure
Database: conditional SQLite/PostgreSQL engine, in-memory test DB, dialect-aware date formatting, optimized stats query
Docker: .dockerignore, pinned yfinance_plus commit
Dependencies: removed duplicates, added version upper bounds, removed unused axios
Frontend: custom _document/_error pages, adminApi client, Layout standardization, ESLint version update

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 5 months ago