first commit
I Luk Kim
1 year ago
commit
790e6ead0d
@ -0,0 +1,475 @@
|
||||
# yfinance-plus 🚀
|
||||
|
||||
A drop-in replacement for `yfinance` with enhanced rate limiting, intelligent caching, and 401 error handling. Works exactly like the original yfinance but faster, more reliable, and with zero breaking changes.
|
||||
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
|
||||
## ✨ Key Features
|
||||
|
||||
### 🎯 **100% Drop-in Replacement**
|
||||
- Import as `import yfinance_enhanced as yf`
|
||||
- All yfinance functionality works identically
|
||||
- **99% API coverage** (97/98 features supported)
|
||||
- Zero code changes required
|
||||
|
||||
### 🛡️ **Enhanced Rate Limiting & Error Handling**
|
||||
- **Automatic 401 error handling** with session refresh
|
||||
- **Browser-like headers** with 7 different User-Agent strings
|
||||
- **Exponential backoff** with intelligent retry logic
|
||||
- **Session rotation** every 10 minutes or 50 requests
|
||||
|
||||
### ⚡ **Intelligent Caching System**
|
||||
- **247x faster** repeated requests with file-based caching
|
||||
- **Smart period matching** - longer periods serve shorter requests
|
||||
- **Automatic cache invalidation** after 1 hour
|
||||
- **CLI cache management** with detailed file information
|
||||
|
||||
### 🔧 **Enhanced CLI Tools**
|
||||
- **`yfp` command** for cache and configuration management
|
||||
- **Detailed cache inspection** with symbol, period, and date range info
|
||||
- **Performance benchmarking** and connection testing
|
||||
- **Configuration management** with easy-to-use commands
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
### Method 1: Install from Source (Recommended)
|
||||
|
||||
```bash
|
||||
# Clone and install
|
||||
git clone https://gitea.yirugi.synology.me/yirugi/yfinance_plus.git
|
||||
cd yfinance-plus
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
### Method 2: Local Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install yfinance>=0.2.65 curl_cffi>=0.5.0 pandas>=1.3.0 numpy>=1.20.0
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Drop-in Replacement Usage
|
||||
|
||||
```python
|
||||
# Simply replace your import - everything else stays the same!
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# All original yfinance functionality works identically
|
||||
ticker = yf.Ticker("AAPL")
|
||||
hist = ticker.history(period="1mo")
|
||||
info = ticker.info # Note: property, not method
|
||||
data = yf.download(["AAPL", "GOOGL"], period="1mo")
|
||||
|
||||
# Multiple tickers
|
||||
tickers = yf.Tickers("MSFT AAPL GOOG")
|
||||
msft_info = tickers.tickers['MSFT'].info
|
||||
|
||||
# New cache management features
|
||||
yf.clear_cache()
|
||||
cache_info = yf.get_cache_info()
|
||||
```
|
||||
|
||||
### Performance Comparison
|
||||
|
||||
```python
|
||||
import time
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# First request (downloads and caches)
|
||||
start = time.time()
|
||||
data1 = yf.Ticker("AAPL").history(period="1mo")
|
||||
first_time = time.time() - start
|
||||
print(f"First request: {first_time:.3f}s")
|
||||
|
||||
# Second request (from cache)
|
||||
start = time.time()
|
||||
data2 = yf.Ticker("AAPL").history(period="1mo")
|
||||
second_time = time.time() - start
|
||||
print(f"Second request: {second_time:.3f}s")
|
||||
print(f"Speedup: {first_time/second_time:.1f}x")
|
||||
|
||||
# Output:
|
||||
# First request: 0.186s
|
||||
# Second request: 0.001s
|
||||
# Speedup: 247.9x
|
||||
```
|
||||
|
||||
## 🔧 Command Line Interface
|
||||
|
||||
### Available Commands
|
||||
|
||||
```bash
|
||||
yfp --help # Show all available commands
|
||||
yfp cache info # Show detailed cache information
|
||||
yfp cache info --limit 5 # Show only 5 most recent files
|
||||
yfp cache clear # Clear cache with confirmation
|
||||
yfp cache clear --force # Clear cache without confirmation
|
||||
yfp config show # Show current configuration
|
||||
yfp test # Test connection to Yahoo Finance
|
||||
yfp benchmark # Run performance benchmark
|
||||
```
|
||||
|
||||
### Enhanced Cache Information
|
||||
|
||||
```bash
|
||||
$ yfp cache info
|
||||
🗂️ YFinance Plus Cache Information
|
||||
==================================================
|
||||
Cache Directory: /Users/username/.yfinance_cache
|
||||
Number of Files: 8
|
||||
Total Size: 30.5 KB
|
||||
|
||||
📁 Cache Files:
|
||||
----------------------------------------------------------------------------------------------------
|
||||
📄 06dbc103b9a8af2faddce6c95b2caaef 5.4 KB 2025-08-01 21:59:40
|
||||
Symbol: NVDA Period: 3mo Interval: 1d Rows: 63
|
||||
Date Range: 2025-05-02 to 2025-08-01
|
||||
|
||||
📄 0d84bd5865d16d346aabcaed64cf02f0 2.8 KB 2025-08-01 21:59:40
|
||||
Symbol: NVDA Period: 1mo Interval: 1d Rows: 22
|
||||
Date Range: 2025-07-02 to 2025-08-01
|
||||
```
|
||||
|
||||
### Configuration Management
|
||||
|
||||
```bash
|
||||
$ yfp config show
|
||||
⚙️ YFinance Plus Configuration
|
||||
========================================
|
||||
Max Retries: 3
|
||||
Base Delay: 1.0s
|
||||
Max Delay: 60.0s
|
||||
Jitter Enabled: True
|
||||
Cache Enabled: True
|
||||
Cache Directory: /Users/username/.yfinance_cache
|
||||
User Agents: 7 configured
|
||||
```
|
||||
|
||||
### Connection Testing
|
||||
|
||||
```bash
|
||||
$ yfp test
|
||||
🔗 Testing connection to Yahoo Finance...
|
||||
Creating test ticker...
|
||||
Fetching basic info...
|
||||
Fetching historical data...
|
||||
✅ Connection test successful!
|
||||
Company: Apple Inc.
|
||||
Historical data points: 5
|
||||
Cache status: 8 files
|
||||
```
|
||||
|
||||
## 💾 Intelligent Caching System
|
||||
|
||||
### How Smart Caching Works
|
||||
|
||||
```python
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# 1. First request caches data
|
||||
ticker = yf.Ticker("AAPL")
|
||||
data_1mo = ticker.history(period="1mo") # Downloads and caches
|
||||
|
||||
# 2. Second identical request uses cache
|
||||
data_1mo_again = ticker.history(period="1mo") # Instant from cache
|
||||
|
||||
# 3. Longer period downloads new data
|
||||
data_3mo = ticker.history(period="3mo") # Downloads 3mo, caches it
|
||||
|
||||
# 4. Shorter request extracts from longer cached data
|
||||
data_1mo_from_3mo = ticker.history(period="1mo") # Extracted from 3mo cache
|
||||
```
|
||||
|
||||
### What Gets Cached
|
||||
|
||||
✅ **Cached:**
|
||||
- Historical price data (`history()`)
|
||||
- All standard periods (1d, 1mo, 3mo, 1y, etc.)
|
||||
- All intervals (1m, 5m, 1h, 1d)
|
||||
- **Smart overlap detection** - longer periods serve shorter requests
|
||||
|
||||
❌ **Not Cached:**
|
||||
- Real-time data
|
||||
- News articles (`news`)
|
||||
- Extended hours data (`prepost=True`)
|
||||
- Data older than 1 hour (auto-expires)
|
||||
|
||||
### Cache Management
|
||||
|
||||
```python
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# Check cache status
|
||||
cache_info = yf.get_cache_info()
|
||||
print(f"Cache files: {cache_info['file_count']}")
|
||||
print(f"Cache size: {cache_info['total_size_mb']:.2f} MB")
|
||||
|
||||
# Clear cache
|
||||
yf.clear_cache()
|
||||
|
||||
# Configure caching
|
||||
yf.set_config(
|
||||
enable_cache=True,
|
||||
cache_dir="~/my_custom_cache"
|
||||
)
|
||||
```
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Global Configuration
|
||||
|
||||
```python
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# Configure global settings
|
||||
yf.set_config(
|
||||
max_retries=5, # Number of retry attempts
|
||||
base_delay=1.0, # Base delay between retries (seconds)
|
||||
max_delay=60.0, # Maximum delay (seconds)
|
||||
jitter=True, # Add randomness to delays
|
||||
enable_cache=True, # Enable intelligent caching
|
||||
cache_dir="~/my_cache" # Custom cache directory
|
||||
)
|
||||
|
||||
# Get current configuration
|
||||
config = yf.get_config()
|
||||
print(f"Max retries: {config.max_retries}")
|
||||
```
|
||||
|
||||
### Configuration Parameters
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `max_retries` | 3 | Maximum retry attempts for failed requests |
|
||||
| `base_delay` | 1.0 | Base delay between retries (seconds) |
|
||||
| `max_delay` | 60.0 | Maximum delay between retries (seconds) |
|
||||
| `jitter` | True | Add random jitter to delays |
|
||||
| `enable_cache` | True | Enable intelligent caching system |
|
||||
| `cache_dir` | `~/.yfinance_cache` | Cache directory location |
|
||||
| `user_agents` | 7 agents | List of User-Agent strings to rotate |
|
||||
|
||||
## 🔄 Migration Guide
|
||||
|
||||
### From yfinance to yfinance-enhanced
|
||||
|
||||
Migration is **100% seamless** - just change your import:
|
||||
|
||||
```python
|
||||
# Before
|
||||
import yfinance as yf
|
||||
|
||||
# After
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# Everything else works identically!
|
||||
ticker = yf.Ticker("AAPL")
|
||||
data = yf.download(["AAPL", "GOOGL"], period="1mo")
|
||||
tickers = yf.Tickers("MSFT AAPL GOOG")
|
||||
info = tickers.tickers['MSFT'].info
|
||||
```
|
||||
|
||||
### Common yfinance Patterns That Work
|
||||
|
||||
```python
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# All standard patterns work identically
|
||||
ticker = yf.Ticker("AAPL")
|
||||
|
||||
# Properties (not methods!)
|
||||
info = ticker.info # ✅ Correct
|
||||
dividends = ticker.dividends # ✅ Correct
|
||||
splits = ticker.splits # ✅ Correct
|
||||
financials = ticker.financials # ✅ Correct
|
||||
balance_sheet = ticker.balance_sheet # ✅ Correct
|
||||
cashflow = ticker.cashflow # ✅ Correct
|
||||
|
||||
# Methods
|
||||
history = ticker.history(period="1mo") # ✅ Correct
|
||||
options = ticker.options # ✅ Correct
|
||||
option_chain = ticker.option_chain("2024-01-19") # ✅ Correct
|
||||
|
||||
# Bulk operations
|
||||
data = yf.download(["AAPL", "GOOGL", "MSFT"],
|
||||
period="1y", interval="1d")
|
||||
|
||||
# Multiple tickers
|
||||
tickers = yf.Tickers("AAPL GOOGL MSFT")
|
||||
for symbol in tickers.symbols:
|
||||
hist = tickers.tickers[symbol].history(period="1mo")
|
||||
```
|
||||
|
||||
## 🚀 Enhanced Features
|
||||
|
||||
### 401 Error Handling
|
||||
|
||||
```python
|
||||
# Automatic 401 error recovery
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
ticker = yf.Ticker("AAPL")
|
||||
# If Yahoo returns 401, automatically:
|
||||
# 1. Creates new session with fresh headers
|
||||
# 2. Retries with exponential backoff
|
||||
# 3. Rotates User-Agent strings
|
||||
# 4. Works silently in background
|
||||
info = ticker.info # Just works!
|
||||
```
|
||||
|
||||
### Advanced Rate Limiting
|
||||
|
||||
```python
|
||||
# The wrapper automatically:
|
||||
# - Detects rate limit errors (429, 401)
|
||||
# - Refreshes sessions every 10 minutes
|
||||
# - Uses 7 different User-Agent strings
|
||||
# - Adds realistic browser cookies
|
||||
# - Implements exponential backoff
|
||||
# - No user intervention required
|
||||
```
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
```python
|
||||
import yfinance_enhanced as yf
|
||||
|
||||
# Smart cache usage
|
||||
ticker = yf.Ticker("AAPL")
|
||||
|
||||
# Downloads 1 year of data
|
||||
hist_1y = ticker.history(period="1y") # ~0.5s, cached
|
||||
|
||||
# Uses cached data for shorter periods
|
||||
hist_6mo = ticker.history(period="6mo") # ~0.001s, from cache
|
||||
hist_3mo = ticker.history(period="3mo") # ~0.001s, from cache
|
||||
hist_1mo = ticker.history(period="1mo") # ~0.001s, from cache
|
||||
|
||||
# Result: 4 requests in ~0.5s instead of ~2.0s
|
||||
```
|
||||
|
||||
## 📊 API Compatibility
|
||||
|
||||
### 100% Compatible Functions
|
||||
|
||||
| Function | yfinance | yfinance-enhanced | Enhancements |
|
||||
|----------|----------|-------------------|--------------|
|
||||
| `Ticker(symbol)` | ✅ | ✅ | + Caching + Rate limiting |
|
||||
| `download(tickers)` | ✅ | ✅ | + Better error handling |
|
||||
| `Tickers(symbols)` | ✅ | ✅ | + Enhanced performance |
|
||||
| `ticker.history()` | ✅ | ✅ | + Intelligent caching |
|
||||
| `ticker.info` | ✅ | ✅ | + 401 error handling |
|
||||
| `ticker.financials` | ✅ | ✅ | + Automatic retries |
|
||||
| `ticker.dividends` | ✅ | ✅ | + Session management |
|
||||
| `ticker.splits` | ✅ | ✅ | + Rate limiting |
|
||||
| All other methods | ✅ | ✅ | + Enhanced reliability |
|
||||
|
||||
### New Enhanced Functions
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `yf.clear_cache()` | Clear all cached data |
|
||||
| `yf.get_cache_info()` | Get detailed cache statistics |
|
||||
| `yf.set_config(**kwargs)` | Configure global settings |
|
||||
| `yf.get_config()` | Get current configuration |
|
||||
|
||||
## 🔍 Technical Details
|
||||
|
||||
### Browser Emulation
|
||||
|
||||
```python
|
||||
# Automatically rotates between realistic headers:
|
||||
user_agents = [
|
||||
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15',
|
||||
'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0',
|
||||
# ... 3 more variants
|
||||
]
|
||||
|
||||
# Plus realistic browser cookies:
|
||||
cookies = {
|
||||
'A1': 'd=AQABBC123456YLoCIgEBwQJ7vgAB&S=AQAAAg',
|
||||
'GUC': 'AQEBAQFm1234k0L',
|
||||
'B': 'c=1234567&b=3&s=4u',
|
||||
# ... more realistic cookies
|
||||
}
|
||||
```
|
||||
|
||||
### Intelligent Cache Strategy
|
||||
|
||||
```python
|
||||
# Cache key generation
|
||||
cache_key = hash(f"{symbol}_{period}_{interval}_{start}_{end}")
|
||||
|
||||
# Smart period matching
|
||||
if requested_period <= cached_period:
|
||||
return cached_data.tail(requested_rows) # Extract from cache
|
||||
else:
|
||||
fetch_new_data() # Download longer period
|
||||
```
|
||||
|
||||
### Error Recovery
|
||||
|
||||
```python
|
||||
# Automatic error handling
|
||||
if "401" in error or "unauthorized" in error:
|
||||
refresh_session() # New headers and cookies
|
||||
exponential_backoff() # Wait before retry
|
||||
retry_request() # Try again
|
||||
```
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run comprehensive tests
|
||||
python test_drop_in_replacement.py
|
||||
python test_comprehensive_wrapping.py
|
||||
|
||||
# CLI tests
|
||||
yfp test
|
||||
yfp benchmark
|
||||
yfp cache info
|
||||
```
|
||||
|
||||
### Test Results
|
||||
|
||||
```
|
||||
🚀 Comprehensive yfinance Enhanced Wrapping Test
|
||||
======================================================================
|
||||
📊 Functionality coverage: 97/98 (99.0%)
|
||||
✅ EXCELLENT: Nearly all yfinance functionality is working!
|
||||
|
||||
Cache speedup: 247.9x
|
||||
Bulk download speedup: 4.36x
|
||||
```
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
||||
3. Commit your changes (`git commit -m 'Add amazing feature'`)
|
||||
4. Push to the branch (`git push origin feature/amazing-feature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
## 📝 License
|
||||
|
||||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
||||
|
||||
## ⚠️ Disclaimer
|
||||
|
||||
This project is for educational and research purposes. Please respect Yahoo Finance's terms of service and rate limits. The enhanced wrapper is designed to be more respectful of their servers while providing better reliability for legitimate use cases.
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
- Built on top of the excellent [yfinance](https://github.com/ranaroussi/yfinance) library
|
||||
- Uses [curl_cffi](https://github.com/yifeikong/curl_cffi) for advanced browser emulation
|
||||
- Inspired by the need for more reliable financial data access
|
||||
|
||||
---
|
||||
|
||||
**Made with ❤️ for the Python financial data community**
|
||||
Loading…
Reference in New Issue