diff --git a/.gitignore b/.gitignore index 5ed07d3..42717d3 100644 --- a/.gitignore +++ b/.gitignore @@ -140,6 +140,12 @@ cython_debug/ # PyCharm .idea/ +# VS Code +.vscode/ + +# GitHub +.github/ + # Custom files run.py chromedriver* diff --git a/Docs/CODEBASE_ANALYSIS.md b/Docs/CODEBASE_ANALYSIS.md new file mode 100644 index 0000000..eea943e --- /dev/null +++ b/Docs/CODEBASE_ANALYSIS.md @@ -0,0 +1,359 @@ +# Twitch Channel Points Miner v2 - Codebase Analysis + +**Date:** February 17, 2026 +**Purpose:** Understanding the existing codebase before converting to a Twitch Drops-focused bot +**Original Project:** https://github.com/rdavydov/Twitch-Channel-Points-Miner-v2 + +--- + +## 📋 Executive Summary + +This bot is a Python-based application designed to automatically watch Twitch streams to earn channel points. It includes existing drops functionality that we can leverage and expand upon for our Twitch Drops-focused implementation. + +**Key Finding:** The bot already has a solid drops infrastructure including: +- Drop claiming mechanisms +- Campaign tracking +- Inventory synchronization +- Drop progress monitoring + +--- + +## 🏗️ Architecture Overview + +### Core Components + +#### 1. **TwitchChannelPointsMiner** (Main Entry Point) +- **Location:** `TwitchChannelPointsMiner/TwitchChannelPointsMiner.py` +- **Purpose:** Main orchestrator class that manages the entire bot lifecycle +- **Key Features:** + - Handles authentication via `Twitch` class + - Manages multiple streamers simultaneously + - Coordinates WebSocket connections for real-time updates + - Implements priority system for streamer selection + - Spawns threads for monitoring and synchronization + +**Important Parameters:** +```python +- username: Twitch account username +- password: Optional password (prompts if not provided) +- claim_drops_startup: Auto-claim all drops on startup (Boolean) +- priority: List defining watch priority [STREAK, DROPS, ORDER, etc.] +- enable_analytics: Track statistics (Boolean) +- logger_settings: Logging configuration +- streamer_settings: Default settings for all streamers +``` + +#### 2. **Twitch Class** (API Handler) +- **Location:** `TwitchChannelPointsMiner/classes/Twitch.py` +- **Purpose:** Handles all Twitch API interactions via GraphQL +- **Key Methods for Drops:** + - `claim_drop(drop)` - Claims a single drop + - `claim_all_drops_from_inventory()` - Claims all available drops from inventory + - `sync_campaigns(streamers, chunk_size)` - Synchronizes campaign data every 30 minutes + - `__get_inventory()` - Fetches user's drops inventory + - `__get_drops_dashboard(status)` - Gets active/expired campaigns + - `__get_campaigns_details(campaigns)` - Gets detailed campaign information + - `__sync_campaigns(campaigns)` - Updates campaign progress + +#### 3. **Drop Entity** +- **Location:** `TwitchChannelPointsMiner/classes/entities/Drop.py` +- **Purpose:** Represents a single drop reward +- **Properties:** + ```python + - id: Unique drop identifier + - name: Drop name + - benefit: Reward description + - minutes_required: Watch time needed + - current_minutes_watched: Progress + - percentage_progress: Completion percentage + - is_claimable: Whether drop can be claimed + - is_claimed: Whether drop has been claimed + - drop_instance_id: Instance identifier for claiming + ``` + +#### 4. **Campaign Entity** +- **Location:** `TwitchChannelPointsMiner/classes/entities/Campaign.py` +- **Purpose:** Represents a Twitch Drops campaign +- **Properties:** + ```python + - id: Campaign identifier + - game: Associated game information + - name: Campaign name + - status: Campaign status (ACTIVE, EXPIRED, etc.) + - drops: List of Drop objects + - channels: Eligible channel IDs + - start_at/end_at: Campaign timeframe + ``` +- **Key Methods:** + - `clear_drops()` - Removes expired/claimed drops + - `sync_drops(drops, callback)` - Updates drop progress + +#### 5. **Priority System** +- **Location:** `TwitchChannelPointsMiner/classes/Settings.py` +- **Enum Values:** + ```python + Priority.STREAK # Watch streak priority + Priority.DROPS # Drops collection priority + Priority.ORDER # Custom ordering + Priority.SUBSCRIBED # Subscribed channels first + Priority.POINTS_ASCENDING / POINTS_DESCENDING + ``` + +--- + +## 🔄 Workflow & Data Flow + +### Current Bot Flow: + +1. **Initialization** + - Authenticate with Twitch + - Load streamer list + - Configure logger and settings + - Check for updates + +2. **Startup Phase** + - If `claim_drops_startup=True`, claim all available drops + - Initialize WebSocket pool for real-time updates + - Start analytics server (if enabled) + +3. **Main Loop** + - **Streamer Selection:** Choose streamer based on priority system + - `Priority.STREAK`: Prioritize maintaining watch streaks + - `Priority.DROPS`: Prioritize streamers with active drop campaigns + - `Priority.ORDER`: Use custom ordering + + - **Watching:** Join stream via WebSocket, simulate viewing + + - **Background Threads:** + - `minute_watcher_thread`: Tracks watch time, handles bonuses + - `sync_campaigns_thread`: Syncs drops every 30 minutes + +4. **Drop Handling** (Current Implementation) + ``` + Every 30 minutes: + └─ sync_campaigns() + ├─ claim_all_drops_from_inventory() + ├─ Fetch active campaigns from dashboard + ├─ Update campaign/drop progress + ├─ Check if drops are claimable + └─ Auto-claim claimable drops + ``` + +5. **Events & Notifications** + - Events fired: `Events.DROP_CLAIM`, `Events.DROP_STATUS` + - Notifications via: Discord, Telegram, Matrix, Pushover, Gotify, Webhook + +--- + +## 📊 Key Files & Directory Structure + +``` +TwitchBot/ +├── TwitchChannelPointsMiner/ +│ ├── TwitchChannelPointsMiner.py # Main orchestrator +│ ├── classes/ +│ │ ├── Twitch.py # API handler +│ │ ├── TwitchLogin.py # Authentication +│ │ ├── TwitchWebSocket.py # WebSocket handling +│ │ ├── WebSocketsPool.py # Socket pool manager +│ │ ├── Chat.py # Chat interaction +│ │ ├── Settings.py # Global settings & enums +│ │ ├── AnalyticsServer.py # Flask analytics server +│ │ ├── entities/ +│ │ │ ├── Drop.py # Drop model +│ │ │ ├── Campaign.py # Campaign model +│ │ │ ├── Streamer.py # Streamer model +│ │ │ ├── Stream.py # Stream model +│ │ │ ├── Bet.py # Betting model +│ │ │ └── ... +│ │ └── [Notification classes] # Discord, Telegram, etc. +│ ├── constants.py # API endpoints & constants +│ ├── logger.py # Logging configuration +│ └── utils.py # Helper functions +├── example.py # Configuration template +├── requirements.txt # Dependencies +└── Docs/ # Documentation (new) +``` + +--- + +## 🎯 Current Drops Implementation Details + +### How Drops Currently Work: + +1. **Campaign Discovery:** + - Fetches active campaigns from Twitch Drops Dashboard + - Filters campaigns by date range (start_at < now < end_at) + - Associates campaigns with eligible streamers + +2. **Progress Tracking:** + - Watches are tracked via WebSocket "watching" messages + - Progress updates come from inventory sync (every 30 min) + - Drop progress calculated: `currentMinutesWatched / minutesRequired` + +3. **Claiming Logic:** + - Drop becomes claimable when: + - `currentMinutesWatched >= minutesRequired` + - `hasPreconditionsMet == true` + - `drop_instance_id` is available + - Claim via GraphQL mutation: `DropsPage_ClaimDropRewards` + - Automatic retry with 5-10 second random delay + +4. **Per-Streamer Configuration:** + ```python + StreamerSettings( + claim_drops=True, # Enable drops for this streamer + ... + ) + ``` + +### Current Limitations for Our Use Case: + +1. **Primary Focus:** Channel points, not drops + - Drops are secondary to point mining + - Priority system favors streaks over drops + - Limited drop-specific logging + +2. **Fixed Sync Interval:** 30-minute campaign sync + - Could miss time-sensitive drops + - Not optimized for drop-focused workflow + +3. **Limited Drop Intelligence:** + - No prediction of drop completion times + - No optimization for multi-campaign scenarios + - No handling of channel-specific drop requirements + +4. **Streamer Selection:** + - Priority system doesn't optimize for drop efficiency + - No automatic discovery of best streams for active campaigns + +--- + +## 🔍 Dependencies Analysis + +**Core Dependencies:** +``` +requests # HTTP requests +websocket-client # WebSocket connections +pillow # Image processing (for OCR?) +python-dateutil # Date parsing +emoji # Emoji support +millify # Number formatting +colorama # Colored terminal output +flask # Analytics server +irc # IRC chat (minimal usage) +pandas # Analytics data +pytz # Timezone handling +validators # Input validation +pre-commit # Development tool +``` + +**All dependencies are standard and well-maintained.** + +--- + +## 🎨 Notification System + +The bot supports multiple notification channels for events: + +- **Discord** - Via webhook +- **Telegram** - Via bot API +- **Matrix** - Matrix protocol +- **Pushover** - Push notifications +- **Gotify** - Self-hosted notifications +- **Webhook** - Generic webhooks + +**Relevant Events for Drops:** +- `Events.DROP_CLAIM` - Drop claimed +- `Events.DROP_STATUS` - Drop progress update + +--- + +## 💡 Recommendations for Drops-Focused Bot + +### What to Keep: +✅ Drop & Campaign entity models (well-designed) +✅ Twitch API/GraphQL integration (solid) +✅ WebSocket infrastructure (essential) +✅ Notification system (useful for alerts) +✅ Logger system (comprehensive) +✅ Authentication mechanism (working) + +### What to Modify: +🔧 Priority system → Make drops the primary focus +🔧 Sync interval → Reduce to ~5-10 minutes for drops +🔧 Streamer selection → Algorithm optimized for drop campaigns +🔧 Progress tracking → More granular, per-campaign insights +🔧 Configuration → Simplify for drops-only use case + +### What to Remove/Deprecate: +❌ Betting system (not needed for drops) +❌ Channel points tracking (secondary concern) +❌ Community goals (not relevant) +❌ Watch streak priority (not needed) +❌ Analytics system (or simplify significantly) + +### New Features to Add: +➕ Campaign prioritization algorithm +➕ Automatic discovery of best streams per campaign +➕ Drop progress predictions & ETA +➕ Multi-account support for faster farming +➕ Campaign completion tracking +➕ Smart switching between campaigns +➕ Detailed drop statistics dashboard + +--- + +## 🚀 Next Steps + +1. **Phase 1:** Configuration & Setup + - Create drops-focused default config + - Update example.py for drops use case + - Simplify unnecessary features + +2. **Phase 2:** Core Logic Modifications + - Implement drops-first priority system + - Reduce campaign sync interval + - Enhance drop progress logging + +3. **Phase 3:** New Features + - Campaign optimizer + - Smart streamer selection + - Drop ETA predictions + +4. **Phase 4:** Testing & Documentation + - Test with active campaigns + - Document new usage patterns + - Create user guide + +--- + +## 📝 Technical Notes + +### GraphQL Operations Used: +- `DropsPage_ClaimDropRewards` - Claim drop +- `Inventory` - Get user inventory +- `ViewerDropsDashboard` - Get campaigns +- `DropCampaignDetails` - Get campaign details + +### WebSocket Topics: +- `video-playback-by-id` - Stream events +- `predictions-channel-v1` - Predictions +- Drops progress updates come via API polling, not WebSocket + +### Authentication: +- Uses OAuth login flow +- Stores auth token and cookies +- Handles token refresh automatically + +--- + +## 🎓 Conclusion + +The codebase is well-structured and maintainable. The existing drops functionality provides a solid foundation. Our main task is to shift focus from channel points to drops, optimize the workflow, and add drop-specific enhancements. + +**Confidence Level:** High - The codebase is ready for modification with minimal breaking changes needed. + +--- + +*Document created during initial codebase review - February 17, 2026* diff --git a/Docs/PROJECT_ROADMAP.md b/Docs/PROJECT_ROADMAP.md new file mode 100644 index 0000000..ee3508d --- /dev/null +++ b/Docs/PROJECT_ROADMAP.md @@ -0,0 +1,412 @@ +# Twitch Drops Miner - Project Goals & Roadmap + +**Project Name:** Twitch Drops Miner +**Repository:** https://gitea.majjoduran.app/majjo/Twitch-Drops-Miner +**Fork From:** Twitch-Channel-Points-Miner-v2 by rdavydov +**Date Started:** February 17, 2026 + +--- + +## 🎯 Project Vision + +Transform the Twitch Channel Points Miner into a dedicated **Twitch Drops farming bot** that efficiently collects drops from multiple campaigns across different games, optimizing watch time and maximizing drop acquisition. + +--- + +## 🔑 Key Objectives + +### Primary Goals: +1. ✨ **Drops-First Approach:** Make drops collection the primary and only focus +2. 🎮 **Multi-Campaign Support:** Handle multiple active campaigns simultaneously +3. 🤖 **Smart Automation:** Automatically find best streams for active campaigns +4. 📊 **Progress Visibility:** Clear, real-time drop progress tracking +5. ⚡ **Efficiency:** Minimize wasted watch time, maximize drops/hour + +### Secondary Goals: +- Multi-account support (future) +- Drop value tracking (rarity, market data) +- Campaign notification system +- Historical stats and analytics + +--- + +## 📋 Development Phases + +### **Phase 1: Foundation & Setup** ✅ (Current) +**Status:** In Progress +**Timeline:** Day 1 + +**Tasks:** +- [x] Fork and clone repository +- [x] Set up git remotes (origin → Gitea, upstream → original) +- [x] Install dependencies +- [x] Review and document codebase +- [x] Add `.github` and `.vscode` to `.gitignore` +- [ ] Initial commit and push to Gitea +- [ ] Create project README + +--- + +### **Phase 2: Simplification & Cleanup** +**Status:** Not Started +**Timeline:** Days 2-3 + +**Tasks:** +- [ ] Remove/disable betting system +- [ ] Remove/disable channel points tracking mechanisms +- [ ] Simplify priority system to drops-only +- [ ] Remove unnecessary analytics features +- [ ] Strip out community goals and prediction code +- [ ] Clean up UI/logging to focus on drops +- [ ] Update configuration templates + +**Deliverables:** +- Leaner codebase focused solely on drops +- Updated `example.py` with drops-only config +- Simplified settings structure + +--- + +### **Phase 3: Core Drops Enhancement** +**Status:** Not Started +**Timeline:** Days 4-7 + +**Tasks:** +- [ ] Reduce campaign sync interval (30min → 5-10min) +- [ ] Implement real-time drop progress updates +- [ ] Add drop completion ETA calculations +- [ ] Enhanced drop progress logging with visual bars +- [ ] Add campaign priority ranking system +- [ ] Implement smart campaign switching logic +- [ ] Add drop history tracking + +**Deliverables:** +- More responsive drop detection +- Better progress visibility +- Smarter campaign management + +--- + +### **Phase 4: Intelligent Streamer Selection** +**Status:** Not Started +**Timeline:** Days 8-10 + +**Tasks:** +- [ ] Implement campaign-aware streamer discovery +- [ ] Auto-find eligible streams for active campaigns +- [ ] Prioritize streams by viewer count (balance detection risk) +- [ ] Handle channel-specific drop restrictions +- [ ] Add fallback logic when no eligible streams online +- [ ] Stream quality optimization (lowest bandwidth) + +**Deliverables:** +- Automatic streamer discovery per campaign +- Optimized watch strategy +- Reduced manual configuration + +--- + +### **Phase 5: Configuration & Usability** +**Status:** Not Started +**Timeline:** Days 11-12 + +**Tasks:** +- [ ] Create simple drop-focused configuration wizard +- [ ] Implement configuration validation +- [ ] Add campaign whitelist/blacklist +- [ ] Game preference system +- [ ] Dry-run mode for testing +- [ ] Better error messages and troubleshooting + +**Deliverables:** +- User-friendly setup process +- Flexible campaign filtering +- Easier debugging + +--- + +### **Phase 6: Advanced Features** +**Status:** Not Started +**Timeline:** Days 13-15 + +**Tasks:** +- [ ] Multi-account support (parallel farming) +- [ ] Drop value/rarity tracking +- [ ] Campaign notifications (new campaigns, completion) +- [ ] Web dashboard for monitoring +- [ ] Mobile notifications support +- [ ] Drop statistics and reporting + +**Deliverables:** +- Professional-grade farming tool +- Comprehensive monitoring +- Historical analytics + +--- + +### **Phase 7: Testing & Documentation** +**Status:** Not Started +**Timeline:** Days 16-18 + +**Tasks:** +- [ ] Comprehensive testing with real campaigns +- [ ] Performance optimization +- [ ] Memory leak testing +- [ ] Complete user documentation +- [ ] API documentation for developers +- [ ] Troubleshooting guide +- [ ] FAQ section + +**Deliverables:** +- Stable, production-ready bot +- Complete documentation +- User guides and tutorials + +--- + +### **Phase 8: Polish & Release** +**Status:** Not Started +**Timeline:** Days 19-20 + +**Tasks:** +- [ ] Code cleanup and refactoring +- [ ] Add license and attribution +- [ ] Create release notes +- [ ] Setup CI/CD (optional) +- [ ] Docker container support +- [ ] Release v1.0.0 + +**Deliverables:** +- First public release +- Docker image +- Installation guides + +--- + +## 🛠️ Technical Architecture Changes + +### Current Architecture: +``` +Channel Points Focus +├── Watch streams for points +├── Make predictions/bets +├── Claim bonus points +└── Drops as secondary feature +``` + +### Target Architecture: +``` +Drops Focus +├── Monitor active campaigns +├── Auto-discover eligible streams +├── Watch for drop progress +├── Claim drops automatically +└── Optimize for maximum drops/time +``` + +--- + +## 🔧 Key Technical Changes + +### 1. **Priority System Overhaul** +```python +# OLD (Multi-priority) +priority=[Priority.STREAK, Priority.DROPS, Priority.ORDER] + +# NEW (Drops-focused) +priority=DropsPriority.CAMPAIGN_URGENCY # Expiring campaigns first +``` + +### 2. **Campaign Sync Optimization** +```python +# OLD: 30-minute sync interval +sync_interval = 1800 # seconds + +# NEW: 5-10 minute interval + smart triggers +sync_interval = 300 # More responsive +trigger_sync_on = ["drop_claimable", "campaign_progress"] +``` + +### 3. **Streamer Selection Logic** +```python +# OLD: Manual list + priority order +streamers = [Streamer("user1"), Streamer("user2")] + +# NEW: Auto-discovery based on campaigns +def get_eligible_streamers(campaign): + """Discover live streams eligible for campaign drops""" + # Query Twitch directory for game + # Filter by campaign requirements + # Return optimal stream to watch +``` + +### 4. **Configuration Simplification** +```python +# OLD: Complex multi-feature config +TwitchChannelPointsMiner( + username="user", + priority=[...], + streamer_settings=StreamerSettings( + make_predictions=True, + follow_raid=True, + claim_drops=True, + watch_streak=True, + # ... 10+ more options + ) +) + +# NEW: Drops-focused config +TwitchDropsMiner( + username="user", + auto_discover_streams=True, + campaign_filters={ + "games": ["Overwatch 2", "Valorant"], + "priority": "expiring_first" + }, + notifications=["drop_claimed", "campaign_complete"] +) +``` + +--- + +## 📊 Success Metrics + +### Performance Targets: +- ✅ Drop claim success rate: >95% +- ✅ Campaign detection latency: <5 minutes +- ✅ False positive rate: <1% +- ✅ Memory usage: <500MB +- ✅ CPU usage: <10% average + +### User Experience Targets: +- ⭐ Setup time: <10 minutes +- ⭐ Configuration complexity: Minimal +- ⭐ Log clarity: Excellent +- ⭐ Error recovery: Automatic + +--- + +## 🚧 Known Challenges + +### Technical Challenges: +1. **Campaign Discovery:** Twitch API limitations for finding eligible streams +2. **Rate Limiting:** GraphQL request throttling +3. **Drop Detection:** Ensuring timely progress updates +4. **Stream Quality:** Balance between bandwidth and detection + +### Strategic Challenges: +1. **Bot Detection:** Twitch's anti-automation measures +2. **Account Safety:** Avoiding bans +3. **Ethical Considerations:** Respect for streamers/platform +4. **Multi-account Limits:** Twitch TOS compliance + +### Solutions: +- Implement human-like behavior patterns +- Randomize timing and actions +- Respect rate limits +- Clear documentation on safe usage +- Disclaimer about TOS compliance + +--- + +## 🔐 Ethical Considerations + +**Important Notes:** +- This tool is for educational purposes +- Users are responsible for compliance with Twitch TOS +- We do not encourage violations of platform rules +- Use responsibly and respect content creators +- Consider supporting streamers you enjoy + +--- + +## 📚 Documentation Plan + +### User Documentation: +- [ ] Installation guide +- [ ] Quick start tutorial +- [ ] Configuration reference +- [ ] Troubleshooting guide +- [ ] FAQ +- [ ] Best practices + +### Developer Documentation: +- [ ] Architecture overview +- [ ] API reference +- [ ] Contributing guidelines +- [ ] Code style guide +- [ ] Testing procedures + +--- + +## 🤝 Contributing Guidelines (Future) + +When ready for contributions: +- Clear code of conduct +- Issue templates +- PR guidelines +- Development setup guide +- Testing requirements + +--- + +## 📅 Timeline Summary + +| Phase | Duration | Status | +|-------|----------|--------| +| Phase 1: Setup | 1 day | 🟡 In Progress | +| Phase 2: Cleanup | 2 days | ⚪ Not Started | +| Phase 3: Core Enhancement | 4 days | ⚪ Not Started | +| Phase 4: Smart Selection | 3 days | ⚪ Not Started | +| Phase 5: Configuration | 2 days | ⚪ Not Started | +| Phase 6: Advanced Features | 3 days | ⚪ Not Started | +| Phase 7: Testing & Docs | 3 days | ⚪ Not Started | +| Phase 8: Release | 2 days | ⚪ Not Started | +| **Total** | **~20 days** | **5% Complete** | + +--- + +## 🎉 Milestone Celebrations + +- 🎊 **Milestone 1:** First successful auto-claimed drop +- 🎊 **Milestone 2:** Complete one full campaign automatically +- 🎊 **Milestone 3:** Support 5 simultaneous campaigns +- 🎊 **Milestone 4:** 100 drops claimed in testing +- 🎊 **Milestone 5:** v1.0.0 Release + +--- + +## 📝 Notes & Ideas + +### Future Enhancement Ideas: +- Browser extension for manual claiming backup +- Mobile app for monitoring +- Drop marketplace integration +- Campaign calendar/schedule +- Community drop trading (if allowed) +- AI-powered campaign selection +- Distributed farming across multiple machines + +### Community Features: +- Campaign sharing/recommendations +- Drop tracking leaderboard +- Community statistics +- Best stream suggestions + +--- + +## 🔄 Change Log + +**2026-02-17:** +- Initial project setup +- Repository forked and cloned +- Dependencies installed +- Codebase analysis completed +- Project roadmap created +- Added `.github` and `.vscode` to `.gitignore` + +--- + +*This roadmap is a living document and will be updated as the project progresses.* + +**Last Updated:** February 17, 2026