Initial commit Scrapyard

README.md

@@ -0,0 +1,83 @@
+# Scrapyard
+
+A self-hosted aggregator for refurbished laptops. Scrapes multiple Swedish second-hand stores, enriches CPU specs from Intel ARK, and serves a filterable web UI.
+
+---
+
+## Requirements
+
+- Python 3.11+
+- Firefox (used by Playwright for JavaScript-heavy pages)
+
+---
+
+## Installation
+
+```bash
+# 1. Clone the repo
+git clone <repo-url>
+cd scrapyard
+
+# 2. Create and activate a virtual environment
+python -m venv .venv
+
+# Windows
+.venv\Scripts\activate
+
+# macOS / Linux
+source .venv/bin/activate
+
+# 3. Install dependencies
+pip install -r requirements.txt
+
+# 4. Install the Playwright Firefox browser
+playwright install firefox
+
+# 5. Start the app
+python main.py
+```
+
+Open **http://localhost:5000** in your browser.
+
+The database (`database.db`) is created automatically on first run. It ships with CPU spec data pre-populated — no scraping needed to browse those.
+
+---
+
+## Scraping data
+
+Use the **Scraping** panel at the bottom of the sidebar in the UI, or trigger scrapers directly:
+
+| Button | What it does |
+|---|---|
+| Scrape BilligTeknik | Crawls BilligTeknik listing and detail pages |
+| Scrape Nuvoo | Crawls Nuvoo via the Shopify JSON API |
+| Enrich CPUs | Fetches detailed CPU specs from Intel ARK |
+
+Scrapers run in the background — the UI stays responsive while they work.
+
+---
+
+## Updating
+
+```bash
+# Pull latest changes
+git pull
+
+# Install any new dependencies
+pip install -r requirements.txt
+
+# Restart the app
+python main.py
+```
+
+---
+
+## Pre-push database wipe
+
+The database is not committed to the repo (see `.gitignore`). Before pushing, run the purge script to clear all scraped listings while keeping CPU data intact:
+
+```bash
+python purge_db.py
+```
+
+It will show a summary and ask for confirmation before deleting anything.