A self-hosted aggregator for refurbished laptops. Scrapes multiple Swedish second-hand stores, enriches CPU specs from Intel ARK, and serves a filterable web UI.

Requirements

Python 3.11+
Firefox (used by Playwright for JavaScript-heavy pages)

Installation

# 1. Clone the repo
git clone <repo-url>
cd scrapyard

# 2. Create and activate a virtual environment
python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Install the Playwright Firefox browser
playwright install firefox

# 5. Start the app
python main.py

Open http://localhost:5000 in your browser.

The database (database.db) is created automatically on first run. It ships with CPU spec data pre-populated — no scraping needed to browse those.

Scraping data

Use the Scraping panel at the bottom of the sidebar in the UI, or trigger scrapers directly:

Button	What it does
Scrape BilligTeknik	Crawls BilligTeknik listing and detail pages
Scrape Nuvoo	Crawls Nuvoo via the Shopify JSON API
Enrich CPUs	Fetches detailed CPU specs from Intel ARK

Scrapers run in the background — the UI stays responsive while they work.

Updating

# Pull latest changes
git pull

# Install any new dependencies
pip install -r requirements.txt

# Restart the app
python main.py

Pre-push database wipe

The database is not committed to the repo (see .gitignore). Before pushing, run the purge script to clear all scraped listings while keeping CPU data intact:

python purge_db.py

It will show a summary and ask for confirmation before deleting anything.