Jump to content

BestBuy Parser


Recommended Posts

  • Superuser

A personal tool using Python's Scrapy framework to scrape Best Buy's product pages for RTX 3080 TIs and notify if available/not sold out.

My first project using Python's Scrapy framework.


I'm using this project personally for a couple friends of mine and I. Basically, it scrapes a products listing page from BestBuy that lists RTX 3080 TIs. It scans each product and if the c-button-disable class doesn't exist within each entry (indicating it is not sold out and available), it will email a list of users from the settings.py file. It keeps each ID tracked in SQLite to make sure users don't get emailed more than once.


The Scrapy framework is required and may be installed with the following.


python3 -m pip install scrapy

Settings are configured in the src/bestbuy_parser/bestbuy_parser/settings.py file. The following are defaults.


# General Scrapy settings.
BOT_NAME = 'bestbuy_parser'

SPIDER_MODULES = ['bestbuy_parser.spiders']
NEWSPIDER_MODULE = 'bestbuy_parser.spiders'


# The User Agent used to crawl.
USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'

# Obey robots.txt rules

# Best Buy Parser-specific settings.

# The email's subject to send.
MAIL_SUBJECT = "RTX 3080 TI In Stock On Best Buy!"

# Where the email is coming from.
MAIN_FROM = "[email protected]"

# The email body.
MAIL_BODY = '<html><body><ul><li><a href="https://www.bestbuy.com{link}">{name}</a></li><li>{price}</li></ul></body></html>


Running The Program
You must change the working directory to src/bestbuy_parser/bestbuy_parser via cd. Afterwards, you may run the following.


python3 parse.py

This will run the program until a keyboard interrupt.


Systemd Service
A systemd service is included in the systemd/ directory. It is assuming you cloned the repository into /usr/src (you will need to change the systemd file if this is not correct).


You may install the systemd service via the following command as root (or ran with sudo).


sudo make install



GitHub Repository & Source Code

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Create New...

Important Information

By using this site you agree to the Terms of Use and Privacy Policy. We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.