Skip to content

A very simple Python package that scrapes the Vinted website to retrieve information about its items.

License

Notifications You must be signed in to change notification settings

TestStudent156/vinted_scraper

 
 

Repository files navigation

Vinted Scraper

Package Version Python Version codecov License FOSSA Status

A very simple Python package that scrapes the Vinted site to retrieve information about its items.

Installation

Stable

You can install Vinted Scraper using pip:

    pip install vinted_scraper==2.4.0

If you are on Python 3.6 you also have to install dataclasses: pip install dataclasses

Alpha

We move from requests to httpx to support Async API call. Now, you can await AsyncVintedScraper or AsyncVintedWrapper. I haven't finish to update all the docs but you can check async quick starts to understand how they work.

To install the alpha version with pip:

    pip install vinted_scraper==3.0.0a1

Compatible from python 3.8+

For more info about Alpha check the roadmap, and please if you find a bug open a issue!

Functions

The package offers the following functions:

search - (gets all the items present on the listing page)

Parameters

name type data type description
params optional Dict Query parameters like the pagination and so on
item - (gets the information about an item, and its seller present on the item detail page)

It is currently not working and will throw a 404 status code (see #78). Vinted has changed this endpoint, and we didn't find a replacement.

Parameters

name type data type description
id required str The unique identifier of the item to retrieve
params optional Dict I don't know is they exist

Usage

To obtain the scraped data as a vinted_scraper.models.VintedItem, so you can:

import vinted_scraper.VintedScraper


def main():
    scraper = VintedScraper("https://www.vinted.com")  # init the scraper with the baseurl
    params = {
        "search_text": "board games"
        # Add other query parameters like the pagination and so on
    }
    items = scraper.search(params)  # get all the items


if __name__ == "__main__":
    main()

VintedScraper returns structured data that are parsed and converted into a vinted_scraper.models.VintedItem object. If some attributes are None means that it wasn't found in the response, maybe because they are returned from other API. Also, I discard some attribute that I thought was useless but feel free to open an issue or a PR to add them.

If you want to manage the JSON response directly, you should use the VintedWrapper object instead of VintedScraper.

Here's the way of how to use it:

import vinted_scraper.VintedWrapper


def main():
    wrapper = VintedWrapper("https://www.vinted.com")  # init the scraper with the baseurl
    params = {
        "search_text": "board games"
        # Add other query parameters like the pagination and so on
    }
    items = wrapper.search(params)  # get all the items


if __name__ == "__main__":
    main()

Logging

You can configure logging for the package using the built-in logger utility:

from vinted_scraper.utils import configure_logger

# Set up logging (INFO level by default)
logger = configure_logger(level="DEBUG")

# Use in your code
logger.info("Logger is configured!")

You can also set the log format, date format, and output stream. All internal logs (e.g., from VintedWrapper) will use this logger if configured.

Documentation

HTML documentation is generated with pdoc:

pdoc src/vinted_scraper -o docs

The generated docs can be found in the docs/ directory. To preview them locally, open docs/index.html in your browser.

To publish as GitHub Pages:

  • Set the GitHub Pages source to the /docs folder in your repository settings.
  • Or, automate deployment with GitHub Actions.

License

This project is licensed under the MIT License - see the LICENSE file for details.

FOSSA Status

About

A very simple Python package that scrapes the Vinted website to retrieve information about its items.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 96.7%
  • Makefile 2.9%
  • JavaScript 0.4%