Skip to content

fix(deepcopy): TypeError related to pickling '_thread.RLock' objects #376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

PeriniM
Copy link
Contributor

@PeriniM PeriniM commented Jun 12, 2024

Related to #374

Implements a custom deep copy function in SmartScraperMultiGraph to handle non-picklable objects and prevent TypeError during object copying.

  • Custom Deep Copy Function: Introduces a custom deep copy function within the __init__ method of SmartScraperMultiGraph. This function is designed to safely copy configurations, including handling of non-picklable objects like _thread.RLock, which resolves the issue of TypeError when attempting to pickle such objects.
  • Configuration Copying: Utilizes the custom deep copy function to copy the config dictionary passed to SmartScraperMultiGraph. This ensures that the configurations for AzureChatOpenAI and AzureOpenAIEmbeddings instances are copied without raising exceptions related to pickling.

For more details, open the Copilot Workspace session.

Copy link

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails

Scanned Manifest Files

@PeriniM PeriniM changed the base branch from main to pre/beta June 12, 2024 20:26
@PeriniM PeriniM closed this Jun 26, 2024
@shishirkmr
Copy link

We might need the same fixes in SearchGraph as well.

scrapegraphai/graphs/search_graph.py

I am getting the same error in SearchGraph .
TypeError: cannot pickle '_thread.RLock' object

@VinciGit00 VinciGit00 deleted the PeriniM/fix-pickling-error branch September 3, 2024 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants