Skip to content

Replace direct http connection with requests.Session() Using connection pool #1733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 19, 2024

Conversation

vbrodsky
Copy link
Contributor

@vbrodsky vbrodsky commented Jul 17, 2024

Description

Switch from direct http access to using Session and related Request and PreparedRequest classes. This gives us a number of advantages:

  • automatic use of http connection pool and keep-alive via Session
  • making the code cleaner via Session, Request, PreparedRequest classes

Immediate benefit is fix for network connections issues we had with our sdk integration tests in codefresh
This was tested as follows:

  • Make kubernetes context for gke_lb-cicd-codefresh_us-central1_env0 is configured locally
  • Start sdk integration tests in codefresh
  • Docker id is part of the codefresh run url
  • Connect to codefresh docker like kubectl -n codefresh-workers exec -it dind-6697ec95c9240c841e27d1f6 -- bash
  • docker ps to find the id of our test docker container
  • docker exec -it 3b17acc97be7 /bin/bash
  • to confirm failures run the following python snipper:
import requests
while True:
    response2 = requests.get('https://app.lb-stage.xyz/signin', timeout=10)
    print(response2.status_code)
  • to confirm using a requests session fixes the issue, run
import requests
s = requests.Session()
while True:
    response = s.get('https://app.lb-stage.xyz/signin', timeout=10)
    print(response.status_code)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you updated any code comments, as applicable?

@vbrodsky vbrodsky requested a review from a team as a code owner July 17, 2024 16:14
@vbrodsky vbrodsky force-pushed the VB/client-connection-pool branch from d839245 to 1d6a8d9 Compare July 17, 2024 16:32
@vbrodsky vbrodsky marked this pull request as draft July 18, 2024 20:02
@vbrodsky vbrodsky marked this pull request as ready for review July 18, 2024 21:30
Val Brodsky added 5 commits July 18, 2024 22:05
@vbrodsky vbrodsky force-pushed the VB/client-connection-pool branch from 254fae3 to ce94a3e Compare July 19, 2024 02:05
@vbrodsky vbrodsky merged commit dd2e3b4 into develop Jul 19, 2024
27 checks passed
@vbrodsky vbrodsky deleted the VB/client-connection-pool branch July 19, 2024 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants