Skip to content

Commit 192c7e6

Browse files
authored
[bugfix] download_needed param error in kernels_output (#629)
When I use the "kaggle kernel output" to download files, it always re-downloads files that already exist. Additionally, I found some issues in the kernels_output method. 1) When calling download_needed, the first parameter is url, but it expects a response object instead. 2) After fixing issue 1, the method does skip existing files. However, it’s still slow, and system monitoring tools show high network usage, indicating that the files are actually being downloaded again. The download_needed method only requires response.headers, but requests.get downloads the entire file by default. This can be fixed by setting the stream=True parameter to change the download behavior.
1 parent 1b27df4 commit 192c7e6

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

kaggle/api/kaggle_api_extended.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2616,8 +2616,8 @@ def kernels_output(self, kernel, path, force=False, quiet=True):
26162616
for item in response['files']:
26172617
outfile = os.path.join(target_dir, item['fileName'])
26182618
outfiles.append(outfile)
2619-
download_response = requests.get(item['url'])
2620-
if force or self.download_needed(item, outfile, quiet):
2619+
download_response = requests.get(item['url'], stream=True)
2620+
if force or self.download_needed(download_response, outfile, quiet):
26212621
os.makedirs(os.path.split(outfile)[0], exist_ok=True)
26222622
with open(outfile, 'wb') as out:
26232623
out.write(download_response.content)

src/kaggle/api/kaggle_api_extended.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2600,8 +2600,8 @@ def kernels_output(self, kernel, path, force=False, quiet=True):
26002600
for item in response['files']:
26012601
outfile = os.path.join(target_dir, item['fileName'])
26022602
outfiles.append(outfile)
2603-
download_response = requests.get(item['url'])
2604-
if force or self.download_needed(item, outfile, quiet):
2603+
download_response = requests.get(item['url'], stream=True)
2604+
if force or self.download_needed(download_response, outfile, quiet):
26052605
os.makedirs(os.path.split(outfile)[0], exist_ok=True)
26062606
with open(outfile, 'wb') as out:
26072607
out.write(download_response.content)

0 commit comments

Comments
 (0)