How to download a PDF file with a link that show up in your browser as a PDF?

[GHRyunosuke]

1. login to a website (automated with SeleniumVBA)
2. Ask for the download of a specific PDF on the website, which would send me a link to the PDF (automated with SeleniumVBA)
3. Click the link and download the PDF (got stuck here)

Here is the problem in detail:
The link of the PDF I got is in the format of "https://XXX.net/account/download_cfd_statement/2024-01-28T22:00:00Z/pdf/", which if you click this link, it opens as a PDF file within your browser as shown below (your need to maintain login status of the website for the pdf to show in the browser, so I guess there is some kind of verification of this link). Any ideas on how to download this PDF automatically?

BWT, dirver.NavigateTo "https://XXX.net/account/download_cfd_statement/2024-01-28T22:00:00Z/pdf/" would only redirect to the web page as shown below and could NOT download the pdf. So for now, I have to download it manually...

[GCuser99]

Hi @GHRyunosuke, did you set the download preferences before navigating to the pdf url?
If the new way (experimental) below does not work for you, then comment it out and try the old way:

Sub test_file_download2()
    'WARNING - this currently will fail in Edge/Chrome if running in Incognito mode 
    'see https://github.com/GCuser99/SeleniumVBA/issues/87 for work-around(s)
    Dim driver As WebDriver
 
    Set driver = New WebDriver
    driver.StartChrome
    
    'set the directory path for saving download to
    Dim caps As WebCapabilities
    Set caps = driver.CreateCapabilities
    caps.SetDownloadPrefs ".\"
    driver.OpenBrowser caps
    
    'delete legacy copy if it exists
    driver.DeleteFiles ".\dummy.pdf"
    
    driver.NavigateTo "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
    
    driver.WaitForDownload ".\dummy.pdf"
    
    driver.CloseBrowser
    driver.Shutdown
End Sub

CORRECTED!

[GHRyunosuke]

Well, i got the following run time error at "driver.SetDownloadFolder ".\"" (same issue for "caps.SetDownloadPrefs ".\"" as well) ...
BTW, in Japanese system, "\" becomes "￥" which is fine.

Is below "VBE windows caption" (taken from my VBE) ?

[GHRyunosuke]

@GCuser99 Confirmed that the "old way" you suggested will download the pdf (while the "new way" only shows it inside browser). Therefore the pdf download issue is solved by the old way. Thank you so much !

[6DiegoDiego9]

Hi @GHRyunosuke, that error indicates that the regex used by SeleniumVBA to extract the file name out from your VBE window, that is
"^Microsoft Visual Basic[^-]* - (.?)(?:| [[^]]]) - [[^]]*]$"
doesn't match with the caption of your VBE window, that is
"Microsoft Visual Basic for Application - All_in_One.xlsm"

Apparently, and I remember that we already saw this before, your caption format is different than our caption format, where our format includes the module name too, like this:
"Microsoft Visual Basic for Application - All_in_One.xlsm - [moduleName (Code)]"
while your format shows the module name as title of the sub-window (assuming your module is "MyAutomation").

Go to the WebShared class module and change the regex to this fixed version:
"^Microsoft Visual Basic[^-]* - (.?)(?:| - [[^]]])$"

Please let us know if this fix worked.

[GHRyunosuke]

Thank you @6DiegoDiego9, however, it is not fixed...

I found something interesting about how to reproduce this issue and also how to avoid this issue (i.e. no need to change the regex).

1. This issue does NOT happen if you maximize your code editor window
   

2. This issue will happen if you un-maximized the code editor window, then the caption would be different and triggers this issue. --> Plus, even I modify the regex into "^Microsoft Visual Basic[^-]* - (.?)(?:| - [[^]]])$" the same issue remains.
   

[GHRyunosuke]

Hello @GCuser99 thanks for your suggestion, here comes the result:

1. New way - "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf" will be opened inside the browser and will not be downloaded.
   

2. Old way - this works for "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf", it will not be opened inside the browser, but instead it be downloaded as a pdf file !!
   

Tomorrow, i will test my case which is "https://xxx.net/account/download_cfd_statement/2024-01-28T22:00:00Z/pdf/" with the "old way" to see if it could be downloaded or not.

[6DiegoDiego9]

Oh, I see from your screenshots that you have three different formats for the caption of the main VBE window, which is the only one of interest to us:

A)

B)

C)

Can you please confirm that?
(how do you obtain the first one?)
Can you also confirm that, in order to get the file name, we (the new regex) can ignore all the Japanese parts?

On my system, I can only see these:

D)

A)

Once I'll see all the formats, I'll make a new fixed regex supporting them all.

[GHRyunosuke]

Q: Can you also confirm that, in order to get the file name, we (the new regex) can ignore all the Japanese parts?
A: Yes, I think so and I could help with testing.

Q: how do you obtain the first one?
A1: Unmaximized the editor window and that's all.

A2: Close all the editor windows

[GCuser99]

Thanks @GHRyunosuke for helping us debug this issue and thanks @6DiegoDiego9 for sharing your expertise on RegExp. BTW, @6DiegoDiego9, a heads-up - the pattern that you gave @GHRyunosuke to try in your post above was not the same as you used in https://regex101.com/ - I think the GitHub Markdown is escaping characters from the pattern when you copy it from https://regex101.com/.

[6DiegoDiego9]

@GHRyunosuke, I understood that you obtain both A) and C) by unmaximizing the editor window. Can you please clarify the different way you obtain A) vs C)?

@GCuser99 oh yes you're right, it's likely because of markdown rendering. I'll be careful when I paste the new fixed version. Thanks!

[GHRyunosuke]

@6DiegoDiego9 , You got c) by running/executing the code, that is the caption while the code is running.

[6DiegoDiego9]

Here is the fixed regex: ^Microsoft Visual Basic[^-]*?- ([^[]+?\.xl\w\w?)(?:| [^$]+?)$

Please let us know if it works for you with all the formats, although I doubt you can have the C) type in real cases.

[GHRyunosuke]

@6DiegoDiego9 Above regex works & confirmed the issue is fixed! Thank you so much.

[GCuser99]

@6DiegoDiego9 works for me too!

[GCuser99]

@GHRyunosuke, so it seems that you have found your way to download the pdf(?). Plus with @6DiegoDiego9's help, we made the relative path logic more robust!

On my "new way" versus "old way" for downloading a file, I had forgotten that those two "ways" are really not comparable. The "driver.SetDownloadFolder" just allows you to change the download folder AFTER you have set the capabilities through "caps.SetDownloadPrefs". Here would be an example using those two methods correctly:

Sub test_file_download2()
    'WARNING - this currently will fail in Edge/Chrome if running in Incognito mode 
    'see https://github.com/GCuser99/SeleniumVBA/issues/87 for work-around(s)
    Dim driver As WebDriver
 
    Set driver = New WebDriver
    driver.StartChrome
    
    'set download preferences through capabilities
    Dim caps As WebCapabilities
    Set caps = driver.CreateCapabilities
    caps.SetDownloadPrefs "%USERPROFILE%\Documents\myFolder"
    driver.OpenBrowser caps

    'delete legacy copy if it exists
    driver.DeleteFiles "%USERPROFILE%\Documents\myFolder\dummy.pdf"
    
    driver.NavigateTo "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
    
    driver.WaitForDownload "%USERPROFILE%\Documents\myFolder\dummy.pdf"
  
    'now change the download folder without redoing capabilities
     driver.SetDownloadFolder "%USERPROFILE%\Documents\myDifferentFolder"

    'delete legacy copy if it exists
    driver.DeleteFiles "%USERPROFILE%\Documents\myDifferentFolder\dummy.pdf"
    
    driver.NavigateTo "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
    
    driver.WaitForDownload "%USERPROFILE%\Documents\myDifferentFolder\dummy.pdf"
  
    driver.CloseBrowser
    driver.Shutdown
End Sub

Hope that clarifies how to use those two methods. I will modify the Wiki to explain that properly. Sorry for the confusion.

[GHRyunosuke]

Thank you @GCuser99
Yes, i have solved the pdf download issue with the "old way" you suggested (without this setting it only shows pdf inside the browser).
Now, with your kind explanation above I am closer to the "Certified SeleniumVBA Specialist" :)

[6DiegoDiego9]

Revised regex to work with even more tricky file names like "test.xls - abcd.xls.xlam":
^Microsoft Visual Basic[^-]*- (.*\.xl\w{1,2})(?:|(?:| -) \[.*\])$