Support Forum: Get Support for Patch My PC Products and Services

Microsoft Configuration Manager and Intune (Enterprises/Paid) => Report Publishing/Download/Hash Issues (Enterprises Using ConfigMgr and Intune) => Topic started by: quertest on July 19, 2019, 11:04:41 AM

Title: Timeout when downloading large files
Post by: quertest on July 19, 2019, 11:04:41 AM

Our firewall scans all downloads before forwarding. For this reason, the proxy temporarily sends only very small data packets until the virus scan is completed. Currently, however, Patch My PC cancels the download after only 40 seconds ("An error occurred while downloading the file:  The operation has timed out"). This is too small for larger downloads like LibreOffice.

Is it possible to configure the (proxy) timeout for downloading applications?

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on July 19, 2019, 11:41:56 AM

We will have to look into the methods available in .NET to see if increasing the timeout may help in your scenario.

Quote from: quertest on July 19, 2019, 11:04:41 AM

Our firewall scans all downloads before forwarding. For this reason, the proxy temporarily sends only very small data packets until the virus scan is completed. Currently, however, Patch My PC cancels the download after only 40 seconds ("An error occurred while downloading the file:  The operation has timed out"). This is too small for larger downloads like LibreOffice.

Is it possible to configure the (proxy) timeout for downloading applications?

Title: Re: Timeout when downloading large files
Post by: quertest on August 12, 2019, 11:04:34 AM

Have you possibly found a way to adjust the proxy timeout? 

The problem also affects other applications (such as Adobe Acrobat DC). By the way, it is currently not 40 seconds to the timeout as I wrote before, but 1 minute and 40 seconds.

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on August 12, 2019, 11:06:35 AM

Curious if you have run any type of network capture during the download timeout?

Quote from: quertest on August 12, 2019, 11:04:34 AM

Have you possibly found a way to adjust the proxy timeout? 

The problem also affects other applications (such as Adobe Acrobat DC). By the way, it is currently not 40 seconds to the timeout as I wrote before, but 1 minute and 40 seconds.

Title: Re: Timeout when downloading large files
Post by: quertest on August 12, 2019, 11:27:18 AM

I haven't done a network capture yet. However, most other downloads work fine. The proxy always waits for the complete download to complete, so that it can perform a virus scan before forwarding it to the client. For the affected products the download is obviously too slow to download the file within 100 seconds, check for viruses and then forward it to the Patch-My-PC service. I currently assume that adjusting the proxy timeout to e.g. 300 seconds could solve the problem.

However, I will check again in the next few days via network capture whether there could be another cause.

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on August 12, 2019, 11:29:48 AM

As a workaround, you could use this option for now?

(https://i.imgur.com/cp1lWSv.png)

Title: Re: Timeout when downloading large files
Post by: quertest on August 12, 2019, 11:42:13 AM

Yes, as a workaround this is possible. However, I would like to avoid manual downloads in the future.

In principle, we can also switch off the virus scan for these downloads on the proxy as a workaround. Then the downloads will be forwarded to the Patch-My-PC service as supplied by the source. However, I would like to avoid this for security reasons.

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on August 12, 2019, 11:47:43 AM

We already perform VirusTotal scans. Can you exclude it for just the server? https://patchmypc.com/deep-dive-into-security-validation-of-third-party-software-updates-in-microsoft-sccm

Title: Re: Timeout when downloading large files
Post by: quertest on August 12, 2019, 12:03:02 PM

I'll check that out.

Please consider the following note. If you are using the HttpClient from .Net, then the download time can be extended via the Timeout property.  It is 100 seconds by default, which corresponds to the time where the download is currently aborted in Patch-My-PC (which may also be a coincidence) https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient.timeout?view=netframework-4.8 (https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient.timeout?view=netframework-4.8)

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on August 12, 2019, 12:18:00 PM

There are some other things we need to take into account when increasing the timeout. What's the longest you see for the largest update when downloading through your proxy.

Title: Re: Timeout when downloading large files
Post by: quertest on August 13, 2019, 09:50:56 AM

I ran some tests on the server with the download URLs aborted by Patch-My-PC using the Powershell function Invoke-WebRequest (which has a -TimeoutSec switch for extending the timeout). Whether the values would be comparable with the .Net download function you used, I cannot estimate.

http://ftp.osuosl.org/pub/tdf/libreoffice/stable/6.1.6/win/x86_64/LibreOffice_6.1.6_Win_x64.msi	640 sec
http://armdl.adobe.com/pub/adobe/acrobat/win/AcrobatDC/1901220035/AcrobatDCUpd1901220035.msp	539 sec
https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2019.1.39.0.msi	1468 sec

But the main reason seems to be that these sources provide the files only with limited bandwidth.

As already written, for all other downloads we use, the 100 sec time window seems to be sufficient for the download.

Title: Re: Timeout when downloading large files
Post by: quertest on August 14, 2019, 12:20:13 PM

I repeated the tests because I didn't really trust the very high values of yesterday. This time I used the Powershell function Start-BitsTransfer. The download times are much shorter and probably more comparable with Patch-My-PC.

http://ftp.osuosl.org/pub/tdf/libreoffice/stable/6.1.6/win/x86_64/LibreOffice_6.1.6_Win_x64.msi	195 sec
http://armdl.adobe.com/pub/adobe/acrobat/win/AcrobatDC/1901220035/AcrobatDCUpd1901220035.msp	120 sec
https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2019.1.39.0.msi	21 sec

The Remote Desktop Manager was probably a temporary problem yesterday.

On the basis of these measured times an extension of the timeout to 300 seconds would seem to be good.

Title: Re: Timeout when downloading large files
Post by: quertest on September 18, 2019, 12:52:49 AM

Could you check whether a configuration of the download timeout time would be possible?
Unfortunately, there are problems with a few large downloads or certain download providers due to the currently fixed limitation to 100 seconds.

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on September 18, 2019, 05:01:16 AM

Could you submit this feature request at: https://ideas.patchmypc.com/?

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on October 11, 2019, 04:44:32 AM

This is available in preview build 1.5.4.2: https://patchmypc.com/advanced-configurations-available-using-the-registry-for-patch-my-pcs-publishing-service

Title: Re: Timeout when downloading large files
Post by: quertest on October 11, 2019, 06:34:28 AM

Thank you for the implementation!!! The configuration of the Download Timeout works.

But could you raise the configurable upper limit to at least 600 seconds?

There is still a download that obviously takes longer than 5 minutes (http://ftp.osuosl.org/pub/tdf/libreoffice/stable/6.2.7/win/x86_64/LibreOffice_6.2.7_Win_x64.msi).

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on October 11, 2019, 07:30:46 AM

Yeah, in the next preview build we could allow it up to 600 I think.

Quote from: quertest on October 11, 2019, 06:34:28 AM

Thank you for the implementation!!! The configuration of the Download Timeout works.

But could you raise the configurable upper limit to at least 600 seconds?

There is still a download that obviously takes longer than 5 minutes (http://ftp.osuosl.org/pub/tdf/libreoffice/stable/6.2.7/win/x86_64/LibreOffice_6.2.7_Win_x64.msi).

Title: Re: Timeout when downloading large files
Post by: quertest on October 14, 2019, 01:58:59 AM

Thanks for the adjustment! With the registry key DownloadTimeout=600 and the new preview version 1.5.4.2 all current downloads were successful for the first time!
The only thing that doesn't seem to be right is the display in the log. There I still see "Download timeout is 300000 milliseconds".

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on October 14, 2019, 04:55:46 AM

The current preview only allows up to 300 seconds. That's why in the log you see that.

Quote from: quertest on October 14, 2019, 01:58:59 AM

Thanks for the adjustment! With the registry key DownloadTimeout=600 and the new preview version 1.5.4.2 all current downloads were successful for the first time!
The only thing that doesn't seem to be right is the display in the log. There I still see "Download timeout is 300000 milliseconds".

Title: Re: Timeout when downloading large files
Post by: quertest on October 14, 2019, 06:24:38 AM

I just saw in the log that the download of the LibreOffice x64 version on Sunday took only 3 minutes 42 seconds (=222 seconds). The days before were always more than 5 minutes (=300 seconds) required for the download.

Since the required download time obviously varies a lot, an increase to 600 seconds would be very helpful for us in any case.

Title: Re: Timeout when downloading large files
Post by: Justin Chalfant on October 14, 2019, 07:28:15 AM

np, we are planning to bump the max to 600 in the next preview build.

Quote from: quertest on October 14, 2019, 06:24:38 AM

I just saw in the log that the download of the LibreOffice x64 version on Sunday took only 3 minutes 42 seconds (=222 seconds). The days before were always more than 5 minutes (=300 seconds) required for the download.

Since the required download time obviously varies a lot, an increase to 600 seconds would be very helpful for us in any case.