The mystery remains

[davidb2b]

I hired a software developer familiar with OpenCart to investigate. Here is his response:

"I have run multiple tests from different devices and using several internet channels. Still cannot reproduce this issue.
So I don't understand what to fix now. This does not look like a php timeout I thought."

He is in Europe, so his downloads were a good test. None of them failed.

The only other thing I have done is to activate the PhP cache on the dedicated server itself. This was off by default, and they claim it "only provides benefit for websites using mod_php or FCGID". Not sure if it applies but shouldn't do any harm.

So, if the problem persists, I truly don't know where to go from here.

[threeheadedmonkey]

I've just tried my usual file again, and it's still failing.
Did you or your software developer try out the speed-limited download manager like I suggested?

I just did some googling, and found some (very old!) forum threads starting in 2011 but with newer posts going all the way to last year.
They all seem to have a very similar problem to yours, with the suggested fix being to install an Apache extension or to purchase one for opencart that may or may not fix the issue:
https://forum.opencart.com/viewtopic.php?f=20&t=36746
https://forum.opencart.com/viewtopic.php?t=172408

Quote from the top thread:

The php script's memory consumption increases exponentially
as the script is left open. Once the download is held open for a while,
the script hits our shared server's memory limit and is killed off.

If it really is exponential, then a moderate change of RAM allowance probably doesn't make much difference.

[davidb2b]

I've just tried my usual file again, and it's still failing.
Did you or your software developer try out the speed-limited download manager like I suggested?

I just did some googling, and found some (very old!) forum threads starting in 2011 but with newer posts going all the way to last year.
They all seem to have a very similar problem to yours, with the suggested fix being to install an Apache extension or to purchase one for opencart that may or may not fix the issue:
https://forum.opencart.com/viewtopic.php?f=20&t=36746
https://forum.opencart.com/viewtopic.php?t=172408

Quote from the top thread:

The php script's memory consumption increases exponentially
as the script is left open. Once the download is held open for a while,
the script hits our shared server's memory limit and is killed off.

If it really is exponential, then a moderate change of RAM allowance probably doesn't make much difference.

I didn't try the limiter. A notable difference here is that this is not a shared server, and the memory limit is actually 63 GB. That's the biggest there is available, at least at my hosting company. Nevertheless, it should be more than adequate.

There's a bit to unpack here. From 2013 until just a few months ago, I used OpenCart 1.5.5.1, and with a few exceptions, no one had trouble downloading large files, typically 2 GB in size. If the problem arose solely because the server kills a process after 500 seconds/600 seconds (back then, it was set at the default of 30 seconds), many buyers should have experienced this problem then, but they didn't. At the moment, I am running my forced stripping store using OpenCart 2.3, on the same server, and the problem does not occur there either. So why has it become an issue with OpenCart 3? Moreover, customers in places like Japan, Australia and Russia must experience long download times, and yet very few have reported a problem with their downloads not completing.

The problem for me is that the issue is not consistent. If it were purely the process timing out, it should affect everyone whose download takes more than 500 seconds, but it doesn't. It affects only a handful of users. I can increase the process time a little more for Apache, but there comes a point where it will become detrimental to the operation of the server.

I did switch on the server cache which seems to have sped things up for at least one user having this problem, although it might just be that his connection was faster on that day. Since doing this, the number of cases has certainly dropped. I have not had a single report of a failed download in the past six days, other than this one, and from another user I asked to run a test. Even there, of his 8 files he originally could not download, he was able to download 7 after the server cache was switched on. Is there a connection between the two events, or is it coincidence? I don't know.

I will read the thread regarding an Apache extension, but it may not be something I am permitted to do. I assume it just keeps the process open for longer, presumably by passing it to something outside of OpenCart? However, I am not willing to hack the code in OpenCart. That's what made upgrading impossible last time and necessitated a build from scratch, losing all customer data along the way. It's not something I ever want to repeat.

[threeheadedmonkey]

Everything you say makes sense, and I agree that hacking OpenCart would be the road to madness.

Like I said before, I have zero experience with your tech stack / OpenCart, so doing a bit of googling was the last vaguely helpful thing I could think of.
If you ever do find a fix (or something that looks like one and needs testing), feel free to drop me a line and I'll try it out.

In the meantime, when this conversation originally started, you mentioned that you'd be able to get the files to me some other way.
Would you mind doing that for me, please?

[davidb2b]

Everything you say makes sense, and I agree that hacking OpenCart would be the road to madness.

Like I said before, I have zero experience with your tech stack / OpenCart, so doing a bit of googling was the last vaguely helpful thing I could think of.
If you ever do find a fix (or something that looks like one and needs testing), feel free to drop me a line and I'll try it out.

In the meantime, when this conversation originally started, you mentioned that you'd be able to get the files to me some other way.
Would you mind doing that for me, please?

Of course. Just write to me at north@bound2burst.net and I'll get that sorted out for you.

II have increased the "Packet Keep-alive" setting in Apache in an effort to prevent processes getting killed off mid-stream, but I have no idea yet if it is working. There have been fewer issues of late. I begin to think this may be unsolvable with a platform like OpenCart. It served me well when the file sizes were typically 500 MB, but these days, almost nothing is much short of a GB, and many files are significantly larger. Just imagine if I started publishing 4K files!