improve error message when http timeout

[prary]

Environment:

• Jib version: 1.4.0
• Build tool: Maven 3.5.4
• *OS: Windows

Description of the issue:
While running mvn compile jib:build, mvn timesout it pushes 50% of the built image.
Below is the screenshot of the build failure.

      <configuration>
          <to>
            <image>images.test.net/container/kube/tmp/python/
          </to>
        </configuration>

timeout in 28 second
only image is placeholder is modified, user and password are supplied in m2 settins file

Expected behavior:
It should push the build image

Steps to reproduce:

1. Ran the mvn compile jib:build in example/hello-world

[prary]

tag was missing, since I am on corporate network default base image is not accessible.
Can we give desciptive or more appropriate error than timeout?

[chanseokoh]

Hi @prary,

Seems like you found a cause and the issue is resolved.

Aside, the timeout error (I assume an exception stack trace) should already be fairly obvious in that a network call timed out. What error message do you see exactly and why do you think it is not descriptive enough? Is it the part that the Jib connecting to Google Container Registry to use the default base image gcr.io/distoless/java is not obvious? If not, what descriptive message would you want to see exactly?

[prary]

Hi @chanseokoh it did not show it is trying to connect to gcr/distr...... I have tried with - X flag, only thing it showed was timeout and a traceback of java classes. On reading documentation I got to know about default case image, hence I was able to troubleshoot.

[chanseokoh]

Ah, we'll look into this. We should show the endpoint (or the image reference) that timed out in the error.

[chanseokoh]

Things to consider:

From #1917, Jib 1.5.0 using the new version of Google HTTP Client seems to treat timeout as ConnectionException instead of SocketTimeoutException. If <allowInsecureRegistries> is set, this usually makes Jib fall back to HTTP. Need to verify the behavior change and how we can determine timeout if it throws ConnectionException. Also need to decide if we want to fall back on timeout. (Docker seems to do so, if what is reported in #1917 is true.) However, falling back on timeout is practically useless without first fixing #946, as explained in #1917 (comment). An argument against falling back could be that timeout is often an intermittent failure. Now I think I don't like the fact we have to try HTTPS first to fall back to HTTP, as too much magic and non-deterministic nature. The user should already be clear to know if the registry is at port 80 or 443. Perhaps the user should make it explicit in the config which protocol and port Jib should use (for both base and target registries).

[chanseokoh]

Also from #1917 (comment), apparently the default connection timeout of 20 seconds at the socket level isn't enforced or effective. I'd like to figure out why.

[chanseokoh]

Also from #1917 (comment), apparently the default connection timeout of 20 seconds at the socket level isn't enforced or effective. I'd like to figure out why.

Did some digging, and I think it's a bug in the new Google HTTP Client. Filed googleapis/google-http-java-client#799.

From #1917, Jib 1.5.0 using the new version of Google HTTP Client seems to treat timeout as ConnectionException instead of SocketTimeoutException.

And I think this is a bug in Apache HttpClient (not Google HTTP Client). Filed HTTPCLIENT-2013.

[chanseokoh]

On my machine with OpenJDK 8 and Oracle JDK 12, Socket.connect() sometimes throws java.net.SocketTimeoutException: connect timed out and other times java.net.ConnectException: Connection timed out (Connection timed out), depending on the timeout value. An Apache dev claims that this is a bug in JDK and it should not throw ConnectException. Filed a bug against Oracle JDK, but it is still in internal review.

So, it wasn't that the newer Apache library in 1.5.0 made a breaking behavioral change to throw a different exception when a timeout happens. Rather, it was that the timeout was not set at all due to the bug in Google HTTP Client, which made the JDK throw a different exception.

In any case, I think it is unlikely that JDK will fix this bug soon (if it is a bug), probably not at all for Java 8, so I think it is better for us to try to identify the timeout case by examining the exception message as best as we can.

I also intend to streamline the Jib behavior so that timeout does not trigger the HTTP-fallback, but we can revisit this later. Perhaps the whole HTTP-fallback magic.

[chanseokoh]

Filed a bug against Oracle JDK, but it is still in internal review.

They've done the internal review, and the bug is now posted: JDK-8230528 Their first comment is kind of what I anticipated.