Skip to content

Commit 12a0fd3

Browse files
tsegismonttsegismont
authored
HttpClient: introduce a connection timeout in ProxyOptions (#5650)
* HttpClient: introduce a connection timeout in ProxyOptions See #5634 In Netty's ProxyHandler, a default timeout of 10 seconds is set for the connection to be established successfully. Successfully means slightly different things, depending on the proxy type: - for an HTTP proxy, it means the LastHttpContent message of the proxy response to the CONNECT request has been received, and the status code is 200 - for a SOCKS proxy, it means the SOCKS handshake ended with the SUCCESS status. Sometimes, users have to deal with external servers behind their proxy that they don't control and that are slow to establish the connection. In such cases, it is required to increase the ProxyHandler timeout. Signed-off-by: Thomas Segismont <[email protected]> * Use java.time.Duration for ProxyOptions connection timeout Signed-off-by: Thomas Segismont <[email protected]> --------- Signed-off-by: Thomas Segismont <[email protected]>
1 parent 444aeff commit 12a0fd3

File tree

11 files changed

+273
-64
lines changed

11 files changed

+273
-64
lines changed

‎vertx-core/src/main/asciidoc/http.adoc‎

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2234,14 +2234,12 @@ The addresses of the handlers are given by {@link io.vertx.core.http.WebSocket#b
22342234

22352235
=== Using a proxy for HTTP/HTTPS connections
22362236

2237-
The http client supports accessing http/https URLs via an HTTP proxy (e.g. Squid) or _SOCKS4a_ or _SOCKS5_ proxy.
2238-
The CONNECT protocol uses HTTP/1.x but can connect to HTTP/1.x and HTTP/2 servers.
2237+
The {@link io.vertx.core.http.HttpClient} supports accessing HTTP/HTTPS URLs via an HTTP proxy (e.g. Squid), a _SOCKS4a_, or a _SOCKS5_ proxy.
2238+
The `CONNECT` protocol uses HTTP/1.x but can connect to HTTP/1.x and HTTP/2 servers.
22392239

2240-
Connecting to h2c (unencrypted HTTP/2 servers) is likely not supported by http proxies since they will support
2241-
HTTP/1.1 only.
2240+
Connecting to `h2c` (unencrypted HTTP/2 servers) is likely not supported by http proxies since they will support HTTP/1.1 only.
22422241

2243-
The proxy can be configured in the {@link io.vertx.core.http.HttpClientOptions} by setting a
2244-
{@link io.vertx.core.net.ProxyOptions} object containing proxy type, hostname, port and optionally username and password.
2242+
The proxy can be configured in the {@link io.vertx.core.http.HttpClientOptions} by setting a {@link io.vertx.core.net.ProxyOptions} object containing proxy type, hostname, port and optionally username and password.
22452243

22462244
Here's an example of using an HTTP proxy:
22472245

@@ -2250,11 +2248,9 @@ Here's an example of using an HTTP proxy:
22502248
{@link examples.HTTPExamples#example58}
22512249
----
22522250

2253-
When the client connects to an http URL, it connects to the proxy server and provides the full URL in the
2254-
HTTP request ("GET http://www.somehost.com/path/file.html HTTP/1.1").
2251+
When the client connects to an HTTP URL, it connects to the proxy server and provides the full URL in the HTTP request, like `GET http://www.somehost.com/path/file.html HTTP/1.1`.
22552252

2256-
When the client connects to an https URL, it asks the proxy to create a tunnel to the remote host with
2257-
the CONNECT method.
2253+
When the client connects to an HTTPS URL, it asks the proxy to create a tunnel to the remote host with the `CONNECT` method.
22582254

22592255
For a SOCKS5 proxy:
22602256

@@ -2263,27 +2259,37 @@ For a SOCKS5 proxy:
22632259
{@link examples.HTTPExamples#example59}
22642260
----
22652261

2266-
The DNS resolution is always done on the proxy server, to achieve the functionality of a SOCKS4 client, it is necessary
2267-
to resolve the DNS address locally.
2262+
The DNS resolution is always done on the proxy server, to achieve the functionality of a SOCKS4 client, it is necessary to resolve the DNS address locally.
22682263

2269-
Proxy options can also be set per request:
2264+
{@link io.vertx.core.net.ProxyOptions} can also be set per request:
22702265

22712266
[source,$lang]
22722267
----
22732268
{@link examples.HTTPExamples#perRequestProxyOptions}
22742269
----
22752270

2276-
NOTE: client connection pooling is aware of proxies (including authentication), consequently two requests to the same host through different proxies
2277-
do not share the same pooled connection
2271+
[NOTE]
2272+
====
2273+
Client connection pooling is aware of proxies (including authentication).
2274+
Consequently, two requests to the same host through different proxies do not share the same pooled connection.
2275+
====
22782276

2279-
You can use {@link io.vertx.core.http.HttpClientOptions#setNonProxyHosts} to configure a list of host bypassing
2280-
the proxy. The lists accept `*` wildcard for matching domains:
2277+
You can use {@link io.vertx.core.http.HttpClientOptions#setNonProxyHosts} to configure a list of host bypassing the proxy.
2278+
The list accepts `*` wildcard for matching domains:
22812279

22822280
[source,$lang]
22832281
----
22842282
{@link examples.HTTPExamples#nonProxyHosts}
22852283
----
22862284

2285+
By default, a 10 seconds connection timeout is set for the proxy handler in the Vert.x HTTP client.
2286+
If the target server takes longer than that to accept the connection, or if the proxy is too busy and delays completion of the handshake with the client, you might increase this timeout:
2287+
2288+
[source,$lang]
2289+
----
2290+
{@link examples.HTTPExamples#proxyOptionsConnectTimeout}
2291+
----
2292+
22872293
==== Handling of other protocols
22882294

22892295
The HTTP proxy implementation supports getting ftp:// urls if the proxy supports

‎vertx-core/src/main/generated/io/vertx/core/net/ProxyOptionsConverter.java‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,11 @@ static void fromJson(Iterable<java.util.Map.Entry<String, Object>> json, ProxyOp
3737
obj.setType(io.vertx.core.net.ProxyType.valueOf((String)member.getValue()));
3838
}
3939
break;
40+
case "connectTimeout":
41+
if (member.getValue() instanceof Number) {
42+
obj.setConnectTimeout(java.time.Duration.of(((Number)member.getValue()).longValue(), java.time.temporal.ChronoUnit.MILLIS));
43+
}
44+
break;
4045
}
4146
}
4247
}
@@ -59,5 +64,8 @@ static void toJson(ProxyOptions obj, java.util.Map<String, Object> json) {
5964
if (obj.getType() != null) {
6065
json.put("type", obj.getType().name());
6166
}
67+
if (obj.getConnectTimeout() != null) {
68+
json.put("connectTimeout", obj.getConnectTimeout().toMillis());
69+
}
6270
}
6371
}

‎vertx-core/src/main/java/examples/HTTPExamples.java‎

Lines changed: 39 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,21 +13,53 @@
1313

1414
import io.netty.handler.codec.compression.GzipOptions;
1515
import io.netty.handler.codec.compression.StandardCompressionOptions;
16-
import io.vertx.core.*;
16+
import io.vertx.core.AbstractVerticle;
17+
import io.vertx.core.AsyncResult;
18+
import io.vertx.core.DeploymentOptions;
19+
import io.vertx.core.Expectation;
20+
import io.vertx.core.Future;
21+
import io.vertx.core.Handler;
22+
import io.vertx.core.MultiMap;
23+
import io.vertx.core.Vertx;
1724
import io.vertx.core.buffer.Buffer;
1825
import io.vertx.core.file.AsyncFile;
1926
import io.vertx.core.file.FileSystem;
2027
import io.vertx.core.file.OpenOptions;
21-
import io.vertx.core.http.*;
28+
import io.vertx.core.http.ClientForm;
29+
import io.vertx.core.http.ClientMultipartForm;
30+
import io.vertx.core.http.Cookie;
31+
import io.vertx.core.http.HttpClient;
32+
import io.vertx.core.http.HttpClientAgent;
33+
import io.vertx.core.http.HttpClientConnection;
34+
import io.vertx.core.http.HttpClientOptions;
35+
import io.vertx.core.http.HttpClientRequest;
36+
import io.vertx.core.http.HttpClientResponse;
37+
import io.vertx.core.http.HttpConnectOptions;
38+
import io.vertx.core.http.HttpHeaders;
39+
import io.vertx.core.http.HttpMethod;
40+
import io.vertx.core.http.HttpResponseExpectation;
41+
import io.vertx.core.http.HttpResponseHead;
42+
import io.vertx.core.http.HttpServer;
43+
import io.vertx.core.http.HttpServerOptions;
44+
import io.vertx.core.http.HttpServerRequest;
45+
import io.vertx.core.http.HttpServerResponse;
46+
import io.vertx.core.http.PoolOptions;
47+
import io.vertx.core.http.RequestOptions;
48+
import io.vertx.core.http.ServerWebSocket;
49+
import io.vertx.core.http.WebSocket;
50+
import io.vertx.core.http.WebSocketClient;
51+
import io.vertx.core.http.WebSocketConnectOptions;
52+
import io.vertx.core.http.WebSocketFrame;
2253
import io.vertx.core.json.JsonObject;
23-
import io.vertx.core.net.endpoint.LoadBalancer;
2454
import io.vertx.core.net.NetSocket;
2555
import io.vertx.core.net.ProxyOptions;
2656
import io.vertx.core.net.ProxyType;
57+
import io.vertx.core.net.endpoint.LoadBalancer;
2758
import io.vertx.core.net.endpoint.ServerEndpoint;
2859
import io.vertx.core.streams.Pipe;
2960
import io.vertx.core.streams.ReadStream;
3061

62+
import java.time.Duration;
3163
import java.util.List;
3264
import java.util.concurrent.TimeUnit;
3365

@@ -1290,6 +1322,10 @@ public void perRequestProxyOptions(HttpClient client, ProxyOptions proxyOptions)
12901322
});
12911323
}
12921324

1325+
public void proxyOptionsConnectTimeout(ProxyOptions proxyOptions) {
1326+
proxyOptions.setConnectTimeout(Duration.ofSeconds(60));
1327+
}
1328+
12931329
public void example60(Vertx vertx) {
12941330

12951331
HttpClientOptions options = new HttpClientOptions()

‎vertx-core/src/main/java/io/vertx/core/net/ProxyOptions.java‎

Lines changed: 39 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,13 @@
1111

1212
package io.vertx.core.net;
1313

14-
import java.util.Objects;
15-
1614
import io.vertx.codegen.annotations.DataObject;
1715
import io.vertx.codegen.json.annotations.JsonGen;
1816
import io.vertx.core.json.JsonObject;
1917

18+
import java.time.Duration;
19+
import java.util.Objects;
20+
2021
/**
2122
* Proxy options for a net client or a net client.
2223
*
@@ -33,7 +34,7 @@ public class ProxyOptions {
3334

3435
/**
3536
* The default port for proxy connect = 3128
36-
*
37+
* <p>
3738
* 3128 is the default port for Squid
3839
*/
3940
public static final int DEFAULT_PORT = 3128;
@@ -43,11 +44,17 @@ public class ProxyOptions {
4344
*/
4445
public static final String DEFAULT_HOST = "localhost";
4546

47+
/**
48+
* The default timeout for proxy connect = 10 seconds
49+
*/
50+
public static final Duration DEFAULT_CONNECT_TIMEOUT = Duration.ofSeconds(10);
51+
4652
private String host;
4753
private int port;
4854
private String username;
4955
private String password;
5056
private ProxyType type;
57+
private Duration connectTimeout;
5158

5259
/**
5360
* Default constructor.
@@ -56,6 +63,7 @@ public ProxyOptions() {
5663
host = DEFAULT_HOST;
5764
port = DEFAULT_PORT;
5865
type = DEFAULT_TYPE;
66+
connectTimeout = DEFAULT_CONNECT_TIMEOUT;
5967
}
6068

6169
/**
@@ -69,6 +77,7 @@ public ProxyOptions(ProxyOptions other) {
6977
username = other.getUsername();
7078
password = other.getPassword();
7179
type = other.getType();
80+
connectTimeout = other.getConnectTimeout();
7281
}
7382

7483
/**
@@ -200,4 +209,31 @@ public ProxyOptions setType(ProxyType type) {
200209
this.type = type;
201210
return this;
202211
}
212+
213+
/**
214+
* Get the connection timeout , defaults to {@code 10} seconds.
215+
* <p>
216+
* A connection to the proxy is considered successful when:
217+
*
218+
* <ul>
219+
* <li>the client received a {@code 200} response to the {@code CONNECT} request for HTTP proxies, or</li>
220+
* <li>the {@code SOCKS} handshake ended with the {@code SUCCESS} status, for SOCKS proxies.</li>
221+
* </ul>
222+
*
223+
* @return the connection timeout
224+
*/
225+
public Duration getConnectTimeout() {
226+
return connectTimeout;
227+
}
228+
229+
/**
230+
* Set the connection timeout.
231+
*
232+
* @param connectTimeout the connection timeout
233+
* @return a reference to this, so the API can be used fluently
234+
*/
235+
public ProxyOptions setConnectTimeout(Duration connectTimeout) {
236+
this.connectTimeout = connectTimeout;
237+
return this;
238+
}
203239
}

‎vertx-core/src/main/java/io/vertx/core/net/impl/ChannelProvider.java‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,17 @@
1212
package io.vertx.core.net.impl;
1313

1414
import io.netty.bootstrap.Bootstrap;
15-
import io.netty.channel.*;
16-
import io.netty.handler.proxy.*;
15+
import io.netty.channel.Channel;
16+
import io.netty.channel.ChannelFuture;
17+
import io.netty.channel.ChannelHandlerContext;
18+
import io.netty.channel.ChannelInboundHandlerAdapter;
19+
import io.netty.channel.ChannelInitializer;
20+
import io.netty.channel.ChannelPipeline;
21+
import io.netty.handler.proxy.HttpProxyHandler;
22+
import io.netty.handler.proxy.ProxyConnectionEvent;
23+
import io.netty.handler.proxy.ProxyHandler;
24+
import io.netty.handler.proxy.Socks4ProxyHandler;
25+
import io.netty.handler.proxy.Socks5ProxyHandler;
1726
import io.netty.handler.ssl.SslHandler;
1827
import io.netty.handler.ssl.SslHandshakeCompletionEvent;
1928
import io.netty.resolver.NoopAddressResolverGroup;
@@ -213,6 +222,10 @@ private void handleProxyConnect(Handler<Channel> handler, SocketAddress remoteAd
213222
: new Socks4ProxyHandler(proxyAddr);
214223
break;
215224
}
225+
long connectTimeout = proxyOptions.getConnectTimeout().toMillis();
226+
if (connectTimeout > 0) {
227+
proxy.setConnectTimeoutMillis(connectTimeout);
228+
}
216229

217230
bootstrap.resolver(NoopAddressResolverGroup.INSTANCE);
218231
java.net.SocketAddress targetAddress = vertx.transport().convert(remoteAddress);

‎vertx-core/src/test/java/io/vertx/test/proxy/HttpProxy.java‎

Lines changed: 28 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,7 @@
1111

1212
package io.vertx.test.proxy;
1313

14-
import java.net.UnknownHostException;
15-
import java.util.Base64;
16-
import java.util.Map;
17-
import java.util.concurrent.ConcurrentHashMap;
18-
import java.util.concurrent.TimeUnit;
19-
14+
import io.vertx.core.Future;
2015
import io.vertx.core.MultiMap;
2116
import io.vertx.core.Vertx;
2217
import io.vertx.core.http.*;
@@ -26,6 +21,12 @@
2621
import io.vertx.core.net.NetSocket;
2722
import io.vertx.test.http.HttpTestBase;
2823

24+
import java.net.UnknownHostException;
25+
import java.util.Base64;
26+
import java.util.Map;
27+
import java.util.concurrent.ConcurrentHashMap;
28+
import java.util.concurrent.TimeUnit;
29+
2930
/**
3031
* Http Proxy for testing
3132
*
@@ -117,18 +118,7 @@ public HttpProxy start(Vertx vertx) throws Exception {
117118
client.connect(port, host).onComplete(ar1 -> {
118119
if (ar1.succeeded()) {
119120
localAddresses.add(ar1.result().localAddress().toString());
120-
request.toNetSocket().onComplete(ar2 -> {
121-
if (ar2.succeeded()) {
122-
NetSocket serverSocket = ar2.result();
123-
NetSocket clientSocket = ar1.result();
124-
serverSocket.closeHandler(v -> clientSocket.close());
125-
clientSocket.closeHandler(v -> serverSocket.close());
126-
serverSocket.pipeTo(clientSocket);
127-
clientSocket.pipeTo(serverSocket);
128-
} else {
129-
// Not handled
130-
}
131-
});
121+
toNetSocket(vertx, request, ar1.result());
132122
} else {
133123
request.response().setStatusCode(403).end("request failed");
134124
}
@@ -194,6 +184,26 @@ public HttpProxy start(Vertx vertx) throws Exception {
194184
return this;
195185
}
196186

187+
private void toNetSocket(Vertx vertx, HttpServerRequest request, NetSocket clientSocket) {
188+
Future<HttpServerRequest> fut;
189+
if (successDelayMillis > 0) {
190+
fut = vertx.timer(successDelayMillis).map(v -> request);
191+
} else {
192+
fut = Future.succeededFuture(request);
193+
}
194+
fut.compose(req -> req.toNetSocket().onComplete(ar2 -> {
195+
if (ar2.succeeded()) {
196+
NetSocket serverSocket = ar2.result();
197+
serverSocket.closeHandler(v -> clientSocket.close());
198+
clientSocket.closeHandler(v -> serverSocket.close());
199+
serverSocket.pipeTo(clientSocket);
200+
clientSocket.pipeTo(serverSocket);
201+
} else {
202+
// Not handled
203+
}
204+
}));
205+
}
206+
197207
/**
198208
* Stop the server.
199209
* <p>

‎vertx-core/src/test/java/io/vertx/test/proxy/Socks4Proxy.java‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,7 @@
1515
import io.vertx.core.buffer.Buffer;
1616
import io.vertx.core.internal.logging.Logger;
1717
import io.vertx.core.internal.logging.LoggerFactory;
18-
import io.vertx.core.net.NetClient;
19-
import io.vertx.core.net.NetClientOptions;
20-
import io.vertx.core.net.NetServer;
21-
import io.vertx.core.net.NetServerOptions;
22-
import io.vertx.core.net.NetSocket;
18+
import io.vertx.core.net.*;
2319

2420
import java.util.concurrent.CompletableFuture;
2521
import java.util.concurrent.TimeUnit;
@@ -103,7 +99,11 @@ public Socks4Proxy start(Vertx vertx) throws Exception {
10399
if (result.succeeded()) {
104100
localAddresses.add(result.result().localAddress().toString());
105101
log.debug("writing: " + toHex(connectResponse));
106-
socket.write(connectResponse);
102+
if (successDelayMillis > 0) {
103+
vertx.setTimer(successDelayMillis, tid -> socket.write(connectResponse));
104+
} else {
105+
socket.write(connectResponse);
106+
}
107107
log.debug("connected, starting pump");
108108
NetSocket clientSocket = result.result();
109109
socket.closeHandler(v -> clientSocket.close());

0 commit comments

Comments
 (0)