Bug ID: JDK-8305529 DefaultProxySelector.select(URI) in certain cases returns a List with null element

Type: Bug
Component: core-libs
Sub-Component: java.net
Affected Version: 11,17

Priority: P4
Status: Closed
Resolution: Fixed
OS: linux
CPU: x86_64

Submitted: 2023-03-30
Updated: 2024-11-08
Resolved: 2023-04-13

JDK 21
21 b19Fixed

ADDITIONAL SYSTEM INFORMATION :
OpenJDK Runtime Environment 11.0.18 / Linux 5.15.0-1029-kvm (amd64)

# Environment Variables
http_proxy = 
https_proxy = 
sock_proxy = 

# System Properties
java.version = 11.0.18
java.version.date = 2023-01-17
java.vm.specification.name = Java Virtual Machine Specification
java.vm.specification.vendor = Oracle Corporation
java.vm.specification.version = 11
java.vm.vendor = Ubuntu
java.vm.version = 11.0.18+10-post-Ubuntu-0ubuntu122.04

A DESCRIPTION OF THE PROBLEM :
The new HTTP/2 client unexpected throws NPEs on some end-users systems. The reason seems to be the existence of set but empty environment variables: http_proxy | https_proxy | sock_proxy


Here's the original bug report from our customer:
https://www.filebot.net/forums/viewtopic.php?t=13656


FREQUENCY : often

A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/13424 Date: 2023-04-11 11:24:55 +0000
08-11-2024
Changeset: 3f36dd81 Author: Jaikiran Pai <jpai@openjdk.org> Date: 2023-04-13 01:30:00 +0000 URL: https://git.openjdk.org/jdk/commit/3f36dd811e56ecb4b7c6bf1bf8be8a8de9481ed0
13-04-2023
Thank you for the detailed report and the investigation of this issue in the linked forum threads. There's more than one issue across different libraries here. The JDK has a bug where in its *nix implementation of the DefaultProxySelector, it can sometimes end up returning an array with null element in it, which then causes this NullPointerException. That bug is trivially reproducible when one of the proxy related environment variables is set to an incorrect (but non-empty value) and the java.net.useSystemProxies system property is set to true. I was able to reproduce this one easily. The other part to this issue is that, the reporter notes that the users have been hitting this issue when the proxy related environment variables are set to empty values and java.net.useSystemProxies system property is set to true. The linked forum threads indeed show that the values are empty, like this snippet from one of those logs in that forum: ... SYSTEMD_EXEC_PID = 25216 USER = qbtuser http_proxy = https_proxy = sock_proxy = # Java System Properties .... However, I haven't able to reproduce this issue with empty values. To be sure that the fix I propose would indeed fix this issue, I looked at the underlying library's code that the JDK uses (glib-2.0, whose source is here https://gitlab.gnome.org/GNOME/glib). It appears that as recently as around 9 months back there was a fix made to make sure that this library skips/ignores empty values for such environment variables https://gitlab.gnome.org/GNOME/glib/-/commit/6f83f45db4b859839b81f07cc942a49834663ffc. Before this fix, it appears that the library would use these empty values and then raise errors later on, which then the JDK code would ignore (that's the bug in the JDK). That explains why I couldn't reproduce this issue with empty values - the system where I'm testing this, is on version 2.72.4 of this glib-2.0 library and this version has that fix. It's likely that the users that are running into this issue, on that forum, are on a different version of glib-2.0 library which doesn't have the fix to ignore empty values and that's then triggering the JDK bug.
11-04-2023
Additional information from the submitter: JDK11 and JDK17 on Desktop Linux with Gnome Desktop Manager. Possibly an issue with GConf behaving strange in some odd environments, and then tripping the JDK with unexpected null values. Reproducible for some of our customers, but we haven't been able to reproduce the issue in VMs or docker environments: https://www.filebot.net/forums/viewtopic.php?t=13656 https://www.filebot.net/forums/viewtopic.php?t=13678 A DESCRIPTION OF THE PROBLEM : The new HTTP/2 client fails with an NPE in internal JDK code if the code is running in a oddly configured Linux environment, with set but empty proxy environment variable AND java.net.useSystemProxies=true enabled via Java System Properties. $ printenv \| grep proxy https_proxy= sock_proxy= http_proxy= The NPE originates from this line in HttpRequestImpl::retrieveProxy with the following exception message: Cannot invoke "java.net.Proxy.type()" because "<local4>" is null https://github.com/openjdk/jdk20u/blob/master/src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java#L304 My guess is that the native Proxy[] DefaultProxySelector::getSystemProxies method somehow returns a Proxy array that is not null, but unexpectedly contains null elements for some reason: https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libnet/DefaultProxySelector.c#L460 STEPS TO FOLLOW TO REPRODUCE THE PROBLEM : Sorry, I was not able to reproduce the issue on various Desktop Linux machines using the following test case: groovy -Djava.net.useSystemProxies=true -e 'println java.net.http.HttpClient.newHttpClient().send(java.net.http.HttpRequest.newBuilder().uri(new URI("https://bugs.java.com/bugdatabase/")).GET().build(),java.net.http.HttpResponse.BodyHandlers.ofString())' So this test case is untested, because so far it's worked as expected and I haven't actually managed to reproduce the issue reported by customers. EXPECTED VERSUS ACTUAL BEHAVIOR : EXPECTED - JDK internals should never throw NPEs. Perhaps an extra .filter(Objects::notNull) here would be good just in case: https://github.com/openjdk/jdk20u/blob/master/src/java.base/share/classes/sun/net/spi/DefaultProxySelector.java#L322 CUSTOMER SUBMITTED WORKAROUND : Using -DuseSystemProxies=false resolves the issue. This is also default in the JDK, so it's probably not an issue for most JDK users. FREQUENCY : always
10-04-2023
Additional information from the submitter: The issue affects both JDK11 and JDK17, and up probably. We were able to further narrow down the issue to -Djava.net.useSystemProxies=true being set in combination with invalid system proxy (i.e. set but empty environment variables) settings: https://www.filebot.net/forums/viewtopic.php?p=59805#p59805 The NPE likely originates from this line here because testing on JDK the NPE gives additional information: Cannot invoke "java.net.Proxy.type()" because "<local4>" is null https://github.com/openjdk/jdk20u/blob/master/src/java.net.http/share/classes/jdk/internal/net/http/HttpRequestImpl.java#L304 My guess is that the native Proxy[] DefaultProxySelector::getSystemProxies method somehow returns a Proxy array that is not null, but unexpectedly contains null elements for some reason: https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libnet/DefaultProxySelector.c#L460
05-04-2023
Additional information from the submitter: I'm currently unable to make a reproducible test case because I'm on macOS / JDK17 and cannot reproduce the issue here in my environment. The error does suggest the issue originates from JDK internals though, though I don't know what the code for Linux / JDK11 would look like specifically there.
04-04-2023
Requested a simple Java reproducer from the submitter.
30-03-2023