JDK-8165852 : (fs) Mount point not found for a file which is present in overlayfs
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 8u172,9
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: x86_64
  • Submitted: 2016-09-07
  • Updated: 2019-03-19
  • Resolved: 2018-01-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 8
11 b01Fixed 8u202Fixed
Related Reports
Duplicate :  
JDK-8194283 - java/nio/file/FileStore/Basic.java fails with java.io.IOException: Mount point not found
Relates :  
JDK-8165323 - (fs) Files.getFileStore fails with "Mount point not found" in chroot environment
Relates :  
JDK-8166162 - (fs) Cannot obtain FileStore if device of path is not in /proc/mounts
Description
FULL PRODUCT VERSION :
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
[root@6b8ab63d2271 /]# uname -a
Linux 6b8ab63d2271 4.6.0-0.bpo.1-amd64 #1 SMP Debian 4.6.3-1~bpo8+1 (2016-07-13) x86_64 x86_64 x86_64 GNU/Linux

[root@6b8ab63d2271 /]# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

EXTRA RELEVANT SYSTEM CONFIGURATION :
Host System: Debian Jessie
Runs Docker version 1.12.1, build 23cf638 started with --storage-driver=overlay
Error does occur inside of the docker container which is centos 7

A DESCRIPTION OF THE PROBLEM :
When running java inside of a docker container which root fs is an overlayfs

e.g. from mount:  overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay/7f380aa866f1411985b0e3d9533e89f0907e11adb81c2c1e20a5733fd3112f33/root,upperdir=/var/lib/docker/overlay/58f650cc4005e9becf01c20b2d8144a372115d11add9e70774322232e3fef590/upper,workdir=/var/lib/docker/overlay/58f650cc4005e9becf01c20b2d8144a372115d11add9e70774322232e3fef590/work)

And accessing a file like /home/someuser/somefile.txt

An Exception will be thrown.


A very similar Case seems to exist with btrfs and subvolumes:
https://github.com/elastic/elasticsearch/issues/14337 and 
https://github.com/elastic/elasticsearch/issues/12018


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Start a docker container where dockerd was started with --overlfay and try to access files via Files.getFileStore(path)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The Result should be / (if there is no other mount)
ACTUAL -
java.io.IOException: Mount point not found

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.io.IOException: Mount point not found
        at sun.nio.fs.LinuxFileStore.findMountEntry(LinuxFileStore.java:91)
        at sun.nio.fs.UnixFileStore.<init>(UnixFileStore.java:65)
        at sun.nio.fs.LinuxFileStore.<init>(LinuxFileStore.java:44)
        at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:51)
        at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:39)
        at sun.nio.fs.UnixFileSystemProvider.getFileStore(UnixFileSystemProvider.java:368)
        at java.nio.file.Files.getFileStore(Files.java:1461)
        at MyMain.main(MyMain.java:8)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class MyMain {

	public static void main(String[] args) throws IOException {
		System.out.println(Files.getFileStore(Paths.get(args[0])));
	}
}

---------- END SOURCE ----------
Comments
RFA for jdk8u backport: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2018-July/007713.html
24-07-2018
The problem as it manifests in btrfs is essentially the same situation as observed for overlayfs in docker: the parent of a path component has a different device ID than that of the current path component and so the loop of the upwards traversal towards the root is broken out of but there is no entry for the current path component in /proc/mounts. This was observed to occur specifically for the case of a file contained in a btrfs sub-volume which is not itself explicitly mounted. If the sub-volume is mounted via `mount` it shows up in /proc/mounts and the error does not occur. This failure might be due to the nature of btrfs sub-volumes and unrelated to Docker per se.
30-09-2016
This problem is caused by breaking out of the loop in step 2 in LinuxFileStore.findMountEntry() when the parent's device (stat st_dev) is not equal to the device of the current path but there is no entry for the parent path in the set of mounted file systems listed in /proc/mounts. This was observed to occur in Docker containers running in an Ubuntu 16.04 amd64 VM for the overlay and overlay2 storage drivers but not for the aufs storage driver for the same test conditions. The btrfs and devicemapper storage drivers have not yet been tested. It could be argued that this is a bug in the overlayfs storage drivers by virtue of the inconsistency between what stat(2) and perhaps other system calls return versus what is present in /proc/mounts. Nonetheless it might be worth investigating a benign workaround which would enable this case without breaking others.
29-09-2016