United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-8007275 : build-infra: Create final-images target

Submit Date:
Updated Date:
Project Name:
Resolved Date:
Affected Versions:
Fixed Versions:

Related Reports

Sub Tasks

The new build system tried to make things easier, but along the way, some things instead got more complicated. One such issue is the location of the final image after a build.

In the open, the jdk repo now puts images in <build-dir>/images/j2*-image. In an effort to move away from the overlay images of solaris 64bit, this image is always a full image. Instead the overlay image is put in <build-dir>/images/j2*-overlay-image.

With the closed additions, we have deploy and install repos, both making changes to the images and to support effective incremental builds, we didn't want to allow these repos to pollute the same directories. Instead copies are made for each of them. So we have <build-dir>/deploy/j2*-image and <build-dir>/install/j2*-image. For a full closed release build on all platforms, the result can be found in the install dir. A debug build does not build install however, and so the result is found in the deploy dir, except for solaris 64bit, which doesn't build deploy and so there the image is found in the images/j2*-overlay-images dir.

This is all more complicated than anyone would really like to know and to make things simpler, I would like to create a target "final-images" similar to the jprt specific "bundles" target, that copies the correct images to a new location that will always be the same, for all configurations. I propose <build-dir>/final-images/j2*-image.

URL:   http://hg.openjdk.java.net/jdk8/jdk8/rev/dea045cc48ca
User:  katleman
Date:  2013-02-07 01:08:36 +0000

URL:   http://hg.openjdk.java.net/jdk8/build/rev/dea045cc48ca
User:  erikj
Date:  2013-02-04 14:03:19 +0000

There shouldn't be a need, but as I see it, there currently is. At least if we want to support some kind of incremental builds. I don't trust incremental builds where files are transformed in place by different parts of the build process.

If (when) we attack the deploy and install repos to behave better, we might very well be able to remove these extra copies. When we stop using the overlay image structure for 64bit solaris, we remove another need for this.

On my linux build machine, the current difference in size is 4.0GB for the old build and 4.9GB for the new. The final-images will add some more, but the target is only meant for RE to run. Most people will not be building either deploy or install anyway.
Yes, partial rebuilds shouldn't be necessary and that is mostly true for the parts we did attack. Deploy and install, however, do not build incrementally at all, they seem to always rebuild everything. My hands are itching to attack these too, but that will not happen in jdk8 as far as I know.

So far, the increased disk usage hasn't been a problem and as I understand it, the extra copies of the images won't take up space in the archives either. 
One of the selling points of the new build was that it would be quick enough that incremental builds should not be an issue. But I guess that wasn't considering deploy/install.

The disk space concern is mainly for JPRT as I would expect it would be doing these full builds as well. That's not to say it won't be an issue for hudson.
At least for me, the separation has been very valuable. Cleaning either install or deploy has meant only cleaning that, and I'm then able to rebuild just the repo I want, knowing it gets the exact same input data from the jdk as the last time I built. Comparing this to the old build, where the only way to know you got the same input was to do a complete clean build.
This still seems too complicated. The "image" is the product of the build. If you include deploy and install then they modify that build product. There shouldn't be multiple images all slightly different, nor need for a "final image". We don't have enough disk space to produce all these copied images to start with!


Hardware and Software, Engineered to Work Together