JDK-8185940 : Web native compiled files not removed during gradle clean
  • Type: Bug
  • Component: javafx
  • Sub-Component: web
  • Affected Version: 8,9,10
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2017-08-08
  • Updated: 2018-03-26
  • Resolved: 2017-09-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10 JDK 8
10Fixed 8u171Fixed
gradle clean doesn't remove webview native files

Steps to reproduce 
1. compile web with "gradle -PCOMPILE_WEBKIT=true" 
2. "gradle clean"
3. "hg st -i" // should not have any files from modules/javafx.web/src/*

Changeset: 56669b0b2efb Author: ghb Date: 2017-09-20 07:22 +0530 URL: http://hg.openjdk.java.net/openjfx/10-dev/rt/rev/56669b0b2efb



[~mbilla] webrev posted in my earlier commets is updated (it was supposed to be .02 instead of .01) . Do update your review comments.

Updated webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.02 replaced "../../" with $projectDir as suggested. Compiled and tested on Windows, OS X and Linux.

[~ghb] I still see the relative path in webrev.01. Can you check again? commandLine("perl", "../../src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig")

[~mbilla] Providing complete path might fail in windows Cygwin , Will test and update.

[~ghb] Are you incorporating the above suggested change by Arun ? If yes, can you post the new webrev..

+1 with nit: Instead of relative path, use `projectDir` variable. + commandLine("perl", "../../src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig") commandLine("perl", "$projectDir/src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig")

[~kcr][~arajkumar], Updated webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.01/ Modified the current working directory in build.gradle from ~rt/modules/javafx.web/src/main/native to ~rt/moduels/javafx.web/build/{platform}/ Updated .hgignore. Tested on all 3 platform.

Given the above, I am OK with the current fix as long as you file a (P4) follow-on bug for the cmake-generated files.

[~ghb] Probably we can change the working directory to modules/javafx.web/build/{platform} before invoking build-webkit script from build.gradle.

Relocating "__cmake_systeminformation" directory is not possible with current cmake version as it is creating __cmake_systeminformation from Current working directory ref 1. They might give option to in future. Currently webkit is ingoring the directory in their .gitignore ref 2. Verified on all three platform (i.e hg st -i modules/javafx.web/src show empty except __cmake_systeminformation). 1 : https://gitlab.kitware.com/cmake/cmake/commit/31a700188b6864341fdcbd22cd4181dbe6ace671?view=parallel#cc54196ff9e89133d1a2255ec4d2c487942a6719_2915_2922 2 : webkit master : https://github.com/WebKit/webkit/blob/master/.gitignore#L27

In that case, this looks like a partial fix since I would have expected the cmake files to be removed as well. Do you want to file a separate issue for that? Also, have you tested on all three platforms to ensure that there are no other files leftover? As a follow-on to this, it would be good to file an RFE to have all build-time files generated in the build directory. This could be implemented by copying some source files into a build/gensrc dir or similar if that's the only way to accomplish it.

[~kcr]"Have you verified that there are no other files left over after 'gradle clean' ?" None of the .pyc and .dat files are shown after gradle clean (it doesn't matter with gradle clean as its not generated) rt$hg st -i shows cmake information directory "modules/javafx.web/src/main/native/__cmake_systeminformation" which is added in ignore list "JDK-8172495"

Thanks [~ghb] for the verification. As 2) is not possible (separate directory), im fine with "PYTHONDONTWRITEBYTECODE" approach.

Did a time (performance) between with and without python bytecode set for all the python script which generates derived source, WasmHeader/validator, regex table, etc (42 places). With byte code : 8.967 sec (Total build time 48m55.665s), 9.385 sec (Total build time 39m59.658s) with out byte code : 13.891 sec (Total build time 49m53.532s), 11.92 sec (Total build time 38m2.901s) I could see ~5 Seconds difference. Do you still agree to take different approach (As you are suggesting). 1. when we do "gradle clean", can we just remove (rm *.pyc) ? we can add in build.gradle for web clean "find modules/javafx.web/src/main/native -iname *.pyc | xargs rm", or hg st -i modules/javafx.web/src/main/native | xargs rm, This will fail if the developer has taken only the copy of source code (via zip or other mode, i.e non mercural). 2. (OR ) Can we keep pyc files in a seperate directory and when we do "gradle clean" , we just remove this directory. This is not possible with python 2.7 Ref : https://www.python.org/dev/peps/pep-0304/ & https://stackoverflow.com/questions/471928/way-to-have-compiled-python-files-in-a-separate-folder/16476434#16476434

I don't really like the idea of running 'find' (and definitely not 'hg status -in' for the reason you mentioned) as part of gradle clean, so let's go with your first approach since the difference in time to compile is in the noise. Have you verified that there are no other files left over after 'gradle clean' ?

Thanks Arun, Will remove icudt.dat while commit. diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -76,5 +76,3 @@ # ignore cmake internal generated files modules/javafx.web/src/main/native/__cmake_systeminformation -# ignore icu data library -icudt*.dat

lgtm. Please remove "icudt*.dat" from .hgignore file.

webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.00 Solution : set "PYTHONDONTWRITEBYTECODE" environment variable which will disable python bytecode. icudt51l.dat is extracted to 'rt/modules/javafx.web/build/<platform>/Release/icu/data/ Tried setting "PYTHONDONTWRITEBYTECODE" in OptionsJava.cmake instead of environment variable in build.gradle, which didn't work for custom targets. set(PYTHON_EXECUTABLE "$PYTHON_EXECUTABLE -B") set(ENV{PYTHONDONTWRITEBYTECODE} "1")

Though the approach of PYTHONDONTWRITEBYTECODE setting to true is good (as chrome also doing), i have one concern : As we are avoiding generation of python compiled bytecode files (pyc), I feel, pyc files are good to have, as python need not have to compile python files again and again and can save startup time? Can we have something like below approaches: 1. when we do "gradle clean", can we just remove (rm *.pyc) ? 2. (OR ) Can we keep pyc files in a seperate directory and when we do "gradle clean" , we just remove this directory.