Bug ID: JDK-8185940 Web native compiled files not removed during gradle clean

Type: Bug
Component: javafx
Sub-Component: web
Affected Version: 8,9,10

Priority: P4
Status: Resolved
Resolution: Fixed
OS: generic
CPU: generic

Submitted: 2017-08-08
Updated: 2018-03-26
Resolved: 2017-09-20

JDK 10	JDK 8
10Fixed	8u171Fixed

gradle clean doesn't remove webview native files

Steps to reproduce 
1. compile web with "gradle -PCOMPILE_WEBKIT=true" 
2. "gradle clean"
3. "hg st -i" // should not have any files from modules/javafx.web/src/*

Changeset: 56669b0b2efb Author: ghb Date: 2017-09-20 07:22 +0530 URL: http://hg.openjdk.java.net/openjfx/10-dev/rt/rev/56669b0b2efb
20-09-2017
+1
12-09-2017
+1
12-09-2017
[~mbilla] webrev posted in my earlier commets is updated (it was supposed to be .02 instead of .01) . Do update your review comments.
12-09-2017
Updated webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.02 replaced "../../" with $projectDir as suggested. Compiled and tested on Windows, OS X and Linux.
12-09-2017
[~ghb] I still see the relative path in webrev.01. Can you check again? commandLine("perl", "../../src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig")
08-09-2017
[~mbilla] Providing complete path might fail in windows Cygwin , Will test and update.
07-09-2017
[~ghb] Are you incorporating the above suggested change by Arun ? If yes, can you post the new webrev..
06-09-2017
+1 with nit: Instead of relative path, use `projectDir` variable. + commandLine("perl", "../../src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig") commandLine("perl", "$projectDir/src/main/native/Tools/Scripts/set-webkit-configuration", "--$webkitConfig")
31-08-2017
[~kcr][~arajkumar], Updated webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.01/ Modified the current working directory in build.gradle from ~rt/modules/javafx.web/src/main/native to ~rt/moduels/javafx.web/build/{platform}/ Updated .hgignore. Tested on all 3 platform.
31-08-2017
Given the above, I am OK with the current fix as long as you file a (P4) follow-on bug for the cmake-generated files.
29-08-2017
[~ghb] Probably we can change the working directory to modules/javafx.web/build/{platform} before invoking build-webkit script from build.gradle.
29-08-2017
Relocating "__cmake_systeminformation" directory is not possible with current cmake version as it is creating __cmake_systeminformation from Current working directory ref 1. They might give option to in future. Currently webkit is ingoring the directory in their .gitignore ref 2. Verified on all three platform (i.e hg st -i modules/javafx.web/src show empty except __cmake_systeminformation). 1 : https://gitlab.kitware.com/cmake/cmake/commit/31a700188b6864341fdcbd22cd4181dbe6ace671?view=parallel#cc54196ff9e89133d1a2255ec4d2c487942a6719_2915_2922 2 : webkit master : https://github.com/WebKit/webkit/blob/master/.gitignore#L27
29-08-2017
In that case, this looks like a partial fix since I would have expected the cmake files to be removed as well. Do you want to file a separate issue for that? Also, have you tested on all three platforms to ensure that there are no other files leftover? As a follow-on to this, it would be good to file an RFE to have all build-time files generated in the build directory. This could be implemented by copying some source files into a build/gensrc dir or similar if that's the only way to accomplish it.
16-08-2017
[~kcr]"Have you verified that there are no other files left over after 'gradle clean' ?" None of the .pyc and .dat files are shown after gradle clean (it doesn't matter with gradle clean as its not generated) rt$hg st -i shows cmake information directory "modules/javafx.web/src/main/native/__cmake_systeminformation" which is added in ignore list "JDK-8172495"
14-08-2017
Thanks [~ghb] for the verification. As 2) is not possible (separate directory), im fine with "PYTHONDONTWRITEBYTECODE" approach.
11-08-2017
Did a time (performance) between with and without python bytecode set for all the python script which generates derived source, WasmHeader/validator, regex table, etc (42 places). With byte code : 8.967 sec (Total build time 48m55.665s), 9.385 sec (Total build time 39m59.658s) with out byte code : 13.891 sec (Total build time 49m53.532s), 11.92 sec (Total build time 38m2.901s) I could see ~5 Seconds difference. Do you still agree to take different approach (As you are suggesting). 1. when we do "gradle clean", can we just remove (rm .pyc) ? we can add in build.gradle for web clean "find modules/javafx.web/src/main/native -iname .pyc \| xargs rm", or hg st -i modules/javafx.web/src/main/native \| xargs rm, This will fail if the developer has taken only the copy of source code (via zip or other mode, i.e non mercural). 2. (OR ) Can we keep pyc files in a seperate directory and when we do "gradle clean" , we just remove this directory. This is not possible with python 2.7 Ref : https://www.python.org/dev/peps/pep-0304/ & https://stackoverflow.com/questions/471928/way-to-have-compiled-python-files-in-a-separate-folder/16476434#16476434
11-08-2017
I don't really like the idea of running 'find' (and definitely not 'hg status -in' for the reason you mentioned) as part of gradle clean, so let's go with your first approach since the difference in time to compile is in the noise. Have you verified that there are no other files left over after 'gradle clean' ?
11-08-2017
Thanks Arun, Will remove icudt.dat while commit. diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -76,5 +76,3 @@ # ignore cmake internal generated files modules/javafx.web/src/main/native/__cmake_systeminformation -# ignore icu data library -icudt*.dat
10-08-2017
lgtm. Please remove "icudt*.dat" from .hgignore file.
10-08-2017
webrev : http://cr.openjdk.java.net/~ghb/8185940/webrev.00 Solution : set "PYTHONDONTWRITEBYTECODE" environment variable which will disable python bytecode. icudt51l.dat is extracted to 'rt/modules/javafx.web/build/<platform>/Release/icu/data/ Tried setting "PYTHONDONTWRITEBYTECODE" in OptionsJava.cmake instead of environment variable in build.gradle, which didn't work for custom targets. set(PYTHON_EXECUTABLE "$PYTHON_EXECUTABLE -B") set(ENV{PYTHONDONTWRITEBYTECODE} "1")
10-08-2017
Though the approach of PYTHONDONTWRITEBYTECODE setting to true is good (as chrome also doing), i have one concern : As we are avoiding generation of python compiled bytecode files (pyc), I feel, pyc files are good to have, as python need not have to compile python files again and again and can save startup time? Can we have something like below approaches: 1. when we do "gradle clean", can we just remove (rm *.pyc) ? 2. (OR ) Can we keep pyc files in a seperate directory and when we do "gradle clean" , we just remove this directory.
10-08-2017