Bug ID: JDK-4180839 Allow serialized form to be generated in normal run of Javadoc

Type: Enhancement
Component: tools
Sub-Component: javadoc(tool)
Affected Version: 1.2.0

Priority: P4
Status: Closed
Resolution: Fixed
OS: generic
CPU: generic

Submitted: 1998-10-13
Updated: 2002-06-27
Resolved: 2001-06-22

Other
1.4.0 beta2Fixed

Name: dk30142			Date: 10/12/98

NOTE: We have implemented a fix for the main problem;
however this fix requires a separate run of Javadoc
with -private flag.  This synopsis of this bug
was changed on April 18, 2000 to indicate completing
the rest of this bug.
.....

The serialized form information given does not
provide enough information to create a fully
compatible set of core classes, as only the
serialized form of public classes is given.

I am writing an implementation of the Collections
classes for a clean room set of class libraries.
The problem lies in methods such as
java.util.Collections.reverseOrder() which return
private inner classes, and then state that their
return value is Serializable. There are a number
of such methods in the Collections class, and
the AbstractList.subList() method is another
(which may not state explicitly that the return
value is serializable, but it certainly ought to
be!)

I don't know whether Collections is the only API
to suffer from this problem, or whether it is
more general. The solution probably requires work
from both the Collections team AND the javadoc
team.

However, it is a major problem for inter-VM RMI
and for JavaBeans. Please spec it, and fast - you
can't afford for this not to make it into the FCS!
(Review ID: 38641)
======================================================================

CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: merlin-beta2 FIXED IN: merlin-beta2 INTEGRATED IN: merlin-beta2 VERIFIED IN: merlin-rc1
14-06-2004
PUBLIC COMMENTS PackageDoc extened as follows: package com.sun.javadoc; public interface PackageDoc extends Doc { ... /** * Get all classes (including Exceptions and Errors) * and interfaces. * @since J2SE1.4. * * @return all classes and interfaces in this package, filtered to include * only the included classes if filter==true. / ClassDoc[] allClasses(boolean filter); /* * Get all included classes (including Exceptions and Errors) * and interfaces. Same as allClasses(true). * * @return all included classes and interfaces in this package. */ ClassDoc[] allClasses(); }
10-06-2004
EVALUATION Assigned to joe.fialli. Put Josh and Robert on interest list. doug.kramer@Eng 1998-10-12 Two actions items: 1) run javadoc -private to include private classes in the serialized form page and then strip out the links to other docs 2) turn this into a bug to have the serialization form include private classes without requiring the -private option For #1 above, I added a new gnumake target to 1.2 Docs.gmk called "releasedocs-serialpage" that creates a tmp directory, generates javadoc with -private option, then runs a perl script to strip out all links (except in the navigation bar), then copies this file to the docs/api directory and deletes the tmp directory. doug.kramer@Eng 1998-10-19 The email discussion of this is at: http://javaweb.eng/docs/writing/central/api-specs/serialized-form-private.html The list of classes to include are at: http://javaweb.eng/docs/writing/central/api-specs/serialized-form-private-1.2.html Changes were made to the Docs.gmk file: <ws>/build/solaris/makefiles/Docs.gmk Note this has 18 JDC votes. doug.kramer@Eng 1998-11-09 Robert wants to look at this to see how difficult it would be to fix for 1.2.2 Atul, perhaps you could look at this first, then present it to Robert. doug.kramer@Eng 1998-12-14 Since we don't have time to implement this for inclusion in Javadoc for Kestrel, Robert suggests that we write a separate doclet to perform this task that has a deadline after the Kestrel code freeze, meaning Atul could perhaps write it in late June or July, so it would still be ready for generating the Kestrel FCS docs. Then we could fold it into the standard doclet after kestrel FCS, when the workspace unfreezes again. doug.kramer@Eng 1999-04-27 We have resolved action item #1 above by adding @serial include and @serial exclude, running Javadoc with the -private option and copying serialized-form.html to the normal Javadoc run (without -private). I have build this double-run process into the -releasedocs target of Docs.gmk. What remains to resolve is action item #2, which I am doing now: Turn this into a bug to have the serialization form include private classes without requiring the -private option Robert says this will be possible with the Javadoc built on the new compiler. So I changed the synopsis from: Serialized form information is incomplete--private classes are missing to: Allow serialized form to be generated in normal run of Javadoc doug.kramer@Eng 2000-04-25 Can I please have a specification of exactly what you would like to change in the implementation of the javadoc API? neal.gafter@Eng 2001-02-21 Please see the specification at the bottom of "Suggested Fix". doug.kramer@Eng 2001-02-22 ###@###.### wrote: <quote> The StubMaker doclet has in effect solved this problem. It processes all the fields and determines if they are serializable. If all of the these hold: * it is serializable. * it is not primitive * the class type is hidden (i.e. not visible, as defined by the javadoc switches -public, -protected, etc.) * the classes' package matches one of the packages on the command line then: * the class is added to the list of classes to process. * if the class is not serializable, an error is generated * if the "@serial include" tag is not present, an error is generated In that way, otherwise hidden classes are processed, but only if they are serializable and included. The process is of course recursive, so all required classes are found. The same algorithm could be added to the standard doclet to produce a valid Serialized Form, without requiring the use of the -private switch. </quote> ###@###.### 2001-02-23 Is there some reason this approach would be inappropriate for the standard doclet? neal.gafter@Eng 2001-03-02 Eric, Jamie and I just spoke yesterday, and this solution is possible. However, the addition of the allClasses(private) method would allow the doclet to do an additional check. Jamie will implement this, but am leaving it assigned to to implement allClasses(private) or in case you somehow determine allClasses(private) is not needed. Here is Eric's email: (he also enclosed the source code for Jamie) Here is the procedure for generating the serialized form in the standard doclet, and for detecting developer errors in the process. Setup: 1) Use allClasses() to initialize the collection of classes for the serialized form --this step gets all visible classes 2) Run through that collection, removing any that have @serial exclude. 3) Save the excluded classes in a hash table. Class processing: 1) Use serializableFields to get the array of FieldDocs for serialized fields 2) If definesSerialPersistentFields() is true then the array contains one item, which has a special API, otherwise, the array contains the default non-transient fields (whether hidden or visible). 3) Process the array of fields: a) FieldDoc.type().asClassDoc() gets the class b) If the class is visible, check to see if the class is in the excluded table. If it is, generate a warning: [[non-transient field uses an excluded class]] c) If the class is hidden, check to make sure it specifies @serial include. --if it doesn't, generate a warning: [[non-transient field uses a hidden, non-included class]] --otherwise, add it to the collection of of classes for the serialized form 3) When all other processing is completed, process the collection of classes for the serialized form page. Additional Error Check When allClasses(private) becomes available, process the list it returns to do the following: 1) If the class is visible, ignore it. 2) If the class is hidden, see if @serial include was specified. 3) If @serial include was specified, see if the class was actually used (if it was, it will be in the collection of classes for the serialized form page) 4) If the class is not present in that collection, generate a warning: [[class specified with @serial include that is not used by any serialized field]] doug.kramer@Eng 2001-03-15 As most of the functionality is possible with the current API, I'm not committing this for Merlin. neal.gafter@Eng 2001-03-19 Neal, Eric and Jamie have concluded that Eric's algorithm will not work for all cases. (See "Comments" for more details.) Therefore, we are requesting to add allClasses(private) to the Doclet API to solve this problem. doug.kramer@Eng 2001-05-10
10-05-2001
SUGGESTED FIX The following is the solution we arrived at for JDK 1.2.0, which requires human intervention to delete 6 classes at the end. This is copied from the <ws>/build/solaris/makefiles/Docs.gmk file. We need a better solution for 1.2.2, where this list can be generated automatically. # # The following list is the private classes that should be # included in the Serialized Form specification. See: # http://javaweb.eng/docs/writing/central/api-specs/serialized-form-private-1.2.html # PRIVATE_SERIAL_CLASSES = ./../../src/share/classes/java/util/Arrays.java ./../../src/share/classes/java/util/Collections.java ./../../src/share/classes/java/util/PropertyPermission.java ./../../src/share/classes/java/io/FilePermission.java ./../../src/share/classes/java/security/AllPermission.java ./../../src/share/classes/java/security/BasicPermission.java ./../../src/share/classes/java/security/Permissions.java ./../../src/share/classes/java/security/UnresolvedPermissionCollection.java ./../../src/share/classes/java/net/SocketPermission.java # The following is the complete list of private classes to be included above. # # In java/util/Arrays.java # Arrays.ArrayList # In java/util/Collections.java # Collections.ReverseComparator # Collections.SynchronizedCollection # Collections.SynchronizedList # Collections.SynchronizedMap # Collections.SynchronizedSet # Collections.SynchronizedSortedMap # Collections.SynchronizedSortedSet # Collections.UnmodifiableCollection # Collections.UnmodifiableList # Collections.UnmodifiableMap # Collections.UnmodifiableMap.UnmodifiableEntrySet # Collections.UnmodifiableSet # Collections.UnmodifiableSortedMap # Collections.UnmodifiableSortedSet # In java/util/PropertyPermission.java # PropertyPermissionCollection # In java/io/FilePermission.java # FilePermissionCollection # In java/security/AllPermission.java # AllPermissionCollection # In java/security/BasicPermission.java # BasicPermissionCollection # In java/security/Permissions.java # PermissionsHash # In java/security/UnresolvedPermissionCollection.java # UnresolvedPermissionCollection # In java/net/SocketPermission.java # SocketPermissionCollection # When documenting the above source files, you will get these # extra files which you should manually remove: # AllPermission, BasicPermission, FilePermission, # Permissions, PropertyPermission, SocketPermission, ....................... The proposed addition to the Doclet API is: com.sun.javadoc.PackageDoc.allClasses(String access) where access is one of: "public", "protected", "package", "private" This method gets all included classes (including Exceptions and Errors) and interfaces at the specified access level or higher. Thus, allClasses("private") would return all classes included on the command line regardless of their access modifier. doug.kramer@Eng 2001-02-22
22-02-2001