As part of the work for JDK-8247514 and also investigation for JDK-8247272, I came to the realization that pc_to_symbol() lookup on OSX is very broken. There are many bugs, and I doubt it ever worked. symbol_to_pc() works fine, but that uses the completely separate OSX Symbolicator support, and it's implementation is not in any way related to pc_to_symbol().

Note the OSX pc_to_symbol() lookup is not even implemented when working on a live process. I'm not attempting to remedy that with this CR. This CR only focuses on fixing the existing pc_to_symbol() support for core files.

Note that pc_to_symbol() is not really needed as part of SA core functionality (symbol_to_pc() is critical for vtable lookups). It is used by clhsdb "whatis" command, and also the clhsdb "mem" command, which will try to display any native symbol near a given address. Note that neither of these commands currently work since they were written in javascript, which has been removed. However, "whatis" functionality is being added to "findpc" by JDK-8247514, and that is when I noticed this issue. I'll also eventually convert "mem" to java, so it would also run into this issue.

The main bug is in build_symtab(), where it tries to compute the size of the data that the symbol represents by looking at the offset for the next symbol in the symbols array, and comparing with the offset the current symbol. Basically "size of current symbol" == "offset of next symbol" - "offset of current symbol". This is faulty due to symbols not being sorted by offset. Because of this many symbols had extremely large or negative sizes. Then when a lookup was done, the lookup code often incorrectly thought the address fell within the bounds of the symbol when it did not. In fact most addresses matched on the first symbol of the first library, which was actually a symbol that represented a source file name, and due to some bad math had a negative size, which just messed things up even more.

nearest_symbol() is also not aware that the symbol table is not sorted, also resulting in it usually finding the wrong symbol. 

symbol_for_pc() also needs to be fixed to make sure it doesn't try to find a symbol in the wrong library. It doesn't know the size of a library, and as a result the code was trying to find the symbol in the first library checked, and if the symbol wasn't actually in that library it would match on the last symbol since there was no next symbol to check and see if the offset is greater. It needs to be fixed by first finding which library the symbol is in by taking advantage of the fact that libraries are sorted by address.