JDK-8043507 : (smartcardio) javax.smartcardio.CardTerminals.list() fails on MacOSX
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.smartcardio
  • Affected Version: 7u51
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: os_x
  • Submitted: 2014-05-20
  • Updated: 2016-06-13
  • Resolved: 2014-05-23
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 JDK 9
7u72Fixed 8u20Fixed 9 b15Fixed
Description
The function javax.smartcardio.CardTerminals.list() sometimes fails when called from an app running on MacOSX.

It can be reproduced with ./sun/security/smartcardio/TestDefault.java manual test.
To make the bug more reproducible I've run it in a loop.
The failure occurs once on every few hundred cycles.

java(29245,0x100f21000) malloc: *** mach_vm_map(size=140698833653760) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Exception in thread "main" java.lang.OutOfMemoryError
	at sun.security.smartcardio.PCSC.SCardListReaders(Native Method)
	at sun.security.smartcardio.PCSCTerminals.list(PCSCTerminals.java:80)
	at javax.smartcardio.CardTerminals.list(CardTerminals.java:72)
	at TestDefault1.main(TestDefault1.java:47)
Comments
The existing regtest ./sun/security/smartcardio/TestDefault.java was used to reproduce the failure. The failure was intermittent, and the test failed only 1 or 2 times of every 1000 runs. To make the failure more observable, I ran the test in the loop. There's no guarantee that the failure can be reproduced on every machine. The bug only manifests itself if the garbage in the uninitialized variable happened to be large enough to cause OutOfMemery error.
12-06-2014
Is it covered by existing regression tests? Please provide testing recommendations.
11-06-2014
The problem seems to be due to that CALL_SCardListReaders(_, _, _, &size) expects the size variable to be of size uint32_t on os x, but we provide a 64 bit int instead. As a result, the higher bits may contain garbage upon return, and we try to allocate a too large block of memory. The simplest solution may be to initialize size to zero before the call.
20-05-2014