JDK-8242530 : [macos] Some audio files miss spectrum data when another audio file plays first
  • Type: Bug
  • Component: javafx
  • Sub-Component: media
  • Affected Version: 8u251,openjfx14
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: os_x
  • Submitted: 2020-04-11
  • Updated: 2020-06-09
  • Resolved: 2020-04-16
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 Other
8u261Fixed openjfx11.0.8Fixed
Related Reports
Relates :  
Description
To reproduce this, run the following on macOS:

1. Run Ensemble8
2. Run the "Audio Bar Chart" sample  (or "Advanced Media")
3. Run the "Audio Area Chart" sample
BUG: No audio spectrum data will be shown in the area chart

If you run the "Audio Area Chart" sample first after starting Ensemble, it works correctly.

It works correctly on Windows and fails on macOS (I tried both 10.13.6 and 10.15.4 and it fails on both).

This appears to be a regression from the removal of CoreAudio, JDK-8232589. If I do a build from the commit right before CoreAudio was removed, it works. If I then do a build from right after CoreAudio was removed, it fails.
Comments
Changeset: e82046ec Author: Alexander Matveev <almatvee@openjdk.org> Date: 2020-04-16 22:26:51 +0000 URL: https://git.openjdk.java.net/jfx/commit/e82046ec
16-04-2020

https://github.com/openjdk/jfx/pull/184
15-04-2020

We do actually posting messages to wrong message handler. Problem is that it looks like GStreamer shares instance of base class (my assumption was it is unique for each instance of element). GST_ELEMENT_GET_CLASS will return same pointer for different instances of spectrum element.
15-04-2020

Before CoreAudio removal we sending event to both old event loop thread and new one. This is why it works, but I still do not think we should post event to two event loop threads. AMDEBUG EventQueueThread() Thread[Thread-159,5,main] AMDEBUG CJavaPlayerEventDispatcher:Init(): m_PlayerInstance 0x7f84b7d29708 areJMethodIDsInitialized: 1 AMDEBUG addAudioSpectrumListener() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@66ca3a46 this: com.sun.media.jfxmediaimpl.platform.osx.OSXMediaPlayer@2105551b audioSpectrumListeners.size(): 0 Debug (0:00:01:786): Setting player to READY state Debug (0:00:01:951): Bad pixel format: '420v' Debug (0:00:01:951): Falling back on video format: 2vuy Debug (0:00:02:221): ** Frame size changed (540, 208) AMDEBUG HandleAudioSpectrumEvents() audioSpectrumListeners.size(): 1 this: Thread[Thread-159,5,main] AMDEBUG HandleAudioSpectrumEvents() onAudioSpectrumEvent() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@66ca3a46 AMDEBUG EventQueueThread() Thread[Thread-332,5,main] AMDEBUG CJavaPlayerEventDispatcher:Init(): m_PlayerInstance 0x7f84b7d29838 areJMethodIDsInitialized: 1 Error (0:00:06:503): gst_caps_set_simple: assertion 'IS_WRITABLE (caps)' failed AMDEBUG addAudioSpectrumListener() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@56f4a859 this: com.sun.media.jfxmediaimpl.platform.gstreamer.GSTMediaPlayer@c9d7c1a audioSpectrumListeners.size(): 0 AMDEBUG HandleAudioSpectrumEvents() audioSpectrumListeners.size(): 1 this: Thread[Thread-159,5,main] AMDEBUG HandleAudioSpectrumEvents() onAudioSpectrumEvent() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@66ca3a46 AMDEBUG HandleAudioSpectrumEvents() audioSpectrumListeners.size(): 1 this: Thread[Thread-332,5,main] AMDEBUG HandleAudioSpectrumEvents() onAudioSpectrumEvent() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@56f4a859
15-04-2020

Still not sure if it is regression from CoreAudio removal or something we had for long time, but did not reproduced before CoreAudio removal. Trace: Debug (0:00:00:744): CGstAVPlaybackPipeline::CGstAVPlaybackPipeline() Debug (0:00:00:744): CMedia::CMedia() AMDEBUG EventQueueThread() Thread[Thread-6,5,main] AMDEBUG m_SendAudioSpectrumEventMethod: 0x7fa8006bea48 klass: 0x7fa8013876c0 Debug (0:00:00:965): CGstAVPlaybackPipeline::~CGstAVPlaybackPipeline() --> GStreamer pipeline gets dealoocated, since it cannot play MP4 and we switched to AVFoundation. AMDEBUG EventQueueThread() Thread[Thread-217,5,main] AMDEBUG addAudioSpectrumListener() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@7c43494f this: com.sun.media.jfxmediaimpl.platform.osx.OSXMediaPlayer@3453a2d0 audioSpectrumListeners.size(): 0 AMDEBUG HandleAudioSpectrumEvents() audioSpectrumListeners.size(): 1 this: Thread[Thread-217,5,main] AMDEBUG HandleAudioSpectrumEvents() onAudioSpectrumEvent() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@7c43494f --> We sending spectrum event to Audio Bar Chart AMDEBUG EventQueueThread() Thread[Thread-444,5,main] Error (0:00:06:550): gst_caps_set_simple: assertion 'IS_WRITABLE (caps)' failed AMDEBUG addAudioSpectrumListener() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@34bd1154 this: com.sun.media.jfxmediaimpl.platform.gstreamer.GSTMediaPlayer@63b5da98 audioSpectrumListeners.size(): 0 --> Switched to Audio Area Chart AMDEBUG HandleAudioSpectrumEvents() audioSpectrumListeners.size(): 1 this: Thread[Thread-217,5,main] AMDEBUG HandleAudioSpectrumEvents() onAudioSpectrumEvent() listener:javafx.scene.media.MediaPlayer$_SpectrumListener@7c43494f --> Spectrum event being handled by old event loop thread. See Thread-217, should be Thread-444 and listener 34bd1154.
15-04-2020