The G1 collection cycle does an additional young-only GC between the cleanup pause and the first mixed gc. In some cases, in relatively tight heaps, this makes the mixed gcs start too late, causing to-space overflows and in the end results in undesired full gcs.
There does not seem to be a good reason to do the young-only gc after GC cleanup at all.
Fix this to switch to mixed gcs right after the marking cycle completes.
An "intermediate" fix could be to decrease the IHOP slightly.
A program that shows this problem is e.g. CRM Fuse.