G1 has the notion of a "reserve" set by the G1ReservePercent parameter. The idea behind the reserve is that G1 should always try to leave the reserve memory free in case a collection copies unexpectedly more objects and requires more to-space than expected / predicted. Currently, the reserve is only taken into account when calculating the target number of eden regions to be allocated between an evacuation pause and a subsequent one.
There are two situations which might cause G1 to eat into the reserve:
1) Eden expansion due to the GC locker (6994056: G1: when GC locker is active, extend the Eden instead of allocating into the old gen) - the regions the eden will be expanded by might eat into the reserve.
2) Humongous allocations might fill up the heap between two evacuation pauses and the eden region target number will not be adjusted accordingly.
We should look into taking the reserve into account more dynamically to minimize the chance of eating into it.
An additional improvement could by the default value of the reserve. Currently, it is set to a very generous / conservative value (10% of the heap). Maybe we can reconsider this and, when the user does not set it explicitly, we can maybe make it less conservative or adjust it according to the current heap parameters.