Ha, figured out the issue ! I was actually expanding the alpha radius during my fill pass, which created those gaps.
So it's mostly working okay now, trying to adjust how I tweak the focus range to make it easier to play with (I like the idea of a start/stop positions).