The current idea I wanna try is doing a copy of the first downsample (full or smaller res) and blend it into the next frame downsample. Just to see if it helps with the spatial/temporal aliasing.
Will figure out ghosting issues afterward if it becomes promising.