This avoids some warnings, which were originally fixed by ignoring loads with a value of zero (see 636bedb207784db2d58b9986464f6a863677b59e / #3242).
Note that FifoCI will report some changes, but only on the first frame; these seem to be timing related as they don't happen if a different write is used to replace skipped ones.