To do their job, the pilot waves must travel to the far slit and return in time to influence the electron's path depending on whether that slit is open.