Pulsars in relativistic binary systems have emerged as fantastic natural laboratories for testing theories of gravity, the most prominent example being the double pulsar, PSR J0737–3039. The HTRU-South Low Latitude pulsar survey represents one of the most sensitive blind pulsar surveys taken of the southern Galactic plane to date, and its primary aim has been the discovery of new relativistic binary pulsars. Here we present our binary pulsar searching strategy and report on the survey’s flagship discovery, PSR J1757–1854. A 21.5-ms pulsar in a relativistic binary with an orbital period of 4.4 hours and an eccentricity of 0.61, this double neutron star (DNS) system is the most accelerated pulsar binary known, and probes a relativistic parameter space not yet explored by previous pulsar binaries.