We report on a new scheme for efficient continuous-wave (CW) mid-infrared generation using difference frequency generation (DFG) inside a periodically poled lithium niobate (PPLN)-based optical parametric oscillator (OPO). The pump sources were two CW fiber lasers fixed at 1018 nm and 1080 nm. One worked as the assisted laser to build parametric oscillation and generate an oscillating signal beam while the other worked at low power (${\leqslant}3~\text{W}$) to induce DFG between it and the signal beam. The PPLN temperature was appropriately adjusted to enable OPO and DFG to synchronously meet phase-matching conditions. Finally, both low-power 1018 nm and 1080 nm pump beams were successfully converted to $3.1~\unicode[STIX]{x03BC}\text{m}$ and $3.7~\unicode[STIX]{x03BC}\text{m}$ idler beams, respectively. The conversion efficiencies of the 1018 nm and 1080 nm pumped DFG reached 20% and 15%, respectively, while their slope efficiencies reached 19.6% and 15%. All these data were comparable to the OPOs pumped by themselves and never realized before in traditional CW DFG schemes. The results reveal that high-efficiency frequency down-conversion can be achieved with a low-power near-infrared pump source.