Thiophene and its derivatives have been extensively used in organic electronics, particularly in the field of polymer solar cells (PSCs). Significant research efforts have been dedicated to modifying thiophene-based units by attaching electron-donating or withdrawing groups to tune the energy levels of conjugated materials. Herein, we report the design and synthesis of a novel thiophene derivative, FE-T, featuring a monothiophene functionalized with both an electron-withdrawing fluorine atom (F) and an ester group (E). The FE-T unit possesses distinctive advantages of both F and E groups, the synergistic effects of which enable significant downshifting of the energy levels and enhanced aggregation/crystallinity of the resulting organic materials. Shown in this work are a series of polymers obtained by incorporating the FE-T unit into a PM6 polymer to fine-tune the energetics and morphology of this high-performance PSC material. The optimal polymer in the series shows a downshifted HOMO and an improved morphology, leading to a high PCE of 16.4% with a small energy loss (0.53 eV) enabled by the reduced non-radiative energy loss (0.23 eV), which are among the best values reported for non-fullerene PSCs to date. This work shows that the FE-T unit is a promising building block to construct donor polymers for high-performance organic photovoltaic cells.