Carbon nanotubes (CNT) and short carbon fibers were incorporated into an epoxy matrix to fabricate a high performance multiscale composite. To improve the stress transfer between epoxy and carbon fibers, CNT were also grown on fibers through chemical vapor deposition (CVD) method to produce CNT grown short carbon fibers (CSCF). Mechanical characterization of composites was performed to investigate the synergy effects of CNT and CSCF in the epoxy matrix. The multiscale composites revealed significant improvement in elastic and storage modulus, strength as well as impact resistance in comparison to CNT–epoxy or CSCF–epoxy composites. An optimum content of CNT was found which provided the maximum stiffness and strength. The synergic reinforcing effects of combined fillers were analyzed on the fracture surface of composites through optical and scanning electron microscopy (SEM).