In the present paper, a numerical and experimental study has been performed to investigate the heat transfer and pressure drop properties of Al2O3 and SiO2/Base oil nanofluid flow in a helical tube under constant heat fluxes. The nanofluids were prepared by using SN-300 base oil as a working fluid for 0.05%, 0.1%, 0.3%, and 0.5% mass concentrations. The experiments were carried out with two constant heat fluxes of 3950 W/m2 and 5280 W/m2 at 30, 40, 50, and 60 ◦C. The impact of various parameters, including flow Reynolds number, fluid temperature, two different nanoparticles, and various weight concentrations, on the heat transfer factor and pressure drop of the flow, are investigated. To examine the impact of different geometries on the heat transfer rate, simulations were conducted over four different helical tubes by changing the pitch circle diameter and steps. The results indicated that the heat transfer factor and pressure drop were increased by utilizing nanofluid instead of the base fluid. In addition, in the same mass concentration, the Al2O3 nanofluid increased the heat transfer coefficient more than the SiO2 nanofluid. The maximum heat transfer rate corresponds to 0.5% mass concentrations of Al2O3 and SiO2 nanofluids, which are 41.4% and 27.3% more than base oil, respectively. The helical tube compared to the straight tube increased heat transfer by 19.5%. Moreover, heat transfer was improved by 6% and 16.5%, respectively, by reducing helical pitch and pitch circle diameter.