The present paper investigates the influence of using two grooved channel geometries on the hydrothermal performance of a helical microchannel heat sink. The results were compared with those obtained for the plain channel geometry. Based on the results, the highest flow mixing is observed for the staggered grooved configuration. Moreover, the convective heat transfer coefficient enhances by nearly 70% and 17% for the MCHS with the staggered and parallel grooved channels, respectively, as compared to that with the plain channel. The influence of Reynolds number (Re) escalation on the convective heat transfer coefficient is insignificant, while a 21% increase is obtained when NF concentration is increased from 0% to 1%. Furthermore, the increase of φ in the grooved configurations is more effective than that in the plain configuration. In addition, the lowest thermal resistance and temperature uniformity factors were obtained for the staggered grooved configuration. Besides, the parallel grooved configuration represents the highest frictional entropy generation rates among the three studied geometries due to the considerable reduction of the fluid flow cross-section area between the groove ribs. The highest heat transfer coefficient and pressure drop are associated with the staggered grooved configuration. For the plain, parallel grooved, and staggered grooved configurations, the increase of Re from 500 to 2000 escalates the NF pressure drop by 2730%, 4420%, and 4460%, respectively. Therefore, the highest performance evaluation criterion of more than unity was obtained for the MCHS with the staggered grooved channels for the studied range of Re (500–2000). The numerical analysis was also performed to evaluate the influence of groove pitch on the hydrothermal performance of the staggered groove channel for the pitch range of 0.15 mm to 0.75 mm. The results demonstrated that the highest PEC is associated with the groove pitch of 0.75 mm.