The building sector is the major energy consumer, accounting for over 40% of global energy demand. Heating and cooling together with domestic hot water energy consumption are estimated to account for 60% of the required energy for buildings’ maintenance and operation. Energy recovery is a suitable technique to tackle high energy consumption in the building. In this study, a new layout of heat recovery units installation (i.e., primary and secondary) is investigated. The main objective of this study is to reduce energy consumption in an air handling unit through the exergy analysis. Owing to adding heat recovery units, cooling and heating coil loads reduced by 7.8% and 43%, which in turn decreased the total required load of AHU by 17.84%. From the viewpoint of the second law and based on the results, incorporating the primary and secondary heat recovery units into the base AHU in hot and dry climate regions led to decrease in the total irreversibility up to 26.29%, while in hot and humid climate this figure is 14.25%. Consequently, the positive effect of using heat recovery units in the hot and dry climate region is superior to the hot and humid one.