Mugabe, Africa's oldest president aged 90, shook hands with Abel Aziz in front of fellow leaders to applause at the AU summit in the Ethiopian capital.
"By electing me to preside over this august body, with full knowledge of the onerous responsibility that lies ahead, I humbly accept your collective decision," Mugabe said, as he thanked leaders for granting him the position.
Mugabe, a former liberation war hero who is Africa's third-longest serving leader, is viewed with deep respect by many on the continent.
But he is also subject to travel bans from both the United States and European Union in protest at political violence and intimidation.
The former guerrilla leader who has ruled Zimbabwe since independence in 1980, is accused of crushing opponents to ensure his ZANU-PF party won every election for more than three decades.
Mugabe described his memories of attending the AU's forerunner in 1963, the Organization of African Unity (OAU), also in Addis Ababa.
"Africa has come a long way since then," he said.
- AFP