The All Progressives Congress (APC) won 19 of the 28 governor posts in results declared from Saturday's elections.
It is the biggest defeat for the Peoples Democratic Party (PDP) since military ruled ended in 1999.
It lost the presidential poll last month for the first time.
APC leader Muhammadu Buhari will be inaugurated on 29 May, after he defeated President Goodluck Jonathan.
The PDP's decline has been reinforced by results from Saturday's elections for governors and state assemblies, reports the BBC's Bashir Saad Abdullahi from the capital, Abuja.
President Goodluck Jonathan's People's Democratic Party (PDP) had controlled the federal and most state governments since the end of military rule in 1999 but has suffered sweeping losses in the ongoing election cycle.
Jonathan's loss to Buhari in the presidential vote two weeks ago was the first ever democratic change of power at the federal level since Nigeria gained independence from Britain in 1960.