He won elections in March partly on his tough stance on corruption.
He has vowed to trace and recover the “mind-boggling” sums of money that have been stolen from the oil sector.
Oil exports account for around 90% of Nigeria’s foreign currency earnings and because of low oil prices, the country faces tough economic challenges.
Mr Buhari, who took office at the end of May, is expected to announce his cabinet in the coming hours.
Over the last four months, he has been dealing directly with the top civil servants, who run the ministries.
He has already split the state-owned NNPC oil company into two entities in a bid to tackle corruption.
Issue of trust
The BBC‘s Will Ross in Lagos says there had been plenty of speculation that President Buhari would put himself in charge of the crucial oil ministry.
He made the announcement at the UN General Assembly in New York, confirming that a junior minister would take care of the day-to-day running of the ministry while he took overall charge of cleaning up the notoriously corrupt sector.
“We want to see what we have done in reducing the size and redeploying most of the management. We want to see the impact of that before we decide further,” he said.
In the 1970s and 1990s Mr Buhari held key positions in the oil sector so he has some relevant experience for the job, our correspondent says.