The Seleka rebels were said to be in control of Bangui, where witnesses said there was looting and sporadic gunfire.
Reports suggested the president had gone to a neighbouring state. France’s foreign minister confirmed his exit.
The rebels, involved in an on-off rebellion since December, say Mr Bozize failed to honour a peace deal.
One of the rebel leaders on the ground, Colonel Djouma Narkoyo, was quoted by AFP as saying: “We have taken the presidential palace. Bozize was not there.”
He said the rebels were planning to move on to the national radio station in Bangui where rebel leader Michel Djotodia planned to make a speech.
Intense gunfire was reported as rebels advanced through Bangui.
“The rebels control the town,” said a spokesman for the presidency, Gaston Mackouzangba. “I hope there will not be any reprisals.”
A Paris-based rebel spokesman said the rebel leadership was telling its fighters to restrain from “looting or score-settling”.
But Amy Martin of the UN’s humanitarian agency, OCHA, told the BBC World Service that residents in the capital had begun looting.
“The situation in town is chaotic in the sense than communities are looting properties, private properties, even a paediatric hospital we understand has been looted,” she said.
“Our main concern right now is at the community level, with the looting and the possible tensions between various ethnic groups.”
South African peacekeepers suffered some casualties have retreated to their barracks and are seeking safe passage to the airport, Ms Martin said.
She added that Bangui been without power since Saturday, and that this meant water had also been cut.
She also said the situation in the interior thought to be worse than in the capital, more than 170,000 estimated to have been displaced within the country and others fleeing to Chad and the Democratic Republic of Congo.
An unnamed presidential advisor told Reuters news agency that Mr Bozize had crossed the Oubangi River into DR Congo on Sunday morning.
Former colonial power France has called for an emergency meeting of the UN Security Council, and reportedly sent troops to secure the airport.
On Saturday, French officials warned their nationals in the country to stay at home.
France has some 1,250 troops in CAR.
The rebels joined a power-sharing government in January after talks brokered by regional leaders to end a rebellion they launched last year.
But the deal quickly collapsed, with the rebels saying their demands, including the release of political prisoners, had not been met.
BBC Africa editor Richard Hamilton says government soldiers have been unable to fend off the rebels because Mr Bozize fears being overthrown in a coup and is therefore wary of having a strong army.
He came to power himself in a military coup in 2003.
CAR, which has a population of about 4.5 million, has been hit by a series of rebellions since independence from France in 1960.
It is one of the poorest countries in Africa, despite its considerable mineral resources.