A statement from the South African presidency said Mr Mandela, 94, had been admitted just before midnight.
Mr Mandela spent 18 days in hospital in December undergoing treatment for a lung infection and gallstones.
He is widely regarded as the father of the nation for leading the struggle against apartheid.
Mr Mandela served as South Africa’s first black president from 1994 to 1999. However, his health has caused concern for some time.
A government spokesman told the BBC that Mr Mandela was conscious and was receiving the best possible medical treatment.
However, the BBC’s Andrew Harding in South Africa says the abrupt nature of Mr Mandela’s late-night admission is likely to raise concerns.
The government statement said President Jacob Zuma wished “Madiba” – as Mr Mandela is affectionately known in South Africa – a speedy recovery.
“We appeal to the people of South Africa and the world to pray for our beloved Madiba and his family and to keep them in their thoughts. We have full confidence in the medical team and know that they will do everything possible to ensure recovery,” President Zuma said.
It is the fourth time Mr Mandela has been admitted to hospital in just over two years.
He first contracted tuberculosis in the 1980s while detained on the windswept Robben Island where he served 18 of the 27 years he was imprisoned for sabotage.
His lungs are said to have been damaged when he worked in a prison quarry.
The treatment he received in December 2012 was his longest spell in hospital since leaving prison in 1990.
Earlier this month he spent a night in hospital following a check-up.
Mr Mandela retired from public life in 2004 and has been rarely seen in public since.
He lives in Qunu, a small rural village in Eastern Cape province, where he says he spent the happiest days of his childhood.