The diplomatic argument erupted on June 15, when Bashir flew out of South Africa as world powers and activists urged the government to arrest him under a warrant from the global court on charges of masterminding genocide in Sudan's Darfur region.
"Cabinet decided that it will review South Africa's participation in the Rome Statutes of the International [Criminal] Court," Jeff Radebe, minister in the presidency, told reporters during a post-cabinet meeting briefing.
He added the government would only leave as a "last resort".
"Such a decision will only be taken when South Africa has exhausted all the remedies available to it," he said.
Al Jazeera's South Africa correspondent Fahmida Miller said the comments were in line with previous hints from government sources that ties to the ICC could be severed.
"South Africa's stance on the ICC could be an attempt to legitimise its actions to allow President Bashir to leave the country," our correspondent reported.
Bashir, who was in South Africa for a summit of the African Union, was allowed to takeoff even though a Pretoria court had issued an order banning him from leaving until the end of a hearing on his case.
As the row has simmered, top South African officials have said the country should cut ties with the ICC over its alleged bias towards Africa.
The ICC is "dangerous" and South Africa should withdraw from it, Gwede Mantashe, secretary-general of the ruling African National Congress, said on Monday.
A South African judge asked prosecutors on Wednesday to consider charging government officials over the decision to allow Bashir to leave.
South Africa's government was due to issue an affidavit in court on Thursday explaining why it allowed Bashir to leave but its contents were not expected to immediately be made public.
By Al Jazeera