The apology failed to end criticism of Mr Dutton's comments, with Papua New Guinean Prime Minister Peter O'Neill saying on Sunday he was concerned by the "most unfortunate" remarks.

A boom microphone picked up Mr Dutton joking with Prime Minister Tony Abbott about the plight of Pacific Island nations facing rising sea levels as a result of climate change on Friday.

Commenting that a meeting in Canberra was running late, Mr Dutton quipped that it was running to "Cape York time". Mr Abbott, who had just returned from a meeting with Pacific Island leaders about climate change, replied: "We had a bit of that up in Port Moresby."

Mr Dutton then said: "Time doesn't mean anything when you're about to have water lapping at your door."

Mr Dutton originally refused to answer questions on the incident but on Sunday said he regretted his remarks.

"Obviously it was a private conversation - I should have realised the mike was there," Mr Dutton told Sky News. "I didn't; it was directly behind me. I made a mistake and I apologise to anyone who has taken offence to it. It was a light-hearted discussion with the PM and I didn't mean any offence to anyone.

"If anyone has taken offence they should accept my apology. I'm disappointed that it allowed for a distraction from what was a very good policy announcement."

Mr Dutton said the government's decision to accept 12,000 Syrian refugees and participate in air strikes against Islamic State fighters in Syria "should have been the story for the week and I'm sorry for that".

In a statement issued three hours after Mr Dutton's apology, Mr O'Neill said he hoped Mr Dutton's comments would focus attention on the threat posed to Pacific islanders by climate change.

"People around the Pacific are living in fear with each high tide of storm," he said. "People are being forced off the land where their families have lived for thousands of years.

"Connection to the land is very important for Pacific people so having to leave their land is heartbreaking for many people."

Mr O'Neill is chair of the Pacific Islands Forum, which Mr Abbott attended in PNG last week.

Labor foreign affairs spokeswoman Tanya Plibersek said Mr Dutton had simply apologised for "being caught".

"He should be apologising to Pacific leaders and to Aboriginal Australians in Cape York," she told the ABC's Insiders. "He seems to have insulted millions of people in one go and he hasn't apologised for that."

Social Services Minister Scott Morrison said he was disappointed the media had focussed on Mr Dutton's comments rather than substantial issues such as the resettling of Syrian refugees.

Tony de Brum, the Marshall Islands Foreign Minister, said he was aghast by Mr Dutton's remarks.

"Next time there is a king tide and waves are battering my home, I'll ask Peter Dutton over and we'll see if he's still laughing," he said.

Cape York traditional owner Gerhardt Pearson said the reference to "Cape York time" was aimed at portraying Aboriginal people as "lazy good-for-nothings".

"We are constantly burdened with the view espoused by the likes of Dutton; it's one of soft bigotry and low expectations, and it continues to dominate policy responses," he said in a statement.

"Dutton's joke using this tired old stereotype reminds me of how white superintendents ran our lives, dressed in their safari jackets and white helmets, pipe in their mouths, hands in their pockets; and how they would look down at my hard-working grandfather, mother or brother, as if they were his slaves."

- with Adam Gartrell