Australia's peak doctors' group has called on the federal government not to forego mental health for disaster funding.

The group wants mental health funding of $5 billion over the next four years despite pressure on the budget caused by recent natural disasters.

The Australian Medical Association says Labor's response to the growing crisis in mental health has been inadequate to date.

The doctors' call for action comes after Prime Minister Julia Gillard was criticised this week for side-stepping mental health when striking a deal with the states and territories on Sunday at the COAG meeting.

Ms Gillard had previously declared that mental health would be a focus of Labor's second term in office.

"(Numerous experts) have pointed to the need for increased mental health funding and reform of mental health service delivery arrangements," the AMA stated in its budget submission,

"(But) to date responses from the government have been inadequate."

The medical association acknowledges there has been some action on early intervention for young people suffering with mental illness.

However, it argues "there continues to be significant unmet need in mental health care and major gaps in service provision that the government must address in this budget".

Association president Andrew Pesce said it was clear the summer floods, Cyclone Yasi and Western Australia's bushfires had placed pressure on the May budget.

But he argued investing more in health "will underpin the rebuilding and restoration of productivity, cohesion and confidence in the Australian population".

The AMA states the extra $5 billion should help fund community campaigns to increase awareness of mental health issues and reduce stigma.

The cash injection was also needed to fund more early intervention programs, more community-based care, more mental health beds in and out of hospitals, and an increased mental health workforce including psychiatrists.