Police have found items of clothing and belongings believed to belong to the man, who passengers saw fall into the water about 124 kilometres south-east of Port Phillip Heads at 11.40pm.

Spirit of Tasmania I was travelling from Melbourne to Tasmania at the time of the incident, which sparked a massive search of a 247 square kilometre area.

The  Australian Maritime Safety Authority was notified at 12.10am and coordinated the search with two helicopters capable of searching for the man at night,  including the Victoria Police Air Wing, along with Spirit of Tasmania ferries and two cargo ships.

Water Police, four helicopters and the maritime authority's  dedicated search and rescue plane joined the search from first light.

But the plane arrived back in Melbourne shortly after 1pm without locating the man and a spokeswoman for the  maritime  authority confirmed the search has been suspended. 

The spokeswoman said the decision to suspend the search was based on expert medical advice that the man could not have survived past midday, taking into account various factors including water temperature and weather conditions.

Sergeant Paul Bartlett of the Water Police had earlier said the chances of the man surviving were becoming increasingly remote.

"Survival would be very limited in these conditions. It's one to two-metre swells, the water temperature is around 12 degrees, hypothermia would limit their capability of survival."

Sergeant Bartlett said some of the man's clothing and identification had been retrieved from the water.

The circumstances leading to the man going overboard remain unclear, but Tasmanian police will speak with witnesses upon the ship's arrival in Devonport and seek any CCTV footage that may assist with the investigation.

Sergeant Bartlett said passengers who saw the man go overboard alerted the ship's staff and they took immediate emergency action.

"He was seen by passengers going over the side. The ship did manoeuvres to locate that person and did sight the person in the water and applied safety equipment."

Sergeant Bartlett said the man's family has been notified.

A passenger aboard the ship posted on social media that all passengers were asked to gather in the main sitting area for three hours while the ferry joined the search.

Spirit of Tasmania I and Spirit of Tasmania II were released from the search on Saturday morning.

Spirit of Tasmania II arrived in Melbourne at noon while Spirit of Tasmania I was due to reach Devonport by 2.15pm.

Saturday night's scheduled crossings from Melbourne and Devonport will both proceed as normal.

Tasmania Police is assisting with the investigation but the death is not being treated as suspicious. 

Police will prepare a report for the coroner. 

The Age