The silent protest was aimed at Brett Walker, who shared an accountancy office with his late father for decades, and is viewed as key to unlocking the mystery of what happened to the missing life savings of 70, mainly ageing, Hunter retirees.

More than 30 people packed the public gallery with many of them holding signs bearing the words "I don't recall", "I can't recall" and "I honestly can't recall".

Mr Walker, the head of Hunter-based Active Accounting Group, spent much of Tuesday repeatedly telling the hearing he could not recall details.

His memory difficulties continued on Wednesday as he readily admitted - more than 20 times in the first 10 minutes - that he could not recall when asked questions about being a director of one of his father's companies.

Registrar Thomas Morgan was forced to warn the jeering public gallery that if there were anymore outbursts people would be removed from the courtroom.

Mr Walker failed to recall conversations he had with his father about one client’s investment and could not recall what documents he reviewed when preparing another client’s financial records, who had also invested with Ray Walker.

"I don't recall. I don't recall. I just don't recall," he said.

Mr Walker was certain, however, that he was a proactive accountant, who was service orientated and looked after the needs of his clients.

He was also sure that a decision in late 2008 to dissolve the partnership with his father had nothing to do with him being concerned about his father’s investment activities.

Ray Walker, a veteran Newcastle accountant, killed himself in 2015 after his Ponzi scheme was uncovered.

Lake Macquarie’s Richard Galloway – who lost more than $300,000 – described the proceedings as “hugely disappointing”.

“All we wanted were some direct answers and we’ve come away with nothing,” he said.

“So many people have lost their life savings. It’s just wrong, so wrong.

“It’s very frustrating that we are no closer to getting the answers we deserve.”