A new investigation was opened in April.
Brostrom told AFP that she hoped the panel's report would lead to Mr Sidibe's ouster.
"I want to believe that UN Secretary General (Antonio Guterres) sees the danger that the UNAIDS executive director represents for the whole UN system and that he takes this opportunity to demonstrate that his notion of zero tolerance of harassment is not just an empty slogan," she said.
Multiple civil society organisations that work on HIV/AIDS have previously called on Mr Sidibe to resign, but the agency maintained Friday that he had no plans to leave.
"The Executive Director is firmly focused on the future. He (is) fully aware that there is a lot of work to do -- across all levels of the organisation --and he is determined to lead that transformation," agency spokeswoman Sophie Barton-Knott told AFP.
The decision on whether to force Mr Sidibe out likely rests with the agency's British-led oversight body, the Programme Coordinating Board, which meets next week.
But the expert panel detailed an environment that will certainly escalate pressure on him to go.
"The Executive Director ... has created a patriarchal culture tolerating harassment and abuse of authority and in his interviews with the Panel he accepted no responsibility for actions," it said.
The panel also raised doubts about Mr Sidibe's ability to lead a reform effort, describing the solutions he proposes as "superficial and insufficient."
"Our inevitable conclusion from the review is that the state in which we find the organisational culture of UNAIDS is something for which the leadership of the organisation must be responsible and held accountable."
Paula Donovan from the Code Blue pressure group, which has led the effort to expose rot at UNAIDS, called on Guterres to act immediately.
"In 30 years, I have never heard of an independent report that delivered such a scathing indictment of internal UN leadership," she said, referring to the panel report.