Congress on Thursday denied that then prime minister PV Narasimha Rao was in any way involved in the Babri demolition even as it conceded there was anger among some people that the Centre had been negligent in taking action.