US President Joe Biden on Sunday led his countrymen in remembering the victims of the September 11 terrorist attacks 21 years ago.

"I hope we'll remember that amidst those dark days, we dug deep, we cared for each other, and we came together," Biden said in his address to the nation from the Pentagon memorial, which was one of the sites of attacks of the Al-Qaeda terrorists on this day 21 years ago in addition to the twin towers in New York.

Vice President Kamala Harris and the Second Gentlemen Dough Emhoff flew to New York to participate in the 9/11 remembrance ceremonies at the twin-tower. The First Lady Dr Jill Biden travelled to Shanksville in Pennsylvania at the Flight 93 crash site.

Joined by Defence Secretary Lloyd Austin and Chairman of the Joint Chiefs of Staff Mark Milley, Biden participated in the wreath laying ceremony at the Pentagon Memorial.

Commemorating the 21st anniversary of the deadliest attack, Biden quoted from the message of the Queen Elizabeth II, who died early this week, immediately after the terrorist attack that killed almost 3,000 people including several Indian Americans.

"Grief is the price we pay for love, Biden said. This is a day not only to remember, but a day of renewal and resolve, he said, amidst pouring rain at the Pentagon.

"I know for all those of you who have lost someone, 21 years is both a lifetime and no time at all," Biden said at the National 9/11 Pentagon Memorial in Arlington, Virginia.

"It's good to remember. These memories help us heal, but they can also open up the hurt and take us back to that moment when the grief was so raw," he said.

"It's not enough to stand up for democracy once a year or every now and then. It's something we have to do every single day. So, this is a day not only to remember, but a day of renewal and resolve for each and every American and our devotion to this country," he added.

Biden in his speech referred to the recent killing of al-Qaeda leader Ayman al-Zawahiri who was killed in a drone strike earlier this year in Afghanistan. The US "will not rest, we'll never forget, we will never give up," he said. And now Zawahiri can never again threaten the American people," he added.

Just over one year ago, America's war in Afghanistan came to an end. "Yet we continue our relentless focus on combating terrorist threats to our nation. And earlier this summer, the United States delivered justice to Ayman al-Zawahiri, the leader of al-Qaeda, Austin said.

"So make no mistake. America's determination to keep our country safe will never waver. And neither will America's determination to bring justice to those who attack our citizens, he added.

"Surrounding us today are 184 steel benches, each bearing the name of a person murdered in the attack on the Pentagon. And every night, 184 lights come on, lighting up each bench just as the people who they honour lit up the lives of those who loved them, said the defence secretary.