The dead officer has been named as PC Keith Palmer, 48, a husband and father.
PM Theresa May said the attack was "sick and depraved" and struck at values of liberty, democracy and freedom of speech.
The attacker has not been named by police.
Acting Deputy Commissioner and head of counter terrorism at the Metropolitan Police, Mark Rowley, said they think they know who he is and that he was inspired by international and Islamist-related terrorism, but gave no further details.
The attack unfolded at about 14.40 GMT when a single attacker drove a car along a pavement over Westminster Bridge, near the Houses of Parliament in central London, killing at least two people and injuring many more.
The car then crashed into railings outside the Houses of Parliament.
The attacker, armed with a knife, ran to Parliament where he was confronted by the police. PC Palmer - who was not armed - was then stabbed and killed.
The attacker was shot dead by armed officers.
Mr Rowley paid tribute to PC Palmer, saying: "He was someone who left for work today expecting to return home at the end of his shift, and he had every right to expect that would happen."
"Heartbroken" former colleague, Conservative MP James Cleverly, paid tribute to the "lovely man" he had known for 25 years. The pair had served together in the Royal Artillery before PC Palmer became a policeman.
Foreign Office minister Tobias Ellwood - a former Army officer whose brother died in the Bali terrorist bombing in 2002 - attempted mouth-to-mouth resuscitation of Pc Palmer.
"Heartbroken" former colleague, Conservative MP James Cleverly, paid tribute to the "lovely man" he had known for 25 years. The pair had served together in the Royal Artillery before PC Palmer became a policeman.
Foreign Office minister Tobias Ellwood - a former Army officer whose brother died in the Bali terrorist bombing in 2002 - attempted mouth-to-mouth resuscitation of Pc Palmer.
She paid tribute to the "exceptional men and women" of the police force who responded to the attack, saying: "We will all move forward together, never giving in to terror and never allowing the voices of hate and evil to drive us apart."
The prime minister added: "The location of this attack was no accident.
"The terrorist chose to strike at the heart of our capital city where people of all nationalities, religions and cultures come together to celebrate the values of liberty, democracy and freedom of speech."
She is expected to make a statement in the Commons later.
The husband of murdered MP Jo Cox said the "name I will remember" from the Westminster attack was that of PC Keith Palmer - not the attacker.
London mayor Sadiq Khan said: "My message to those that want to harm us and destroy our way of life is: You won't succeed; you won't divide us; we won't be cowed by terrorists."
In latest developments:
- There will be more armed and unarmed officers on duty in London and across the country as a "precautionary measure"
- The prime minister said the UK terror threat level would remain at severe - its second highest - meaning an attack is "highly likely"
- Westminster underground station was shut and remained open for interchange only
- Home Secretary Amber Rudd urged everyone to remain calm but be vigilant and if they see anything they are concerned about report it to the police
- A group of French schoolchildren were on the bridge and three were injured
- 13 students from Edge Hill University in Lancashire were also caught up in the incident - two were taken to hospital and described as walking wounded; two others had minor injuries
- People worried about family and friends can call the police casualty bureau on: 0800 056 0944 or 0207 158 0010. Anyone with images or footage of the incident is urged to send them to www.ukpoliceimageappeal.co.uk
The carnage on Westminster Bridge and inside the grounds of Parliament is the attack that security chiefs here in the UK have long been preparing for.
Terrorism looks not just to kill and maim - but to create panic and such a sense of disorder that it rocks a city or nation to its foundations.
And this attacker sought to do so in as low-tech way as is possible.
The days when terrorism meant large, complex bombs and months of planning are gone: Western security agencies - particularly MI5 and its partner agencies - are very, very good at identifying those plots and disrupting them.
The longer it takes to plan such an attack, the more people who are involved, the more chances there will be for security services to learn what is going on.
Read more from Dom.
Eyewitness Rick Longley said: "We were just walking up to the station and there was a loud bang and a guy, someone, crashed a car and took some pedestrians out.
"They were just laying there and then the whole crowd just surged around the corner by the gates just opposite Big Ben.
"A guy came past my right shoulder with a big knife and just started plunging it into the policeman.
"I have never seen anything like that. I just can't believe what I just saw."
An eye witness, Radoslaw Sikorski, a senior fellow at Harvard's Centre for European Studies, posted a video to Twitter showing people lying injured in the road on Westminster Bridge.
