Sanchez, who is expected to be sworn in by Monday and appoint his cabinet next week, has promised to start talks with the Catalans but said he will not give them an independence referendum. That will mark the end of special controls from Madrid that were imposed on the northeastern region after separatists passed a unilateral declaration of independence in October.
In Berlin, a government spokesman said Germany hoped for a stable government in Madrid.
The reputation of Rajoy's Popular Party's was badly damaged by a court verdict last week that identified it as a beneficiary of a large kickbacks-for-contracts scheme. The incoming leader vowed to address Spaniards' "social urgencies" after years of austerity.
The heavyweights of Spain's Socialists had forced Pedro Sanchez' removal as their leader.
Podemos has already asked to be part of his new government.More news: Iran Foreign Affairs Minister Mohammad Javad Zarif arrives in India
More news: USA stocks tumble as Trump starts trade brawl against the world
More news: Trump Says He Wishes He'd Picked Another Attorney General
Referring to Mariano Rajoy's outgoing government, Rivera told reporters on Friday that "we had to censure this government, but not in this way". Rivera said his party would be "very attentive to the concessions" that are made.
The vote was carried by Sanchez' own PSOE party, which filed the motion and has 84 seats in the parliament, with the help of the leftist alliance Unidos Podemos, which holds 67 seats, and of several regional parties. Only one MP abstained from the vote.
Sanchez, who will be Spain's seventh prime minister since the country's return to democracy in the late 1970s, arrives in power after a spectacular turnaround in his political fortunes. When Rajoy admitted defeat, he told politicians that it was "an honor to leave a better Spain than I found". The final vote has been called for 0900 GMT.
An absolute majority of 180 lawmakers voted for the motion to loud applause and shouts of "Yes we can", converting Rajoy into the first prime minister to be ousted by such a vote since Spain transitioned to democracy in 1977.
The Basque Nationalist Party, whose five seats were key to Sanchez securing enough parliamentary backing, withdrew support from Rajoy after dozens of people linked to his centre-right People's Party (PP) were sentenced to decades in jail in a corruption trial.