no one to remain

55 But if ye will not drive out the inhabitants of the land from before you; then it shall come to pass, that those which ye let remain of them shall be pricks in your eyes, and thorns in your sides, and shall vex you in the land wherein ye dwell.

The reason God demanded that every inhabitant of Canaan should be removed

Thorns in the Side
If faithful to Him, God intended that Israel should go on conquering and to conquer...
But regardless of their high destiny, they chose the course of ease and self-indulgence; they let slip their opportunities for completing the conquest of the land; and for many generations they were afflicted by the remnant of these idolatrous peoples, that were, as the prophet had foretold, as "pricks" in their eyes, and as "thorns" in their sides. Numbers 33:55. {PP 544.2}

Thinking about this topic, I remembered the mixt multitude that came out from Egypt with the people of Israel. They were "pricks" and "thorns"

Num. 11:4 And the mixt multitude that was among them fell a lusting: and the children of Israel also wept again, and said, Who shall give us flesh to eat?

Num. 11:5 We remember the fish, which we did eat in Egypt freely; the cucumbers, and the melons, and the leeks, and the onions, and the garlick:

Num. 11:6 But now our soul is dried away: there is nothing at all, beside this manna, before our eyes.