Would white vinegar ruin leather? cleaning a knife?

i found my grandfather's old knife and it had rusted so i soaked the blade in vinegar and that took care of all the rust on the blade. however there's still rust in the steel part in the middle of the stacked leather handle. can i soak the handle part without ruining the leather? if not, how can i remove the rust?