I googled it, but was not able to find a good answer.
it wasn't a one time thing
It depended what state you lived in and if you were free or in bondage. And if you owned property. Each state had its own voting laws, for example in 1800 if you were white, but didn't own property, you couldn't vote.
Each state has its own felon voting laws, and all but two — Maine and Vermont are the only states that allow prisoners to vote — have some kind of disenfranchisement law that prevents current or former offenders from casting their ballots. The degree of severity varies dramatically by state and crime. The report found that nearly half of the disenfranchised population lives in the eleven states where voting rights for ex-felons are only conditionally restored.
Even after you have served your sentence/probation many states deny felons the right to vote.