About once a month I walk around for 2-3 days with 5 lb leg weights on my legs ( strapped around my ankles ) and of course I drum with them on.
I don't think wearing ankle weights while drumming will help you. The weights would make it harder to pull your leg up and easier to push down, which is the exact opposite of what you should be doing if you want to improve your speed.
Maybe try putting a weight on the beater to make it harder to push down; that would improve your strength for sure.
But that's just my thought on the matter.
Hippie out.....