When the leaves fall, they seldom fall vertically. Often they swirl and rotate, picking up speed on their journey.
It depends also on the height of the tree (leaves falling from a tall tree have more time to pick up more speed), air humidity, density and shape of the leaf, the lens you are using and how far you are from leaf trajectory.
I would estimate that to freeze the falling leaves in the air, you'll have to shoot at least at 1/200s. If you want to have them defined, yet include some blur to capture the motion, 1/50s to 1/100s should work well. Anything slower and you'll see only soft brown lines.
Whatever shutter speed you choose (in the recommended range), you'll find that not all leaves move at the same speed. Best thing is to take many pictures and vary the shutter speed.
To illustrate the point, attached is a photo of a scantily dressed man encountered on Collins Avenue in South Beach Miami. I photographed him from inside a car through the windshield with a 50mm lens at 1/40s. He moves along the street at about 4-5km/h and surprisingly, even at that slow shutter speed, his body and face were captured reasonably sharp. As he waves with his right hand, I would estimate that movement at about 25-40km/h or in other words at 5-8m/s, which would be approximately the speed of a falling or swirling leaf. So if you shoot at 1/40s, you'll see a well defined shape with some blur, and if you shoot at 1/100s or faster, you should be able to freeze the leaves (well, most of them) in the air.