To find out the gram weight of Lithium in your batteries you can contact the customer service department of the battery manufacturer. They can give you the definitive answer. However, you can roughly figure that a battery rated above about 130 Watt Hour will be right at 8 grams of Lithium content. No Lithium still camera battery will be anywhere near that rating. I carry big heavy Anton Bauer video camera batteries that are rated at 90 Watt Hours and they are less than 8 grams of Lithium content. Follow the rules regarding individual packing to protect the terminals from coming in contact with each other and carry your batteries in you carry-on bag and you will be fine.
Thanks--from this info, it looks like all still camera batteries will have no problem.