A common method for hot lights is to use a heat-absorbing glass (which can be almost any glass) between the lamp and polarizing film. Your fresnel is already acting as a heat buffer but I would still be cautious with the Lee filter.
I would clip a piece of polarizing film (approx 12 x12 inches) at the end of the barn doors. If there is still too much heat, I would get a piece of 1/4 inch glass (water-clear if available) the size of the barn door scrim holder and have that cut in half. Example would be 8x8 inch scrim slot now would hold two pieces of 4x8 inch heat-absorbing glass. This is fairly standard in the Hollywood area and it works for open-face hot lamps as well.
I hope this helps.