Right, Rob, but things still were somewhat touch-and-go here less than ten years after the war of 1812. The Unites States actually bought Florida from Spain. By the way, the Castillo still was a U.S. military installation, called Fort St. Mark, right up until I was 3 years old.
Just for fun, here are some tourist snaps from the Castillo.