That's called a "hot spot." These happen only on long exposures, as you've found, and are just a more extreme form of noise (since the longer exposure, the greater the noise, and beyond a certain point the noise is so great that it makes specks like what you're seeing).