Pure cotton rag mat?, more than likely it's a seed, or part of a seed. The best solution, is to take a very sharp point from a no 11 blade, x-acto or similar and try to pick out the spot. If you can get the spot out, then you can try erasing around the area where the spot was to even it out a bit.
I mainly use Bainbridge, or Crescent mats here in the US and have seen this issue with both brands, mainly on the 100% pure cotton rag, but have also seen it with the bright core line also.
Paul C