Well it's finally started printing after hours of trying different file sizes and paper size settings.
I also found a 9880 FAQ page on the epson site asking about borderless printing that had a similar chart to the file size one on page 90, but this one had different values for the file size you should send and also includes a 4th column called "custom size ( with retain size)". The manual alludes to this third option but never mentions how or what to do. Which made the whole thing extra confusing. This FAQ chart was from dec 2007 according to the page, and it seems my manual was printed in September 2007, so it seems some changes were made. The PDF manual is the same as my printed one though. You'd think they'd at least update that.
In the end I found a dpreview thread about a similar error message when trying to do what I'm doing on sheet paper on a 3880, and several members said they'd some times have to reload the paper 10 times to have it aligned properly. So then your paper skew comment finally clicked and I reloaded the paper this time ignoring the roll paper alignment line on the printer, just pulled the roll tight so it would presumably feed as straight as possible. This got it to print straight away. Though when feeding it this way, the left edge of paper was right on the alignment line and the right edge was about 1/4" below. So I guess we'll see if it does in fact print straight.
I'd always assumed that once the paper takes up and the head measures its alignment that its doing so off the cut edge and not the sides of the roll, otherwise what's the point of having the horizontal alignment line. All my non borderless prints have come off straight relative to the roll edges and the cut edge in the past. So none of this is really making sense to me. Maybe it'll print all crooked this time.