My solution to the challenge of building a consistent categorical system has been to acknowledge that “genre” itself does not require a single set of discriminating principles. This allows “formal” categories (long/short-form prose fiction [including viewpoint-convention subtypes] and poetry, drama, and their format-rooted sub-categories) to co-exist with content categories of all sorts (love stories, success stories, crime stories, survival stories, et infinite cetera). For our purposes, it’s useful to have metaphysical categories–that is, to separate “realism” or “mimetic fiction” from “the fantastic,” with the understanding that while fiction is always subjunctive (I think I’m stealing from Delany here), some conjectures are metaphysically profound enough to justify a binary divide. Which leads to a second divide: the fantastic itself can be split into stories that take place in a world that follows rules like X (where X=a rational-materialist cosmos) as distinct from those that require Y (a supernaturalist cosmos). And as Paul W. points out, a serious Christian or Muslim or Rastafarian might locate X and Y in a manner that, say, I would not. Science fiction is fantastic (not-true-to-historical/mimetic realism) but materialist or at least non-supernaturalist. Until Charlie Stross starts playing “Magic, Inc.” games with Lovecraft and letting the Elder Gods leak into our world, or one uncomfortably like it.
I will admit that when I sit down at a blank page, I do think about whether it’s a short story or a novel.
I think about length, but not about genre content, except in a marketing context.
Because other than as a way of identifying where a story might find a home, I don’t give a rat’s ass what anyone calls it.
I’d like to suggest that certain “Secret Histories”, as well as some conspiracy type fiction (Dan Brown, maybe even Robert Anton Wilson), and arguably some near-future thrillers, do at least try to cross the line to the fantastic, and then cross it back to mimetic — or, alternately, to suggest that when they crossed the line they weren’t, really.
There are also those novels that seem to treat the fantastical element as heavily metaphorical — maybe Susan Palwick’s Flying in Place, for one example? The question there, again, is “has the line really been crossed?” But surely Palwick (as I believe she stated in an NYRSF piece) knew very well she was playing with genre when she wrote that book, for all that it appeals very much to non-SF readers (if my wife is any indication).
For another example — what about Karen Joy Fowler? I follow John Clute in reading it as SF, but I think many readers don’t see that aspect of it.
That’s probably just me playing a game, though — I don’t think it really disputes Gary’s point.
Brief comments on the main thrust of this discussion, that I think largely echo what others here who are not writers of fiction have said, like Rusell Letson.
I too am — to a fault, at times — a categorizer, a maker of lists, a labeler, a sorter. I think it helps me understand what I’m reading, and to see where writers (perhaps unconsciously) connect. But I am aware too that that can lead me into danger — to, as some have suggested, missing some of what a writer is trying. It can also lead to laziness — to a quick and dirty positioning of a book without really taking it apart.
And I’ve had writers push back against labels. (Nick Mamatas, for instance, bitterly — and probably to an extent fairly — complained when I suggested he wrote slipstream.) I think a lot of writers resist labels partly because they seem to restrict their personal originality — to limit them, maybe. And that’s sensible, I dare say, in their act of creation. But I think it still makes sense — if we take care — for critics, after the fact, to try to see what a writer has actually done, partly by labeling it.
As for the term Speculative Fiction, I’ve never been fond of it, but if it is used, by all means use it in the Heinlein sense.