This is a theory about something that hasn't yet been revealed - if I'm right, it's a spoiler. If I'm wrong, it doesn't matter. I'm almost certainly wrong. If you don't want to risk having a fairly unimportant bit of plot being revealed, though, don't read the next paragraph. There, I've warned you.

Could the payment for summoning a demon be the theft of one's voice? That would make ‘unspeakable’ a pun and be a real pain for spellcasters. It just seemed like an interesting thought and the green witch didn't speak after the ‘in-’ was cut short. Yeah, even as I write this it seems like I'm making too much of too little. But I could be right!

I was thinking that it was the loss of one's soul or death, but both of these are easily spoken of.

…Maybe it is having a demon constantly try to rip out your soul to use as it's plaything following you, so that you have to be on your guard.

I like it squirrel master, it is a cool concept.

You could be right, Squirrel, but it's also possible I used the word “unspeakable” as a substitute for “I'm not exactly sure what the cost is”. It's also within reason that I have an entirely different joke in mind.

It's also inside the realm of possibility that I resent the fact that you've come up with a better play on words then I had intended.

I await your future joke.

it is a rule of good writing, be vague about such things…

Whatever “unspeakable horror” the author fully describes will only be as such for a few people. Most will either find it distasteful (did we really need a detailed description of scat, rape, bestiality, etc; imagine showing 2girls1cup to an unjaded innocent?), or will find it not scary (yawn, yet another zombie… how droll) or even offensive to consider it scary. (wait, so is the author suggesting that a tentacle demon attacking a man is the unspeakable horror a form of demonizing homosexuality by the author?)

By simply leaving it as “unspeakable horror” we get to fill in the blank with whatever WE currently consider unspeakable; without boring, traumatizing, or offending ourselves.

Take the matrix for example. Matrix 1 had cool special effects, combined with non stop action and some vague references to what is actually going on… People filled in the blanks with cool things.
When we found out what was REALLY going on in Matrix 2 and 3 the whole thing became stupid.