Hmmm, OK. While in some sense these are "conflicting" responsibilities, I think virtually all of these examples are cases of degree, and what I mean here is that in the ideal case, the player can satisfy both constraints simultaneously.
And in some sense I contend that this is still a case of identification. It is a tautology to say, "these two duties are potentially in conflict." What is much more difficult is to say, "these two duties are in conflict
right now." Only then can you move on to "So how do we resolve it?"
Let's take one of your examples:
"DM, make the player's challenges difficult" vs "DM, make the player's challenges survivable"
First off, while the term "survivable" is pretty straight-forward, the term "difficult" is pretty loaded. What constitutes "difficult?" Should the player's character be somehow "damaged" or "diminished" (in resources or what have you)? Is that the metric we're using? Or is it something fictionally related (i.e. it takes a lot of fictional maneuvering, with the real-world resource spent being a player's time)? By extension, how do we identify when a challenge is "too difficult" or "too easy"? And once we've identified what metric we're using, what unit of the metric constitutes "difficult?" Half dead? Mostly dead? Financially inconvenienced? Impoverished?
Aside and complication - if we're using dice (or any other element of randomness) to resolve situations in-game, the DM may not actually
control these things at all. He or she can slant things one way or the other, but if you consistently roll for crap, even an "easy" encounter might leave your character grievously injured.
So, the heart of the issue is one of determining
when the issue is really an issue. How do the players signal to the DM, "This is too hard," or "this is too easy," or "I don't even want to attempt this because I'm pretty certain my character will die trying." In a very real sense, this is an issue of cueing at work, because that's what's really going on - the players and/or DM need to somehow convey the message that something is an
appropriate level of difficulty.
Most traditional ("old school") games do this implicitly. If you don't make challenges survivable and the players' characters keep dying, eventually you'll end up with no players. Similarly, if the challenges are too easy, the players are left with no real sense of accomplishment, and they'll drift away from the game out of boredom.
So how do we make these cues
explicit? How do we structure the rules such that the DM can ratchet up the difficulty but that the PCs have an out of things go pear-shaped?
Dogs in the Vineyard has a great mechanic for this - escalation. When an in-game conflict starts, maybe there's not too much at stake. Some harsh words, maybe a little pushing and shoving. But it gives both the player and the GM the chance to stake increasing amounts of resources (through the fallout mechanic) on the conflict depending on how important something is to their character. Similarly,
The Shadow of Yesterday's "Bringing Down the Pain" mechanic serves a similar role; it is a clear signal that simple failure is not acceptable, and the character is willing to stake more on the conflict. As an aside, it's also a great cue that the player is more fictionally invested in something, because it is also a way to go from high-level conflict resolution into "bullet time" where things are resolved more blow-by-blow.
One of the important facets of both mechanics is that they allow players an "out." At any point, if the stakes become too high, the player can "give," accepting some consequence (be it fictional or mechanical) that represents some sort of loss of character resources (be they fictional or mechanical). This too is a cue: "I deem this as too difficult, so I'm going to stop before I die or lose more than I think the gain is worth."
Another important facet here is one of open and clear communication about the Shared Imaginary Space. If the GM isn't clearly communicating what's going on, the difficulty of a task is not necessarily apparent up-front:
Player: "Hrm, I completely whiffed my defense roll. I guess the troll knocks me down, yeah?"
GM: "Yes. Take 10D20 damage from the lava."
Player: "Wait, what?"
GM: "Oh, did I forget to mention that this bridge lacks rails and crosses a fast-flowing stream of molten rock?"
So the answer to your question ("what tools can help resolve these conflicting duties") starts with a mechanism which allows the players to indicate to each other when something is a problem. Then, once the problem has been identified, gives them a way to reach a compromise.
Alternately, you could split the responsibilities (e.g. by commoditizing difficulty as one possible method). In other words, the
player gets to decide how difficult something is:
Player1: "I want to cross the lava stream, but I'm only rolling a d4."
GM: "OK, but with a d4, you're not going to get anything else other than a simple cross/no cross result."
Player1: "Yeah, that's cool. I don't care about gaining any advantage here."
Player2: "I want to cross the lava stream, but I'm gonna throw a d10. It's important to me to be able to disrupt the evil priests' ritual before something worse gets summoned, so I feel like I need an advantage when I get to the other side."
GM: "You mean 'if' you get to the other side. A d10 is going to expose you to a critical hit if you fail badly enough."
Player2: "Psssh. Whatever. Fortune favors the bold. Let's do this!"
The result of Player2's roll then might result in a couple of "Drama Points" that can be used somehow, either to add to a future roll, improve his or her character, take control of narration, introduce a fact about some element of the setting or an NPC, or whatever. In terms of character improvement, this is how Dungeon World works - in order for you rack up Experience Points, you need to attempt stuff you're likely to fail at, so the decision of how "difficult" to make something is often up to the player rather than the DM.
Another commoditization method is to incentivize certain behaviors a la
FATE. The GM has a limited (albeit large initially) pool of Fate Points.
7th Sea did the same with Drama Dice. Sure, the GM could go whole-hog, drop them all, and make any given test virtually impossible. But doing so shifts those Fate Points into the hands of the players, who will in turn use them to make later tests much easier/more consequential, and so on. Doing this will both a) allow the players and GM to balance the difficulty between them and b) give players and GM clear ways to give fictional "weight" to in-game conflicts; if you're dropping a lot of Fate Points on a particular conflict, it's because the outcome is important to you - which is both and explicit and implicit cue to everyone else at the table.
Vincent's last post is a good one because he's spot-on - every game is going to tackle this differently because every game's underlying principles and mechanics are going to lead to different degrees of conflicting duties in different places. There's no magic bullet.
That said, I think showing people up-front where the conflicting duties might arise in your game is worth doing. Telling someone straight-away, "hey, you have to balance these three things" at least makes them cognizant of the challenges they'll face in fulfilling their role. "Make the world seem real" and "Make the characters' lives not boring" might very well be in conflict - but being told up front that you have to do both at least gets you thinking about it.