How do I avoid railroading in dungeons?

My very first time as a GM, I showed up to the session with a great pile of notes and plot. Half an hour later I threw it out and started improvising because they'd gone in a totally different direction. Over the years most of my players have been willing to follow a railroad if I ask them to, but I've developed a totally different kind of session prep which makes it so I don't have to ask that:

Don't have a plot.

Instead, have a status quo, something which disrupts the status quo, and a reason that the disruption forces action.

This isn't the kind of pre-made adventure we usually see, because it's hard to generalise for a widely published product whose point is to minimise the pressure on the GM mid-session (other systems, like Fate Core, take an approach similar to mine for some of their published adventures). But for individual campaigns it can be very powerful.

This concept is predicated on the idea that the only thing you can count on a group of player characters to do is to be unexpected. Planned plots tend to fall apart at their touch.

I'll run through the process with a quick example, but keep in mind that it's going to be a lot more organic than I make it seem. In practice, I go back and forth between the steps as one part gives me new ideas for another.

What's the status quo?

I need a location, and its inhabitants. The scope is up to me--maybe just a small dungeon, maybe a town and a nearby cave, maybe a whole nation. Let's start small. The reason "dungeons" work so well is that they're contained and limited: the number of options (places to go, people to meet) is more controllable than in a sprawling open area where the players can choose to wander any which way.

Location
A small underground catacomb: crypts, inner chamber, secret chamber connected to both.

Notice that since I don't need to control what ordem the party does things in, I'm free to have the secret chamber connected to both of the other areas. This also increases opportunities for the party to find it, since I really want them to do that.

Now I establish who's living there: why they're there, what they want, and what they are doing to get what they want (intelligent recurring NPCs may also have secondary goals, and backup plans and contingencies!). I'm going to choose things that I think are cool. That way no matter what happens, cool stuff will be involved. I like Sealed Evil In A Can plots; the idea that kobolds worship dragons; and the notion that dragon magic is closely linked to their physicality. I also like setting up situations where it's equally reasonable for the players to conflict or cooperate with an NPC.

Inhabitants
The crypts are home to a small tribe of kobolds who worship and serve the dragonling that lives in the inner chamber. A malevolent spirit is trapped in the catacomb's inner chamber.
- The spirit is the ghost of a tyrannical king. He wants to break free and possess the most cunning person he can find. The major obstacle to the king's freedom is the dragonling. He's influencing the kobolds to be more violent and aggressive, to call attention to the catacomb and thus hopefully get rid of the dragon.
- The dragonling lives in the inner chamber to keep the spirit trapped. Her spells of binding are powerful, but fade quickly in her absence, so she has to remain in the catacomb. She doesn't want to spend her whole life like this, so she's planning an exorcism ritual which requires a holy relic from a nearby village temple.
- The kobolds worship the dragon and serve her, raiding nearby towns for food and valuables so she can concentrate on keeping the spirit entombed. They've just stolen a relic from a temple and (because of the ghost's influence) killed a priest.

So we've set the stage: the ghost and the dragonling have goals they're working towards, and the kobolds are their pawns.

What's disruptive to the status quo?

In most D&D adventures the most disruptive force you can possibly muster is the party of player characters. I've dropped plot hooks into my adventure: stolen religious relic, murderous kobolds, and the promise of treasure. That should be enough cues for any party to start sharpening their swords.

But more specifically, I've engineered the adventure for a specific kind of disruption: the tenuous balance between the ghost and the dragonling will be tipped one way or the other depending on what the party does, but it'll be very hard for them to não tip it at all.

What does this look like in terms of what I bring to the session?

I sketch out maps of the location and its major areas; make the stat blocks for kobolds, dragonling, and ghosts; find a few interesting terrain hazards to use; and sprinkle loot drops through the catacombs as needed (most of it is probably in the dragonling's inner chamber, with another single big piece in the secret chamber). I'll knit these mechanics together into encounters on the fly, as needed depending on the party's actions.

Now I don't have to anticipate what my players will do or try to force them into certain channels in order for plot to happen. If they set foot in the caves, I've got a delicate situation they'll disrupt and NPCs whose responses to the disruption I can improvise on the fly. The order they go through the crypts, and whether they find the secret chamber before they encounter the dragon, doesn't matter to my prep: I've created and statted a living environment rather than making canned encounters which require the players to follow an anticipated set of particular future actions.

This is a lot more work--at least until I got the hang of it--and it often means that parts of my environment don't get used so that preparation effort is wasted. That's sad. But in a prep-heavy system like 4e, this is the best way I've found to avoid railroading: create living environments with NPCs that have plans and goals they're working toward, and provide plenty of spots for the party to mess up their plans. It enables a lot more dynamism than coming up with a plot I need them to goosestep through.

Roll20 is not kind to this approach.

Because you have to input your work to the system ahead of time, it doubles the wasted effort of anything which doesn't get used. Additionally, whipping up an encounter on the fly is easy when you've got a handful of stat blocks and a wet-erase map, but it's a lot harder when you've got to put all of that together beforehand.

All I can suggest is that you use narrowly defined locations (like the catacombs) so you can put in the maps ahead of time, and drop todos the stat blocks into every map so you have them available if you need them. I don't have enough experience with roll20 to give any more specific advice on that front, but I know that it may be a big enough obstacle that my answer could be next to useless for your particular situation.

O que você poder be able to do is create generic setpiece encounters which can be quickly re-fluffed to meet the narrative's current needs. I dunno how much Roll20 will support that, but it shouldn't be too insanely difficult; I've even made sets of generic monsters which I could drop into most any situation with a different name each time.

It sounds like your problem is that your adventure's plot requires your players to have a specific set of encounters in a specific order. This kind of required linear progression is, as you have realised, a railroad. The way to avoid a railroad is not to require any or all of the encounters, and to not require them to play out in a specific order. Have you considered abandoning the planned order of events? That would allow you to modify the adventure such that it is less linear.

If your problem is that "the revelation in event B makes no sense without the context of event A" then you don't have a problem; If the players encounter event B first, it will set up a mystery that event A (or some other encounter) will allow them to solve.

If your problem is that "event B won't make any sense if it happens before event A," again, you do not have a problem: The encounter that would normally be event B can simply play out differently because the context in which the encounter occurs is different.

If your problem is that "allowing event B to play out differently due to event A not yet having occurred would result in part of the planned plot not occurring," then there's still no problem: You are simply allowing your players' actions to influence the outcome of events, which is the very essence of avoiding railroading.

4E has rules for this

So, you know how you can give Quests and Quests can result in XP? Make that your primary way players get XP rather than combat.

Structuring your Adventures/Dungeons

"We need to get the Ruby Cloak to heal the King!"

"So, you need to find out where it is, travel to get there, and acquire it. You can make deals, negotiate, sneak around, trick people, help people so they agree to help you in return, fight, intimidate... whatever would make sense, gets you closer, and gets you XP."

What about in a dungeon? Set up a few different intelligent groups, at odds, and the players have to get through them to get the thing they want. The groups might be suspicious but not necessarily hostile - they all have their own goals and basically they'll treat the PCs based on how they get treated and how much their goals align.

You know how your players show up every session and improvise what their characters do, just by having an idea of who their characters are, and what they're out to do? You set up your NPCs the same way, and you can improvise in return.

Informing your Players

Many players are used to what I call "videogame" roleplaying - you fight your way through everything unless the game forces you to talk to characters. Many players come from this experience and so, they never think to negotiate, ally with, trick, or otherwise try other options towards solving problems.

Tell players that these options are viable, and also worth XP.

While improvising is always the way to go in my opinion, it seems like you're using a system (Roll20) that encourages a more planning-heavy approach. With that in mind, here's what I suggest:

Players feel railroaded when it seems like their decisions don't matter. In order to keep them from feeling railroaded, you need to 1) make them feel as if they had other choices besides the one they made, and 2) show them clear and specific consequences for the choices they do make. You can do this within a fairly linear dungeon, without disrupting your plot too much. You can do this in two ways:

1. Make the order of encounters matter

I ran the World's Largest Dungeon a while ago, and one of the first (and best) sections of the dungeon involves a war between Orcs, Goblins, and Bugbears. Whichever faction the players encounter first will attempt to enlist the party to take out the other factions. The way I ran it was that the Bugbears appealed to honor, the Orcs coerced with violence, and the Goblins bribed with money. Which rooms the party chose to explore completely changed the way the story of the dungeon progressed, and the players were soon playing factions against each other, looking for the best deal.

But even in a less open-ended dungeon, encounter order can matter. Try having two short branches that both lead to the cloaked girl. If they go one way, they find a mechanism that allows them to disable some of the traps she has waiting for them. If they go the other way, perhaps they find allies willing to help them progress. If they miss her altogether, maybe she sneaks outside while they fight the boss, steals their horses and leaves them a cryptic note. The point is, you can break it down to 2 or 3 strictly defined options that you plan in advance, and as long as the consequences of those actions are clear and make sense, players won't feel railroaded.

[Edit: Found Este artigo, which is a step-by step guide to making less linear dungeons. Worth checking out, but might require more redesigning than you're up for]

2. Make combats more than just victory or death

Basically, give your players choices dentro combat. There are more ways to end a fight than simply dismembering all attackers. Some will try to flee when hurt, alerting other monsters further on in the dungeon (and making those encounters harder). Others will try to surrender, giving valuable information about future hazards. Some might want to simply steal some of the party's equipment, or collect some of their blood for a ritual, or kidnap the smallest party member, before running off. Alerting the players to different combat outcomes and goals will allow the party to feel like their choices in combat are effecting the outcome of the larger dungeon.

Ultimately, you're right to be worried about this. It means you're already moving in the right direction as a GM. Making your players feel like their choices matter will always take more effort than the alternative, especially in a planning-heavy campaign, but hopefully with these principles in mind you can keep your workload under control.

Emulate water

A good approach is to make things fluid -- not everything has to be very linear. If the party makes lots of noise in combat, have mobs leave their posts to investigate.

Allow different exploration mechanics

You could try giving the players more options with how to explore, or have set-piece elements that make for memorable experiences, like a wall that's weak and the party can hear or peek through and decide to knock it down. Heck, sometimes its not a bad idea to let your players get straight to the BBEG, and they'll definitely pensar they caught the DM unprepared... "Remember that time we knocked down the wall and killed all those minions? That was epic!" The players will likely circle back and explore the rooms they missed, and likely loosen their guard since the boss has been dealt with.

Prepare branching paths

You seem to setup heavy handed, single focused narrative elements. For each node in that path, try to evaluate if its extremamente necessary to encounter things in the order you plan them. If the party can skip around, how does that change the resulting elements? Even having just a few different binary choices can result in a meaningful sense of freedom for your PCs.