Vou dividir essa resposta em várias partes.
Por que Stefan foi motivado a matar Mikael em vez de Klaus?
É explicado no final de "Homecoming". Klaus ordenou que seus híbridos matassem Damon se Klaus fosse morto. Presumivelmente, alguns dos híbridos de Klaus não foram compelidos por Mikael, então Stefan acreditava que a ameaça era real o suficiente para salvar Klaus, a fim de salvar Damon.
Katherine: He doesn't know where all went wrong...
(She looks to the passenger seat; Stefan is sitting there)
Stefan: He doesn't need to know.
(Katherine stops the car)
Katherine: Are you good from here? I need to put about a million miles between myself and Klaus as soon as possible. Stefan?
(flashback to Salvatore house where Stefan's still down on the floor after Mikael's attack)
Katherine: Stefan. Stefan.
Stefan: Elena!
Katherine: Not exactly... (she gives him a bloodbag) Pep up! I've only got a minute before they realize that I'm not at the party...
(Flashback to the party where Klaus meets Elena/ Kathrine)
Klaus: You seem nervous.
Katherine: I'm not nervous, I just don't like you.
Klaus: People have been after me for a thousand years and I'm always one step ahead. So whatever it is you're thinking of trying, go for it! Give it your best shot! You won't succeed!
Katherine: It won't be for luck of trying.
Klaus: Ah, you'd be smart to tell Damon to mind his manners tonight, because if I die, I've already ensured that he'll die along with me! Even in death my hybrids have their orders. So, you kill me, you're killing him too!
(back in the car with Katherine and Stefan)
Stefan: How did you know that I would stop Damon?
Katherine: I didn't. I was just hoping that you would want to...
(flashback to Salvatore house, Stefan drinking blood, Katherine is with him)
Katherine: If Klaus gets killed he is taking Damon down with him.
Stefan: So pull the plug on the plan.
Katherine: And face the wrath of Mikael? We'll all be dead! Okay, look. I know that you've turned off your humanity and that you don't care, so there's only one solution.
Stefan: What's that?
Katherine: Care, Stefan!! Care enough to safe Damon's life. Because I am going back to that party and I'm seeing this plan through. Klaus will be killed. We'll have our freedom. But then Damon will be dead! Your brother will be dead, Stefan. Unless you care enough to do something about it.
Isso é explicado em mais detalhes no "The New Deal":
Damon: That is for screwing up my plan! You stop me from killing Klaus and you steal his family! Why?? Doesn't make sense! Answer me!!
Stefan: Piece by piece Klaus took everything from me. I'm doing the same to him.
Damon: Go ahead, Stefan! Why did you screw it up?
Stefan: I did it to save you.
Damon: What?? No!!! No way!! You didn't do this for me.
Stefan: He was one step ahead of us. If Klaus died, his hybrids would have killed you.
Damon: I wanna yell it through your head! Stop saving me!!
Assumindo que os irmãos Salvatore poderiam matar todos os híbridos facilmente, por que Stefan não cumpriu o plano?
Só porque os irmãos Salvatore poderiam matar um híbrido, não significa que eles poderiam se defender de um grande número deles. Stefan não sabia ao certo quantos híbridos estavam na festa, então ele estaria arriscando que Damon poderia ser mordido por um lobisomem. O sangue de Klaus é a única maneira de salvar um vampiro de uma mordida de lobisomem. Se Stefan matasse Klaus, então não haveria como salvar Damon se isso acontecesse.
Assumindo que Mikael pudesse forçar os híbridos, por que Stefan não cumpriu o plano?
Foi estabelecido que Mikael não é de confiança de Damon. Você poderia supor que Stefan também não confia em Mikael. Do final de "Homecoming":
Damon: We thought of everything, Elena! Klaus having hybrids. Mikael turning on us. We brought in Katherine so you weren't in danger. Anything that could have gone wrong we were prepared!
Além disso, foi estabelecido que Mikael não tem nenhum problema em arriscar a vida ou matar os irmãos Salvatore para matar Klaus. Um episódio antes em "Pessoas comuns":
(Mikael slams his fist into Damon's chest)
Mikael: Careful, Stefan. One move - and his heart's gone. Now, where is Klaus? What's the trick question, Stefan? One twist and he's dead. Unless you don't care of course.
Damon: His emotions have been shut off. He can't care.
Mikael: Really? His link to Klaus is so strong that he will let his own brother die? No ideas, hmm? Think, Stefan! I guess you're right. He's a dead end. And so are you, Damon.
Stefan: Wait. I can bring him back. I can lure Klaus back to Mystic Falls.
(Mikael takes his hand out of Damon's chest)
Mikael: And there is our loophole. Just had to rile him up a bit. Get Klaus back to Mystic Falls and I will gladly drive a stake through his heart. Fail, I'll drive it through yours.
Assumindo os dois pontos acima, Stefan não confia em Mikael para se preocupar em salvar Damon; Mikael só quer matar Klaus. Se você olhar de perto a conversa entre Mikael e Klaus em "Homecoming", Mikael apenas diz que ele vai obrigar os híbridos que estão atacando ele. Se os híbridos estavam atacando Damon enquanto Mikael estava ocupado matando Klaus com o punhal de carvalho branco, então Mikael não se importaria com a segurança de Damon porque Mikael está conseguindo o que ele quer.
Klaus: Or I could watch my hybrids tear you limb from limb.
Mikael: They can't kill me.
Klaus: True. But it'll make a hell of a party game. All I have to do is rub these two fingers together and they'll pounce.
Mikael: The big bad wolf. You haven't changed. Still hiding by your playthings like a coward. You only forget. They maybe sired by you but they're still part vampire. And they can be compelled by me.
(The female hybrid pushes Elena/Katherine to Mikael)