I got so excited my first time removing caulk that I believe I have removed some grout as well that was behind the caulk.
Of course, I'm most worried about water damage, so is it best to replace the grout that was there? Or would caulking over the gap be good enough to keep water out?
Grout at that location does not last. I think tilers put it there to get the job "done" with only one session. I would think that a proper job would leave out grout there and just use long lasting caulk.
The best way to proceed would be to remove all the grout and seal with caulk or silicone sealer.
EDITAR
But if you remove the grout be careful not to damage the finish on that tub.