You need to be holding both weapons when you Attack in order to use Two-Weapon Fighting.

So no, you cannot RAW get the bonus from Dueling on both attacks, but you could still get it on one of them.

Let's work this through...

1. Yes, you can throw a dagger and then bonus action to attack with your offhand

Mechanically, this works fine. You start your turn wielding two light weapons and throw the first with your attack action. Two-Weapon Fighting (TWF) rules state:

If either weapon has the Thrown property, you can throw the weapon, instead of making a melee attack with it.

Then you can use your bonus action to attack with the second as per the TWF rules. So far so good.

2. And yes, it is possible to get a bonus from Dueling on thrown attacks

The text from the Dueling fighting style (p. 24 SRD) says:

When you are wielding a melee weapon in one hand and no other weapons, you gain a +2 bonus to damage rolls with that weapon.

While, a plain text reading seems to suggest that the Dueling bonus requires you to be empunhando the weapon when damage is rolled, this interpretation has been specifically overruled by Jermey Crawford aqui.

Q: Does the Dueling Style apply its bonus to a thrown melee weapon?

A: Yes

You are of course welcome to rule either way at your own table. Pergunta relacionada.

3. But, you can't draw and attack with your second weapon, as a bonus action, after throwing your first weapon

(Sorry this is long, it seems to be most contentious part of this question.)

The relevant part of the rules for Two-Weapon Fighting states:

When you take the Attack action and attack with a light melee weapon that you’re holding in one hand, you can use a Bonus Action to attack with a different light melee weapon that you’re holding in the other hand.

While it's not completely explicit, this is generally understood to mean that you must already be holding both weapons, when you take the Attack action with the first weapon, in order to have the option of Bonus Action attacking with the second weapon.

This reading is supported by the fact that it's often stated that dnd 5e 'only does what it says it does' (see answers to this question). The text here emphasises that both weapons are being held (bolded above), if holding both weapons was not an important part of this ruling then it could instead have been rendered like this:

After you've taken the Attack action and attacked with a light melee weapon with one hand, you can use a Bonus Action to attack with a different light melee weapon, using your other hand.

Writing the TWF rule like this would have removed any suggestion that both weapons must be held at once, and it would also be more succinct. The fact that they didn't express it like this suggests that those extra words are an important part of the TWF rule, and add something specific to the criteria a PC needs to meet in order to Two-Weapon Fight.

For evidence of consensus on this ruling elsewhere, see these other related questions, aqui e aqui.

So, it's possible to use your free object interaction to draw your second weapon, after having thrown your first weapon (getting a Dueling bonus on that attack). But, you wouldn't then be able to attack with your second weapon on the same turn as you wouldn't have met the TWF criteria needed to trigger the option of a Bonus action (ie. holding both weapons at the point of your first attack).

4. However, you could still get the Dueling damage bonus on your second attack

If you were holding both weapons, when you made your first Attack, then you meet the requirements for Two-Weapon Fighting. Having thrown your first weapon as part of that attack, you now meet all of the requirements to get the Dueling bonus on your bonus action Attack.

When you are wielding a melee weapon in one hand and no other weapons, you gain a +2 bonus to damage rolls with that weapon.

And, as shown in part 2, you can get that bonus Dueling damage regardless of whether you make a melee or a thrown attack with that weapon.

Your second interpretation is correct

Start with weapon 1 drawn -> attack with weapon 1 (either in melee, or thrown if the weapon has the thrown property) -> move if required & draw second weapon -> attack with second weapon in offhand (melee or thrown as appropriate)

The TWF rule doesn’t state that you need to be holding both weapons at the same time. What it does specify are the following:

1. The second weapon must be a different weapon to the first (ie a different object)
2. Both the first and second weapons have to be light melee weapons (ie they need to be classified as a melee weapon and have the “light” property)
3. You must use your Bonus Action to make the second attack happen
4. You need to do the second attack with your other hand
5. The second weapon has to be held in the other hand (or your off hand) when you are making the second attack (ie you can’t get off two attacks with your primary hand, and thus don’t get your normal flat damage bonus to the attack for the bonus action attack)
6. The bonus action attack must occur after the primary attack (ie you can’t swap the order of the two attacks)
   • this is the timing aspect of the attack

The PHB has three general rules that are of use in this situation:

You can also interact with one object or feature of the environment for free, during either your move or your action. (Pg 190)

If you take an action that includes more than one weapon attack, you can break up your movement even further by moving between those attacks. (Pg 190)

You choose when to take a bonus action during your turn, unless the bonus action's timing is specified, and anything that deprives you of your ability to take actions also prevents you from taking a bonus action. (Pg 190)

The first general rule means that you are able to draw the second weapon during the middle of your action.

The second rule means that you can move between making the two attacks that two-weapon fighting gives the character.

The third rule means you have discretion on when to use your bonus action. Two-weapon fighting specifies that the bonus action attack happens after the initial melee attack, but that’s the only timing restriction it imposes. This means you can insert your other in-turn interactions between your initial attack and your bonus action attack.

Support for the ability to draw the second weapon as part of your attack on this site is available aqui. (Note the linked question deals with the interaction of the Dueling Fighting Style and purely melee attacks, which is distinct from the next portion of this answer)

Interaction with Dueling Fighting Style

The Dueling Fighting style applies the following rule:

When you are wielding a melee weapon in one hand and no other weapons, you gain a +2 bonus to damage rolls with that weapon.

The important part of this rule is that you have to only be wielding one weapon when you make the attack for the fighting style to apply.

In the case of throw weapon -> draw -> attack with second weapon you are only wielding one weapon at a time. As a result the damage bonus from the Dueling Fighting Style would apply to both attacks, provided that the weapons being attacked with are melee weapons.

notas: if you are throwing a melee weapon as an attack it needs to have the “thrown” property in order to get the regular damage of the weapon, as otherwise you are making an improvised weapon attack with a light melee weapon.