If a single offer has multiple tiers, can I set different Upsell message for each tier?
No, you can't. When an offer is created, you only get to create one Upsell message per offer, regardless of how many tiers you had set up for that specific offer. Creating another tier for the Upsell message is not possible.
As soon as an offer has been saved, the Upsell message will show in the customer cart page together with the first tier of offer that has been set up.