> AWS spot instances for gpu can have availability issues as well or sometimes be priced higher than on demand (still trying to understand that one).
You always pay the spot market price, not your bid. Your instance gets killed if somebody outbids you. A higher bid increases the probability that your instances don't get killed while at the same time letting you pay spot market price.
By bidding above on-demand price, you are speculating that for the majority of the time, nobody else will bid more than the on-demand price. If you're not the only one doing that, spot price can rise above on-demand price.
You always pay the spot market price, not your bid. Your instance gets killed if somebody outbids you. A higher bid increases the probability that your instances don't get killed while at the same time letting you pay spot market price.
By bidding above on-demand price, you are speculating that for the majority of the time, nobody else will bid more than the on-demand price. If you're not the only one doing that, spot price can rise above on-demand price.