We consider the problem of two agents bargaining over the relative price of two goods they are endowed with. They alternatingly exchange price offers and the utilities are discounted. The recipient of an offer can either accept it and choose the quantities to be traded, or reject and counteroffer a different relative price. We study the set of equilibria as discounting frictions vanish and find that: (1) any generic economy has bargaining equilibria that are inefficient even as discounting frictions vanish; and (2) a bargaining equilibrium converging to a Walrasian outcome exists for some robust types of convergence of the discount factors, but it does not exist for other equally robust convergences. Moreover, in case there exists a bargaining equilibrium converging to a Walrasian outcome, then there is necessarily a multiplicity of them. As a consequence, unlike in Rubinstein's (1982) alternating-offer bargaining, the equilibrium outcome of this set-up is not generically unique and efficient.