Not sure if this has been mentioned before, but there are a lot of private holiday rentals, like air b&b. we have been twice, and used them. While it was not as cheap as hostels, they were cheaper than the hotels.
Also the Icelandic people are the kindest people i have come across, and i am an...