I take advantage of that stuff too, and I still end up with a fair amount of down time. Most notably when doing one of several quests that send you all over Thera. That and walking to out-of-the-way places to check for gear on the ground.

One thing that helps (a lot) is locating items that are 1) likely to be on the ground in large quantities, 2) light and 3) can be bartered for a refresh. Load up and use as needed.

That said, I still end up sleeping a lot.