This seemed counterintuitive, but may be by design. If that's the case then just forget I mentioned it.

Removing a worn item doesn't change its position in the list of items you're carrying. Ex:

I'm carrying two shields: "foo shield" and "bar shield". Currently "1.shield" points to "bar shield". I wear "foo", remove "foo", then "wear shield". The shield that's worn is "bar" instead of "foo", which I just removed.

Seems like the act of removing an item (i.e. sending it back into the pool of stuff you see with the "inv" command) should place it at the "top" of the list.