i've done both.
i've also asked around the areas where i found the garment (or sunglasses, or whatever) to see if anyone lost it and wanted it back.
most of the occasions where i've taken the item home, one or more of the following situations applied:
when in public places, i only remember returning the item to the front desk/cashier, etc. of the place where i found it. i can't recall keeping something for myself in that situation.
when i lose or misplace something myself, i usually consider it gone after searching for it to the degree i use it and/or feel attached to the thing. i generally don't expect to get it back, and i hope someone else gets some enjoyment or usage of it.
over the years, i've tried to let go of more stuff, my attachment to material things, and to worry less about losing things. i figure if i really need and want something, i probably won't lose it in the first place.
i've enjoyed the really, really, free markets i've gone to. those experiences have helped me a lot to freely give and receive without the need for exchange. i realize the difference in intention between RRFM's versus lost and found, but perhaps only in degree - i think i often lose or find something because i wanted to in some way.
ba@, great answer! i really like a few points here. first, the way you expanded further possibilities latent within the question's dichotomy (either/or). second, the notably non-moral, more immediate relationship with 'things,' whether lost or presented to you, based in your need and desire. and the wonder of appearance and disappearance, flux and flow, in sensing gradually dis-placing the semi-planned grid of the market and the march of endless exchanges.