When I was little I picked up the idea from something either heard or read somewhere that one "should" have a hobby and stamp collecting was a good hobby [could have been some Australia Post propaganda :-) ]. So of course, if one collection is "good" then surely more than one is "better". Eventually I ended up with a collection of collections - stamps, bottles, key-rings, baseball style caps... Some of these were effectively started by others on my behalf, along the lines of, "Oh, Laetitia has two interesting bottles - (insert leap of logic) - she must be collecting them; let's give her some more!"
The stamps went years ago to a charity that sells them to middle-men who buy old stamps to on sell. Most of the bottles also went several years ago to another bottle collector. During this last move I scanned articles from about 40 gardening magazines before giving them to a friend and gave most of my key-ring collection to another friend. I still have a few caps but am reconsidering even them (how many heads do I have?!).
My weaknesses now are craft activities, particularly album-making (I'm a dealer) and quilting (mmm, fat quarters...). I play the saxophone but I wouldn't call supplies for that "clutter" as I don't have a great deal of items (e.g. I don't stock up masses of reeds - they'd just go mouldy) and I keep my music fairly tidy.
I intend to keep the quilting in check by only buying material needed for the project at hand, not just because it's a lovely piece of fabric. That is unless it's a piece of fabric suited to a planned, but not yet in progress, project that is waiting for enough fabric to be started.
Of course, over the years there have been the common books and stationery weaknesses. I'm keeping stationery in check by getting DH (teacher) to store them with his school supplies and take them with him to his next post. DH has also been good at getting me to relinquish a number of books - he's got Buckley's though when it comes to the Terry Pratchett and L. M. Montgomery collections. :-)