RB's first sentence is spot on.
the problem of specialization has to do with who defines what the problem is and what the solution is.
what someone's background is has very little to do with this issue. (edit: except insofar as someone's background makes it easier or harder for their definitions to be accepted or rejected by other people, which is where privilege might/does come into the issue.)
your question assumes the logic that is part of the problem, by talking about "the marginalized" as if it is a thing. "the marginalized" is all of us, and none of us, and everything in between. it is not a homogenous group that can be accurately represented or allied with.
none of us know who agrees with us in real life or even what "agreement" looks like from one situation to the next, nor do any of us know how to fix anything but the most superficial of the problems facing us. (this sentence is a bit of a tangent, but my point is that activists/organizers/politicians act like they know and act like it's knowable.)
the presumption of unity, the idea that, for example "i like other anarchists", or other allies, or other marginalized people, or that i hate statists, etc., is a result of generalities that may make day to day life easier to bear (as they provide an outline for my expectations of myself and the world), but have little basis in practice, and little usefulness in theory. and those generalities have been (are being) used by people who manipulate us every day.
edited to flesh out ideas, format, and delete a comma.
edited again - funky, hope this doesn't screw with your yeses! :)