Why would any person ever restrain themselves from taking the liberty of others? Egoists are no different from anyone else. We are all motivated, on some level, by the drive for self-preservation.
With that in mind, it often doesn't make sense to "take the liberty" of others, especially amongst anarchists (who are generally hostile to those who take liberty from them), as the negative consequences are likely to be bigger than the benefit. Why be a tyrant? It's actually just too much work, in my opinion.
that said, I don't particularly define myself as an egoist, so this might not be the best answer ever.