To keep track of social media followers the most important thing is to track your analytics using an app or tool (acc to your preference)
Analytic provide you day to day overall growth in followers.
If you are asking for Twitter, then I will suggest you the tool FollowerAudit!
It is the best Twitter analytics tool which provides followers growth stats, identifies unfollowers, fake followers and inactive followers. It will also provide you with detailed followers analytics.
The best tool I have used so far but the only drawback is it does not work for Instagram.