You cannot compel anyone to follow you.
The shepherd leads by his voice, not with a leash. People have to agree to your leadership, including your children and your wife. Creation and the Holy Spirit together will call people to follow you if you are following Christ from the heart. Let them choose as they will, but make your choice clear.
In this, your very presence is a healing element. Seek to restore divine justice in every context. You will always be limited, but God’s glory is not limited. This is what they will follow.
Covenant manhood means you are an agent of God’s glory.