If you care for your family then keep visiting them on frequent basis. It would be bad to leave them. If you stay from from your family keep calling them. Leaving your family is not a good decision. Try to make balance between your work and family because both are important for life.