Then you don’t hold absolute non violence as a value. You hold non violent unless violence is necessary as a value.
If you did hold absolute non violence as a value, then yes, you would not respond to violence against you with violence even in defense.
Values are not binary.