Broiles and Kirkley were adamant about getting out of the lawsuit, but Mike and Dee were equally adamant about not wanting to sign a letter of apology