Energy is stored predominately as lipid in white adipose tissue (WAT) in distinct anatomical locations, with each site exerting different effects on key biological processes, including glucose homeostasis. To determine the relative contributions of subcutaneous and visceral WAT on glucose homeostasis, comparable amounts of adipose tissue from abdominal subcutaneous inguinal WAT (IWAT), intra-abdominal retroperitoneal WAT (RWAT), male gonadal epididymal WAT (EWAT), or female gonadal parametrial WAT (PWAT) were removed. Gonadal fat removal in both male and female chow-fed lean mice resulted in lowered glucose levels across glucose tolerance tests. Female lean C57BL/6J mice as well as male and female lean FVBN mice significantly improved glucose tolerance, indicated by decreased areas under glucose clearance curves. For the C57BL/6J mice maintained on a high-fat butter-based diet, glucose homeostasis was improved only in female mice with PWAT removal. Removal of IWAT or RWAT did not affect glucose tolerance in either dietary condition. We conclude that WAT contribution to glucose homeostasis is depot specific, with male gonadal EWAT contributing to glucose homeostasis in the lean state, whereas female gonadal PWAT contributes to glucose homeostasis in both lean and obese mice. These data illustrate both critical differences among various WAT depots and how they influence glucose homeostasis and highlight important differences between males and females in glucose regulation.