His paternal grandparents were Celia (Segal) and A. (Abraham) Frank Foster, who was a prominent senator and judge in Boston, Massachusetts.
His paternal grandparents were both from Russian Jewish families that moved to Boston, Massachusetts. His mother, who is from Maryland, has English, French, Irish, and distant Welsh and Scottish, ancestry.