Andrew Goodman, the owner of Bergdorf Goodman who greatly expanded its stature as one of Fifth Avenue's most elegant and lucrative fashion stores in the 1950's and 1960's, died on Saturday at his home in Rye, N.Y. He was 86. Identified with Bergdorf Goodman his entire life, Mr. Goodman had other business interests and was active in philanthropic and civic causes. Among them he was a past chairman of the Fifth Avenue Association and the Better Business Bureau of New York and was executive director of the United Jewish Appeal. His father, Edwin, founded the company. Originally a merchant in Lockport, N.Y., he moved to New York City at the turn of the century as an apprentice to Herman Bergdorf, a women's tailor with a shop on Fifth Avenue near 19th Street. Soon the elder Mr. Goodman became a partner, then bought out Mr. Bergdorf but kept his name, and moved the store uptown, to an upstairs location on Fifth near 57th Street. The shop branched out as a furrier as well as dressmaker and flourished. His father made a bold departure. Instead of subjecting customers to tedious fitting sessions, he decided to offer ready-made high fashion off the rack. The approach was phenomenally successful. Except for two years as a lieutenant in the Navy in World War II, Andrew Goodman continued in the business. He became its president in 1951, and when his father died in 1953, inherited the ownership and became chairman. By 1969 Bergdorf Goodman was the nation's only large high-quality specialty store that remained independently owned. Mr. Goodman sold the operation to Broadway-Hale Stores for $12.6 million in stock in that company, later known as Carter Hawley Hale. He remained as president until 1975. Surviving are his wife of 56 years, the former Nena Manach; a son, Edwin A. of Manhattan; three daughters, Mary Ann Quinson of Manhattan, Vivien Malloy of Waccabuc, N.Y., and Pamela Lichty of Honolulu; 13 grandchildren, and nine great-grandchildren.