Just like fellow mobster Benjamin "Bugsy" Siegel, Goldstein was given the nickname "Buggsy" because he was considered a bit crazy.
Both Goldstein and Reles eventually fell under the tutelage and sway of the Shapiro Brothers--the leaders of Brooklyn's underworld during the 1920s and early 1930s. Soon the men, along with new members such as Harry "Pittsburgh Phil" Strauss began branching out and with the backing of Meyer Lansky, who was looking to further expand into Brooklyn, took up their own rackets in slots. Eventually the men moved on to loan sharking, crap games and labor slugging.
Business was thriving at such levels that they soon drew the wrath of the Shapiros and were placed on their hit list. Reles decided to fight back and knock off the Shapiro brothers.During one ambush attempt both Reles and Goldstein were shot--Goldstein had the tip of his nose blown off. After the men recovered, Reles recruited Italian gangsters Harry "Happy" Maione, Frank "The Dasher" Abbandando and Louis Capone into their crew (what was to be dubbed Murder, Inc.) and they were able to wipe out all three Shapiro brothers. Killed most gruesomely was William Shapiro, who had been abducted, beaten, stuffed into a sack and buried alive on a Canarsie, Brooklyn beach.
As Murder, Inc. blossomed it rapidly became apparent that their main skill was in expertly plotted assassinations. Throughout the 1930s the men embarked on an estimated 1,000 murders, while avoiding any serious convictions. Eventually the group attracted the attention of the national syndicate's leadership and Louis "Lepke" Buchalter and Albert "The Lord High Executioner" Anastasia became the gang's de facto leadership, ordering a frightening number of executions nationwide.
In 1940, fearful of being convicted on a murder charge, Abe Reles turned stool pigeon and began to reveal the gang's inner workings and gruesome misdeeds to the Brooklyn District Attorney.
Based on Reles's information, Goldstein was charged along with Harry Strauss with the September 5, 1939 murder of Irving "Puggy" Feinstein. The hit had been ordered by Anastasia for crossing Vincent Mangano, the head of the crime family he belonged to. Feinstein had been taken to Reles's house, where he was ambushed by the men, among them Abe Reles. During the course of the attack, Feinstein fought back fiercely and managed to bite Strauss's finger, taking a sizable chunk of his flesh. This enraged the men to such an extent that they tied a rope around his neck in a fashion so that the more he struggled, the more Feinstein strangled himself. For good measure he was also ice picked and eventually taken to an abandoned lot where he was set ablaze.
Goldstein along with Strauss was convicted of the murder and ordered to be remanded to Sing Sing for execution. At his sentencing, when asked if had anything to say Goldstein declared, "Judge, I would like to "pee" up your leg." He was executed via electric chair along with Stauss on June 12, 1941.