The collective was founded by Anton Newcombe in San Francisco between 1990 and 1993. Their first albums were compilations of recording sessions and an early demo tape, titled Pol Pot's Pleasure Penthouse. This release became a popular bootleg.
A second album, Spacegirl and Other Favorites, was released in 1993 as a vinyl-only release and was compiled from what Newcombe called his "studio trash".The album includes "Hide and Seek", which was released as a single in 1994. The band's follow-up album, Methodrone, was developed largely out of the concepts explored on 'Spacegirl' and heavily influenced by the shoegaze genre that had gained prominence several years prior to its release. The album's ethereal rock sound is comparable to bands such as Galaxie 500, Spacemen 3 and My Bloody Valentine. Two tracks from the album, "She Made Me" and "Evergreen", were released as a double A-side single in 1992. 'Methodrone' was recorded primarily at a studio in the Hunter's Point region of San Francisco called The Compound, where Naut Humon provided an environment for Newcombe to record for hours on end.
Over the next couple of years, the band shifted its sound from their more shoegaze, goth, and dream pop influences of the 80's and 90's into a 60's retro-futurist aesthetic. As lineup changes persisted, the band continued to record and in 1996 released three full-length studio albums. The first of these, Take It from the Man!, is rooted heavily in the maximum rhythm and blues aesthetic of the 1960s British Invasion. The album includes the song "Straight Up and Down", which was later used as theme music for the HBO television drama series Boardwalk Empire (2010–14), and was engineered by Larry Thrasher of the influential group Psychic TV.
The band's second album released in 1996, Their Satanic Majesties' Second Request reflects a pastiche of 1960s psychedelia. The album also includes vast experimentation with a variety of different instrumentation including Indian drones, sitars, Mellotrons, farfisas, didgeridoos, tablas, congas, and glockenspiels. The title of the album is a play on words of the Rolling Stones' 1967 album Their Satanic Majesties Request.
The third and final album released that year was Thank God for Mental Illness, a more stripped-down effort. Since the band did not have a drummer at the time, Newcombe took the opportunity to showcase more of his acoustic songwriting. The album explores more in-depth genres such as country and folk. At the end of the album Newcombe included an entire EP called "Sound of Confusion", compiled largely from earlier BJM recordings. "Sound of Confusion" features both regular songs and more abstract sound collages.