The station was opened on September 19, 1932 as the most northerly station on the first section of the Piccadilly Line extension from Finsbury Park to Cockfosters. The station acted as the interim terminus of the line until services were further extended to Oakwood on March 13, 1933. The station's name was chosen after much public deliberation - alternatives were "Arnos Park" and "Southgate".

Like the other stations designed by Charles Holden for the extension, Arnos Grove was built in a modern European style using brick, glass and reinforced concrete and basic geometric shapes. A circular drum-like ticket hall of brick and glass panels rises from a low single storey structure and is capped by a flat concrete roof. A similar design was employed by Holden for the rebuilding of Chiswick Park on the District Line (also in 1932), although the drum there is supplemented with an adjacent brick tower.

The centre of the ticket hall is occupied by a disused ticket office (a passimeter in London Underground parlance) which houses an exhibition on the station and the line. Like Holden's other stations on the extension, Arnos Grove is a Grade II listed building. The building features as one of the 12 "Great Modern Buildings" profiled in The Guardian during October 2007, and was summarised by Jonathan Glancey, an architectural critic, as "...truly what German art historians would describe as a gesamtkunstwerk, a total and entire work of art."

Three parallel train tracks pass through the station, with two double-sided platforms positioned between the central track and the outer tracks. The edges of the platforms are labelled platform 1 and 2, and platform 3 and 4, respectively, in such a way that the two outer tracks are accessible by platforms 1 and 4, and the central track, usually used by trains that terminate and reverse at Arnos Grove station, is accessible via platforms 2 and 3. Platforms 1 and 2 are designated for trains to Cockfosters, platforms 3 and 4 for trains toCentral London.

When operational problems occur on the line, Arnos Grove station may act as a temporary terminus of a reduced service - either a shuttle service between Arnos Grove and Cockfosters or a truncated service from Central London. The station has a set of seven sidings to its south for stabling trains.