But don't be intimidated. Walk in, say hello, and head downstairs (inside on the left) to this extraordinary, and completely free to visit, basement museum.
As you would imagine with such a venerable establishment, they have had some well-known customers. Winston Churchill bought his first and last cigars here and he had a special chair reserved for him to use while sampling cigars. This chair is in the museum and you are allowed to sit down so enjoy the photo opportunity.
Oscar Wilde was also a regular customer at James J. Fox. Born in Dublin in 1854, he became a regular customer in 1891. Both his sales ledger and Churchill's are on display.
Another key exhibit is a case of cigars that was included in the Great Exhibition of 1851 as they are reputed to be the oldest cigars in the world.
19 St James's Street
Monday to Saturday 9.30am - 5.30pm