His father was the actor Reginald Sheffield who began as a child star and later turned to character acting. Johnny appeared at age seven on Broadway in the original cast of "On Borrowed Time". When Maureen O'Sullivan wanted out of her Jane role in the Johnny Weissmuller Tarzan series, it was decided that she and Tarzan would adopt a son (they had to adopt, according to the Legion of Decency, because they weren't married) before she died. Weissmuller personally chose Sheffield for the part of Boy, a part inspired by Bobby Nelson's portrayal in Tarzan the Mighty (1928); athletic by nature, he was taught to swim by swimming Olympian Weissmuller. Tarzan Finds a Son! (1939) was such a success that MGM signed Sheffield to six more films as Tarzan's Boy. By the time of Tarzan and the Mermaids (1948), he was too big for the part; the film merely said he was away at school. When Monogram Studios learned he had been dropped, they picked him up for the series of movies based on the Roy Rockwood Bomba: The Jungle Boy (1949) movies. He made twelve of these between 1949 and 1955.