A diminutive of the male given names James, Jeremy, and Jeremiah.
You may know what one man thinks of another by his manner of calling him. Thomas and James and Richard and William are stupid young gentlemen; Tom and Jem and Dick and Will are fine spirited fellows. 1830, Mary Russell Mitford, Our Village: Fourth Series: Cottage Names: