PNG:
http://en.wikipedia.org/wiki/Portable_Network_Graphics
stands for Portable Network Graphics and was designed as a replacement for GIF that does lossless compression which means that your image doesnt get blurry or worse quality as you compress it
JPEG or JPG stands for Joint Photographic Experts Group
http://en.wikipedia.org/wiki/JPEG
it is a lossy compression format for images (the image might get distorted or blurry with higher levels of compression)
the main differences between PNG and JPG is that PNG is specifically designed for website use (non lossy compression/small filesizes) and also supports transparency which JPEG does not
JPEG was originally designed for compressing photographs for easy storage/transmission (much like MP3 was for WAV files)