A .gz file is a compressed file. The extension .gz stands for GNU Zip. You have probably heard of the GNU Project, or GNU Linux, and you may also know how to open a Zip file. GNU Zip or .gz is a free GNU software implementation of a compression algorithm. It's similar to Zip, but free, not proprietary. It does the same sort of job, but the compression algorithms used by GNU Zip .gz are different from those used in Zip files and so you need different software to open .gz files.
A .tar file is a Unix tape archive. That's what the extension tar comes from. It's a specification that was written long ago for archiving multiple files on magnetic tape. It includes headers that give information about the files in the archive, as well as the files themselves. Although designed in the days when computers stored data on magnetic tape, the format and structure can be used to create archives in any type of file.
A .tar.gz file is a tape archive .tar file that has been GNU Zipped. In the Unix and Linux world, the task is performed by two separate utilities. You would first run the tar utility to create a single archive .tar file, and then compress it using GNU Zip, which adds the .gz extension. This is different from Zip tools and other compression utilities commonly used under Windows, where the archiving and the compression are done at the same time by a single process.
When data is archived and compressed on a Unix or Linux system, it is usually in the .tar.gz format. Windows has no built-in support for tar archives or gz compressed files. In order to open a tar.gz file you will need to use another application.