Why is Azerbaijan called "The Land of Fire"? This is why. It's one of the only country in the world where you can observe the "fire mountains" phenomenon. Natural gas seeps through the limestone, and, when lit, continues burning for decades; this particular mountain has been on fire since the 1950s and it's one of the few left, but it was more common in ancient times: both Marco Polo and Alexandre Dumas described similar places during their journeys in Azerbaijan. It's no wonder that the first religions in the area were based around fire worshipping, and that later gave birth to Zoroastrianism, a pre-Islamic religion based around the cult of light.
Other than the fire itself, the Yanardag site also contains some prehistorical artifacts. You can observe ancient stone musical instruments, and stone tablets etched with the sun effigy by the fire worshippers. All of this is accompanied by a useful, although short exhibit explaining the history of the fire mountains and similar phenomenons, and the religious aspect surrounding it.
Practical information: the site is outside Baku, but you can easily reach it with one hour on the 147 bus, which stops directly in front of the reservation.