Head on back to the fork and take the path to the left. It will lead you back into the ruins of the temple. Follow the path into the next room. There are two mages, two blade traps, some archers, and a few reavers waiting. Kill the mages and work on the rest. You can also search the chests for some more goods.
Follow the path through the tunnels and stop once you see Kolgrim up ahead. Save here and get ready to make a bit of a decision.
Kolgrim is the cult leader and he's not too thrilled about the fact that you just killed all of his followers. It turns out that they believe that Andraste has been reborn as a dragon and that once they pour some blood in the urn she will be fully transferred. The important thing for you is that this is the way to get the Reaver specialization for your warrior. It's a decent perk, but you'll need to do a bit of work to get it and it has some consequences.
For now, you just have to decide whether you want to keep the option open. If you taunt him at all, he'll try to kill you. If you refuse to do it, he'll attack. If you placate him, you'll keep the option open and only lose some approval with the “good" members of the party. It's a very small hit, so don't worry if you still haven't decided.
On the other hand, if you draw out the argument and refuse, then Morrigan and the more amoral characters will take a hit to their approval.
On the whole, it basically depends on whether you are a warrior and whether you want the reaver class. That's all it's good for. If you defile the ashes and don't have Leliana or Wynne in your party with you, then you'll face them at camp. You can lie to Leliana with a little skill in Coerce. Wynne will walk out no matter what though. Leliana won't object if you hardened her through her companion quest. If you defile the ashes before you meet Wynne, then it won't matter when you meet her. So it's possible to do this without any negative effects.
You can also save, do the gauntlet and defile the ashes, and then get your reaver perk. For the time being, Bioware has specializations tied to your profile. It should still be there for other games. This means you can just reload and do it your way.
If you're sure that you don't want to defile the ashes, then you can just attack before he even makes his case. This prevents any changes. The fight itself isn't too bad if you can stand his attacks. He can do about 60-70 points per hit with the axe, so you might not be able to just shrug it off. I had to drop down to easy, since I didn't have Wynne around for buffs. It's a pretty standard fight once you manage to take out the two mages behind him. He's just got a lot of health.
Search his body and make sure that you get Kolgrim's horn, if you're playing this on the PC. I'll cover this one in a second. For now, just walk through the back door and reach the Mountain Top.