Yes exactly! There are so many hidden paths and walls (by the way to open a hidden wall just tap x or a on your controller) that my own orientation gets messed up.
Then there is the Pharos Stones that may open a hidden path/wall/ or just doesn't do a damn thing. You are risking a stone every time you use one (which is very hard to come by). Its a gamble and its an element to the game I don't like.