Oh wow, yeah, this would be better suited to the planar tracking. Trying to turn track points into mask vertices on this would take so much tweaking, you'd be better of doing it without any kind of tracking help.
The only way tracking is going to help this is using that planar technique. Although, to get a solid that stays with the wall for the full duration of your shot, with all the panning and zooming in and out, you are going to have to do the ultimate supervised tracks! (which just means you babysit the crap out of them - you watch it, stop it, fix it, alter it, track more, grab a section of points and adjust them a bit, things like that....) If this was a shot at my work, they wouldn't track it, they would just roto it by hand completely.
Honestly, I'm guessing that would be your best bet, rotoing the whole thing by hand with no tracking help. It's hard to make that call without seeing moving footage, but seeing how much the framing changes between shots, and knowing it's handheld, well... that just says to me "i can only be rotoed one way - the long way".
I would recommend putting on the long editions of Lord of the Rings for background ambiance.
You'll make it through all three a few times before you're done.
Or, if you post a little segment of the footage, we can take a look and maybe offer some other ideas. Or, at the very least, show you the most effecient way to roto without tracking help. There are right ways and wrong ways.
Oh, and next time you see the guy that decorated the set, kick him in the shins.