Well, there's alot of different techniques used on heavily motion blurred rotos. This video wasn't really showing roto techniques, more just all these tools that were available in AE at the time.
One thing you can definitely do is turn on motion blur for the rotos. Right click the mask, go to Mask > Motion Blur ON. You might have to change your Comp Advanced Settings though, to get this to look right. The settings should be Shutter Angle 180, Shutter Phase -90, like this:
You could also try using PV Feather. It lets you use a second mask to determine the feathering of your mask edge.
http://www.revisionfx.com/products/pvfeather/However, alot of it is just practice. Its possible to get perfectly fine motion blurred mask edges by just knowing where to put the mask edge and when to animate the feathering.