Finally got around to looking into this. One of the S cams on the driver side had a bad bearing which was causing it to hold the brake shoe against the drum, which would create the pulling affect. Somehow the bearing went bad and the rod would spin inside the S cam, but the cam wouldn't turn. I went ahead and replaced the springs in the assembly for the shoe return wondering if maybe the previous springs were incorrect or not.
Keep an eye on it, let me know if you have any more issues.
P051 In Service