The Mīmāṃsā argument against God as the ruler of karman is (as it is often the case with Mīmāṃsā), an application of what we call Ockham's razor:
Can God alone rule the people's destiny? The standard Indian answer is that He needs karman as His tool. But if karman is anyway needed, why not getting rid of the extra element, i.e., god?
Similarly, if we anyway need the people's karman and/or material elements to create the world/keep it going, why adding on top of them also a god?
Of course, these arguments have nothing to do with God as the object of one's longing and passionate devotion.