The trouble with government work, as opposed to the private sector is that there's a lack of efficiency. It's important to understand, however, that there's nothing intrinsically efficient about private sector work. No magical "it's the free market" dust comes and renders private enterprises effective. Rather, the idea is simply that an inefficiently run private enterprise (and there are many) would simply go out of business. An inefficiently run government office, by contrast, goes out of business when it loses political support and sees its budget grow as long as it maintains political support. Thus, you see public sector dollars flowing to whatever there's a strong political constituency for, whereas private sector dollars flow to wherever well-managed firms are meeting demand.Bingo. This is the main reason why everybody should look at sole-source contracts and hidden bidding processes with enormous suspicion. It doesn't really take that much time or change before something that once seemed honest turns into political cronyism. The fair-minded people who might have propelled privatization aren't necessarily going to retain control of the process, and those that acquire control will usually have friends and supporters who want a little taste as a reward for all their hard work.
Then enter government contractors which, as The New York Times points out, have exploded to unprecedented levels under George W. Bush and the late unlamented Republican congress. Here you have private enterprises displacing government. Why? For the private sector efficiency, of course! But you don't actually get that efficiency. It's still a government program. Funding is still being determined by political support. The cash doesn't go to companies that can do a really good job, it just goes to companies that have political clout -- i.e. ones that recycle a share of their profits into campaign contributions. It's essentially the worst of both worlds, since you get the inherent problems of the public sector plus the need for owners to be taking a slice off the top in profit margins. It is, however, a very good deal for politicians interested in union-busting and for politicians interested in raking money in from government contractors. Shockingly, the GOP loves it.
Besides, as someone pointed out in Matthew's comments, the key is not "private sector", but "competition". There is nothing efficient about a monopoly, even though they're private entities. The desire to maintain monopolies itself often leads to political rent-seeking, because the last thing an inefficient monopoly wants is real competition, and if it's cheaper to buy political clout instead of making the business more efficient, that's the choice they'll make.
(A good thing to remember the next time you hear someone bleating about wonders of "The Market".)