I say, keep the penny. Although...if they can come up with a way of manufacturing it cheaper, that'd be fine by me.
For one thing, the penny has value if only as a matter of American tradition and confidence, and what it suggests about us and the value and solidity of our currency itself. In other words...lesser nations have "inflationed" themselves to the point where they can no longer afford to continue issuing their lesser denominations, but not us!
But for another, I'm guessing the penny offers all sorts of little bits of "value" offsetting the cost of it's production, that we don't even contemplate. As one for instance, when I buy something for 96 cents, I pay a buck for it and put the four change-pennies in the cup on the counter. Not only have I just voluntarily "taxed" myself 4%, but I've also INCREASED the value of that dollar, by 4%! (I received what I perceive as $1 in value, but there are 4 cents left over).
Or...in a somewhat similar fashion, let's say I "burn" 1 cent for every dollar I spend, by putting it in the "cup", tossing it in a jar at home, or...throwing it down the storm drain. In that case, I'm increasing the actual value of each dollar, by (almost) 1%.