Get real - your card number is LESS secure when given over the phone.
If you are going through a secure server (look for the padlock icon in the corner of your window) then it is as safe as using your card anywhere else. Probably safer than a lot of restaurants.