You are right. Very little should take that long. I can FTP my entire site over a DSL connection in about 15 minutes.
What are you using for a FTP tool? Do you get any kind of error messages? I have found that Bravenet's "upload tool" doesn't work very well, in a lot of situations. See if there might be anything on my Using FTP help page that could be useful. I connect with FTP daily and never have a problem.
