Hi
On some '03's the swing arm bolt did not have enough threads, they could not get the proper tention on the swing arm bearings (the new bolt is on all '04's). This also serves as one of the motor mounts, I like you did not have the vibration until I had several miles on the bike (don't remember the number). It was not until the local HD factory service rep provided input to the dealer that the problem was resolved. This person will also ride your bike and let you know their thoughts on normal vibration levels.
Wayne