Did you forget to check robots.txt?