OK, I found a couple more threads that had this problem and were solved by different things.
The first was caused by the person using an old copy of wp-config.php, from an older version of WordPress, on a new install of the new version of WordPress. Again, I want to check if you are doing this.
The second is not as clear, but there is a 50/50 chance it was fixed by deleting .htaccess
