Server and programming technologies vary greatly and search engines react differently to them. Oftentimes, specific optimization techniques and setups are necessary to adhere to define best practices according to the technology being used. These basic setups can sometimes mean the difference between a page not being indexed and filtered out due to duplicate content and a highly optimized and well ranking page.
A major issue today, especially with large dynamic websites managed by content management systems is duplicate content. There are many forms of duplicate content and contrary to popular belief, once found, search engines do not penalize you for it. Instead, the page is usually filtered out of the index because the duplicate information found on that page does not add to the value of the search results page and is therefore excluded from the index or ‘filtered.’ Therefore it is very important to create proper URL’s, redirects, canonical URL versions, and setup a proper content structure to make full use of the entire site’s content and insure complete indexation without any filtrating.
Programming technologies will vary from site to site and search engines treat each one differently. Websites designed in pure HTML or PHP versus JavaScript, Flash, etc. offer their own benefits and disadvantages. While search engines are getting more advanced at understanding and indexing newer technology, they are still not perfect and encompassing the possible rankings of tier-2 and tier-3 engines may bring a substantial amount of traffic as well. Usually, the more advanced and code-heavy a language, the more difficult of a time search engines have at deducing the page’s contents. Making proper use of external exclude files, using an optimal amount of HTML text versus any on-page script is necessary to define what the page is all about. In addition, refraining from techniques such as JavaScript Meta refreshes will leave the site in good standing with the search engines.
