Robots.txt - Its definition and use

Robots.txt : It is a file created by webmasters is a set of instructions for the search engines robots that index the web pages. This is called The Robots Exclusion Protocol.

It includes User-Agent and Disallow. Here is the meaning of these words as these are very important part of robots.txt file :

User-Agent : It is used for the search engine robots. It gives the permission to a particular or all robots to index the pages. Here is the syntax :

User-Agent :* ("*" means all robots have permission to crawl all the files/pages)

Disallow : It denies access to robots to directories/files. Here is the syntax :

Disallow : /images/ (it denies access to images directory.)

A simple robots.txt file should be :

User-Agent: *
Disallow:

It means we give permission to all the robots to visit and the directories and files.

It is very easy to create this file. Open notepad editor. Insert these instructions and save it by name robots in text format. Put it in top-level directory and upload it. Suppose your website name is http://www.abctest.com/ then your robots file's path should be here http://www.abctest.com/robots.txt

You can check your robots.txt file here http://tool.motoricerca.info/robots-checker.phtml.

If you need help to create robots.txt, use robots.txt generator tool here : http://www.seochat.com/seo-tools/robots-generator/

Some more useful Search Engines

There are lots of search engines where we can submit our website free of cost. Organic optimization is very helpful to increase traffic on the websites. I am writing the name of few search engines where you can submit your website. It is not very popular but can be useful to increase business and traffic on the website :

http://www.surfsafely.com
http://www.amidalla.com
http://www.amfibi.com
http://www.splatsearch.com
http://shoula.com
http://www.searchramp.com
http://axxasearch.com
http://www.moffly.com
http://www.nalox.com

How to remove pages from supplemental result

Steps to remove pages from supplemental result :

I already mentioned about the cause of supplemental result and it also decreases visitors. Whole world knows google search engine and they are prefering google for search. It provides quality search to the visitors. So at present we should work optimize or promote our website according to the google search engine. I want to write few steps helpful to remove a page from supplemental index :

a) Strictly follow the guidelines of google.
b) Use a right word to target the page.
c) Avoid query string on the page if possible.
d) Keep file name short and relevant to the content of the page.
e) Make a site map for the website and add it on the index page.
f) Remove useless comments or tags if any from the source of the page.
g) Write relevant meta description for the page and use very few commas not more than 3-4, avoid keyword stuffing.
h) If your website uses duplicate contents, change it.
i) Increase the backward link of your website and the inside pages. In this case link exchange is very helpful.
j) Donot use any affilate links on the pages of website.
k) Check dead link or bad link on your page. Use the URL for it : http://validator.w3.org/checklink/

Google will take about 1-2 months time to remove the pages from supplemental result. But I am not sure about it. It will take more time also.

Keyword Suggestion Tools

I want to write my experience related to keyword analyzing. Good keyword selection is also important to increase traffics. Sometimes it is not possible to find a right keyword for our web pages. It is better to find the relevant keywords used by the visitors. But don't forget to read the content of your page and after reading, use the keyword suggestion tools and insert a word related to the content of your page. After getting result, use the combination of this word with other relevant keywords. Here is the few useful keyword suggestion tools :

http://freekeywords.wordtracker.com/
http://www.keyworddiscovery.com/
http://adwords.google.com/select/KeywordToolExternal
http://www.digitalpoint.com/
http://inventory.overture.com/

How to increase page rank of a website?

Google uses the algorithm for the page rank is it count the external links of your site on other sites. So it is very important to get external links of website from other sites. For this, you must submit your website in Dmoz.org and Yahoo.com directories because it is most important directories consider by the Google in Google's algorithm.

The next you can exchange link with other websites to get more external links.
Link Exchange : In this process both two sites links to each other. It play an important role in improving the page rank. But we should remember about the quality pages. The page where we want link should be related theme to our website otherwise it is wasteless. Suppose if we have the site related to gift and we are requesting for the link exchange to that sites which contains the information on career then it is not relevant. If the career related sites provides the gift related category for the link exchange then it is good.

Search Engine Optimization

Definition : It is the tactics to gain the higher page rank of web-pages in search engines. To do so, we use various techniques. I hereby explain you about this in detail....First we should know about search engine :

Search Engine : Search engine provides the facility to the visitors to search the contents of web pages using key phrase. It stores the information of all the pages of a websites into a large database. Google uses robots to add the text in its large database. Search engine stores the information periodically from the websites.

About Website Submission : It is very important to submit the website in search engines to improve page ranks. Google will provide a good page rank if your website is approved in yahoo directory. So it is necessary also to keep quality title & description of the website in search engines. If your submission in Dmoz directory will approve then it will also added in Yahoo, MSN, Google etc. in a month or two or three. So you must submit your directory in Dmoz first. But you should choose relevant category for the submission and the information of your website i.e. title & description should be relevant and grammatical mistake free. Few of the Search Engines where we should submit our site is khoj.com, Zatka.com, Atlastraveldirectory.com(Travel Related Website)

Submission in Google, Yahoo, MSN

Without listing in search engines, we donot expect traffic to our website. So it is better to submit our website in top search engines when the work has finished. But sometimes google, yahoo and msn search engines crawl the website without submission. The crawler read the address somewhere else and crawl the website. If you have inbound links or if you submitted in the directories or in article sites then it can be possible else not. So better if we manually submit it in the search engines. As we all know about Google, Yahoo and MSN. Sometimes it is very difficult to find the exact page for submission for the fresher. So I want to write here the exact location where you can submit your website directly in these search engines :

Google - http://www.google.com/addurl.html
MSN - http://search.msn.co.in/docs/submit.aspx?FORM=WSDD2
Yahoo - http://search.yahoo.com/info/submit.html(Must have yahoo id to submit website)

Black Hat SEO & White Hat SEO

Definition : Black Hat SEO

Anything that decieves search engine is known as Black Hat SEO. It includes link farms, page redirects, link to bad Neighborhoods, duplicate contents on site, duplicate websites, keyword stuffing, creating doorway pages. Search Engine may panalize the offending site or even remove the site from listing. It is not a good search engine practice to generate traffic and increase business. We should avoid it and use the White Hat SEO for promotion.

White Hat SEO :

White hat SEO follow the guidelines of search engines and design the web pages for the humans to read it easily not for the search engines to use techinque to get listed on top in search engine using wrong method of search engine optimization.

Organic SEO Vs PPC

Organic SEO :

In organic search engine optimization, website owner or webmasters spend time and effort to get listed on top in major search engines. But natural optimization is much better than PPC campaign. It costs nothing. Only we spend time, follow the guidelines of search engines, and search some good websites to increase the backlinks of our website. We need also fresh content for our website pages means no duplicate or copied content. Long term market strategy is organic search engine optimization not PPC. Website owner always focus on nature search not PPC.

Advantages of Organic Search Engine Optimization :

1. Not require permanent investment in PPC campaign.
2. Organic search engine optimization delivers result for long time.
3. The traffic comes through organic search engine results is free and we can utilize the money in our other project without investing money in PPC campaign.

PPC :

It is time saving process and provides immediate results and very useful for the new business. Google and other search engines takes some times to crawl a new website. So it is better to get listed on top in sponsored ads in the search engines like google, yahoo and msn and other popular search engines. It needs good keyword selection to convert visitors into business and attractive content. It is the temporary solution to get traffic on the website. If you have money then you run PPC campaign else not.

We have lots of option to increase traffic and get good position in search engines. We have plenty of free articles directories, blogs. We could use these options to increase the backlinks of our websites. It takes time but only organic seo is the solution not PPC.

Advantages of PPC Campaign :

1. Instant result in business.
2. We can stop our PPC campaign any time.
3. We use any keyword to list our website.
4. We can send visitors to any landing page.
5. Perfect for time limited scheme.
6. It works with not well designed websites or whose ranking is low in search engines.