A Blog from Embitel Technologies India Pvt Ltd

Smart e-Commerce: e Business Information

Search Engine Web Crawler Application

Posted by Nagaraju on July 20, 2010 in SEO with No Comments



The Web Crawler Application is divided into three main modules.

1. Controller
2. Fetcher
3. Parser

Controller Module – This module focuses on the Graphical User Interface (GUI) designed for the web crawler and is responsible for controlling the operations of the crawler. The GUI enables the user to enter the start URL, enter the maximum number of URL’s to crawl, view the URL’s that are being fetched. It controls the Fetcher and Parser.

Fetcher Module - This module starts by fetching the page according to the start URL specified by the user. The fetcher module also retrieves all the links in a particular page and continues doing that until the maximum number of URL’s is reached.

Parser Module – This module parses the URL’s fetched by the Fetcher module and saves the contents of those pages to the disk.

next post »

100 million Facebook Profiles ....

  The personal details of 100 million users of social networking website Facebook ar...

Leave a Reply

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Click to hear an audio file of the anti-spam word

No trackbacks yet.

No post with similar tags yet.

Posts in similar categories

About Smart e-Commerce

Smart e-Commerce is the Blog from Embitel which will provides the latest trends, information, Trends and strategies about e-Retail Solutions, e-Commerce, Social Media Optimization, Blog Marketing, Online Seminars, Search Engine Optimization, Google Rankings and e-Marketing Solutions in India, Germany, UK, Nordic, Australia, Norway, Sweden, Denmark and Finland.

For ECommerce Services
Click Here

//paste ad code here
  • Recent Post