Website tracking

Website monitoring allows interested parties to track the health of a website or web application. A software program can periodically check to see if a website is down, if broken links exist or if errors have occurred on specific pages. For example, a web developer who hosts and maintains a website for a customer may want to be notified instantly if the site goes down or if a web applications returns an error.

Monitoring the web is a critical component for marketing, sales and product support strategies. Over the past decade transactions on the web have significantly multiplied the use of dynamic web page, secure web sites and integrated search capabilities which requires tracking of user behavior on web sites.

Website change detection allows interested parties to be alerted when a website has changed. A web crawler can periodically scan a website to see if any changes have occurred since its last visit. Reasons to track website changes include:

• Enhanced automations :
  • Triggering in Event-driven programming
  • Updating dependent automations (such as screen scraping programs)
  • Link rot mitigation
  • Change trend monitoring
• Triggering human actions :
  • Analyst classifications
  • Updating documentation
  • Competitive monitoring
  • Compliance monitoring
  • Enforcement monitoring
  • Investigation monitoring

This is a parallel application to the offline business of press clippings. For web press clippings, a crawler needs to scour the Internet to find terms that match keywords the clipping service is looking for. So for instance, the Vice President of the United States may have staff looking at web press clippings to see what is being said about the Vice President on any given day. To do this, a web press clipping service (aka Media monitoring service) needs to monitor mainstream websites as well as blogs.

• ereleases.com provides a list of web press clipping services
• Google Alerts and OnWebChange.com provide notifications when keywords are found in newly indexed web content
• RSS feeds can often be used in this case since these are the areas in which relevant content will be found.

This type of service archives a website so that changes to the website over time can be seen. Unless archived, older versions of a website cannot be viewed and may be lost permanently. Fortunately there is at least one web service (see Internet Archive) that tracks changes to most websites for free. Past information about a company can therefore be gleaned from this type of service, which can be very useful in some circumstances.

• Internet Archive, nonprofit organization