Effective usage of Online tools for web based testing

Web Standards

• The World Wide Web Consortium (W3C) is an international consortium where Web standards are developed. Mission : To lead the WWW to its full potential by developing protocols and guidelines that ensure long-term growth for the Web
• A major chunk of the population suffers from some of the impairment. The target segment for accessibility includes – Visual, Hearing, Old age, low Literacy, low bandwidth connections
• Web Content Accessibility Guidelines (WCAG) - Explains to Web content developers how to make content accessible for people with disabilities
• Section 508 and ADA (American with Disabilities Act) are laws that govern Web accessibility in US

Browser Compatibility

• Different web browsers are created by different organizations, user-interface enhancements. The designer must devise a strategy that accounts for this possible variation to ensure that a website can still be used without the enhancement or only use features that are supported by ‘all’ browsers.
• Testing the Browser Compatibility of Web sites across various browsers and operating systems

Readability

• Readability is the measure of how easy to read and comprehend a document
• Reading level algorithms only provide a rough guide, but can give a useful indication as to whether you've pitched your content at the right level for your intended audience
• Gunning Fog, Flesch Reading Ease, and Flesch-Kincaid are reading level algorithms that can be helpful in determining how readable your content is

Web Page Speed Analysis

• Web page speed is based on page size, composition, and download time.
• Page weight is a measurement of the file size of a Web page that includes the combined size of all the elements of the page, including HTML files, images, audio or video, Flash animation, etc.
• Reducing the page weight obviously improve the performance of the web site

Tools Usage

Tools	Description
HTML Validator	To check the HTML code as per W3C standards. This validator checks the markup validity of Web documents in HTML, XHTML, SMIL, MathML etc
Link Checker	To check the Link navigations as per W3C standards. Link Checker looks for issues in links, anchors and referenced objects in a Web page, or recursively on a whole Web site
CSS Validator	To check the CSS as per W3C standards
HTML WDG	To check the HTML code as per WDG standards
Juicy Studio	To check the readability of the content of the website
Page Speed Report	To analyze Web page speed
WAVE	To check for accessibility errors as per WCAG standards

All these evaluators are available with Web Accessibility Tool Bar (WAT) 2.0 for IE.

WAT 2.0 is a Free Ware provided by Web Accessibility Tools consortium. The tool aids in manual examination of web pages for a variety of aspects of accessibility. The functions are

1.Identify components of webpage

2.Provide access to alternate views of page content

3.Facilitate use of 3rd party online applications

Along with these evaluators, WAT2.0 also features tools such as Lynx viewer, Grey scale, FAE, Contrast analyzer, Vischeck color blindness simulator, Resize etc

Other Tool

http://www.webshot.org – Free Online Tool used for Cross Browser Testing

Analysis for a XXX Website

This case study provides the effective usage of free online tools available to test the XXX Website

Evaluator	Standards	Errors	Warning
HTML	W3C	123	107
Link Checker	W3C	1	110
CSS	W3C	49	203
HTML	WDG	30	21

Not an issue: Though these errors are reported, it doesn’t affect the core functionality.

Nice to have: The ease of displaying in various browsers, PDA, Pagers etc.

* All the results reported were w.r.t the XXX Website

Accessibility analysis using WAVE

Web pages	No of Errors
Home page	29
Collection page	55
Product page	46
Shopping Cart	27
Sign in - check out	24
Recipient page	36
Billing page	48
Review order page	79

Not an issue: Within a given system configuration we will not have any issues.

Nice to Have: The web page is assured to meet WCAG standards and also accessible at any situation. (The errors reported are “Alt text” missing for the images, links, form labels. The error will be critical to access, if the page is not fully loaded.)

* Refer appendix for screen shot of WAVE Tool

Browser Compatibility

Browser	Versions	Browser	Versions
Avant	11.7	K-Meleon	(1.1.4), (1.5.0)
Chrome	(0.2.149.30), (1.0.154.65), (0.3.154.9), (0.4.154.25)	Konqueror	4.2
Epiphany	2.22	Minefield	(3.1), (3.2), (3.6)
Firefox	(0.8), (0.9), (1.0.8), (1.5),(2.0.0.4), (3.0.1), (3.1)	MSIE	(5.01), (6.0), (7.0)
Flock	(1.2.6), (2.03)	Navigator	9.0.06
Galeon	2.0.4	Safari	(3.2.1), (4.0)
Iceweasel	3.0.4	SeaMonkey	(1.1.14), (2.0)
Kazehakase	0.5.2	Shiretoko	(3.1), (3.5)
Opera	(8.0), ( 9.25), (10.0), (7.03), (7.11), (7.54), (8.54), (9.26), (9.27), (9.6), (9.62), (9.63), (9.64)

47 browsers showed positive results for the XXX Website. Almost all the widely used browsers were able to access the site

Evaluator	Value
Gunning Fox Index	10.89
Flesh Reading Ease	52.04
Flesh Kincaid Grade	6.92

 

• Gunning Fox index and Flesh Kincaid, the value represents number of schoolings required to read the content. For example if the value is sixteen, the sixteen years of schoolings is required to read the content
• For Flesh Reading Ease, the result is an index number that rates the text on a 100-point scale. The higher the score, the easier it is to understand the document. 60 – 70 is standard value and we stand at only 52.04 (Almost Near but still can reach the Standards)

Web Page Speed Analysis

Test	Units	Remarks	Status
Total HTML	1	The Total HTML should be 1	Pass
Total Objects	93	The Optimum number of object should be <20	Warning
Total Images	69	The images should be lowered for quick loading	Warning
Total CSS	5	The CSS should be lowered for quick loading	Caution
Total Size	735Kb	The optimum size should be < 100Kb	Warning
Total Script	18	The total script should be lowered for quick loading	Warning
HTML Size	67Kb	The Optimum HTML size should be <50Kb	Caution
Image Size	390Kb	The Optimum Image size should be <100Kb	Warning
Script Size	223Kb	The Optimum Script size should be <20Kb	Warning
CSS size	55Kb	The Optimum CSS Size should be <55Kb	Warning
Multi Size	0	The Multimedia size should be < 10Kb	Pass

Nice to Have: The Page size – Images, CSS, HTML can be decreased so that the pages gets loaded faster

Note: This depends on parameters like bandwidth, System Configuration etc..

Merits

The benefits on rectification of these errors are,

Clean code – Any developer could analyze the code easily.

Search Engine friendly – Search engines can track our website with ease

Faster Page Loading – A minor change can help in loading the pages quickly

Standard compliance - Publishing the standard compliance the Client can win the customer confidence

Accessibility – Easy accessibility of website across the various category of people

Once the site is W3C standard compliant, then the site shall have the W3C logo

Conclusion

• The Accessibility can be improvised to WCAG standard and open a new path for Disabled customers
• Writing web pages in accordance with the standards shortens site development time and makes pages easier to maintain. Debugging and troubleshooting become easier, because the code follows a standard
• Cross Browser testing can ensure that the Website can be used by the largest possible audience, with minimal variation in the user-experience
• The Readability Test provides prediction of the reading ease of the document with respect to sentence length and polysyllabic words. The results shall be used with Good writing style guidelines to increase the readability
• Lower page load times create more comfortable visitors. The Website brand perception will be enhanced.

Screen Shots

Web Accessibility Tool bar 2.0

WAVE report

Read More

Effective usage of Online tools for web based testing

Web Standards

• The World Wide Web Consortium (W3C) is an international consortium where Web standards are developed. Mission : To lead the WWW to its full potential by developing protocols and guidelines that ensure long-term growth for the Web
• A major chunk of the population suffers from some of the impairment. The target segment for accessibility includes – Visual, Hearing, Old age, low Literacy, low bandwidth connections
• Web Content Accessibility Guidelines (WCAG) - Explains to Web content developers how to make content accessible for people with disabilities
• Section 508 and ADA (American with Disabilities Act) are laws that govern Web accessibility in US

Browser Compatibility

• Different web browsers are created by different organizations, user-interface enhancements. The designer must devise a strategy that accounts for this possible variation to ensure that a website can still be used without the enhancement or only use features that are supported by ‘all’ browsers.
• Testing the Browser Compatibility of Web sites across various browsers and operating systems

Readability

• Readability is the measure of how easy to read and comprehend a document
• Reading level algorithms only provide a rough guide, but can give a useful indication as to whether you've pitched your content at the right level for your intended audience
• Gunning Fog, Flesch Reading Ease, and Flesch-Kincaid are reading level algorithms that can be helpful in determining how readable your content is

Web Page Speed Analysis

• Web page speed is based on page size, composition, and download time.
• Page weight is a measurement of the file size of a Web page that includes the combined size of all the elements of the page, including HTML files, images, audio or video, Flash animation, etc.
• Reducing the page weight obviously improve the performance of the web site

Tools Usage

Tools	Description
HTML Validator	To check the HTML code as per W3C standards. This validator checks the markup validity of Web documents in HTML, XHTML, SMIL, MathML etc
Link Checker	To check the Link navigations as per W3C standards. Link Checker looks for issues in links, anchors and referenced objects in a Web page, or recursively on a whole Web site
CSS Validator	To check the CSS as per W3C standards
HTML WDG	To check the HTML code as per WDG standards
Juicy Studio	To check the readability of the content of the website
Page Speed Report	To analyze Web page speed
WAVE	To check for accessibility errors as per WCAG standards

All these evaluators are available with Web Accessibility Tool Bar (WAT) 2.0 for IE.

WAT 2.0 is a Free Ware provided by Web Accessibility Tools consortium. The tool aids in manual examination of web pages for a variety of aspects of accessibility. The functions are

1.Identify components of webpage

2.Provide access to alternate views of page content

3.Facilitate use of 3rd party online applications

Along with these evaluators, WAT2.0 also features tools such as Lynx viewer, Grey scale, FAE, Contrast analyzer, Vischeck color blindness simulator, Resize etc

Other Tool

http://www.webshot.org – Free Online Tool used for Cross Browser Testing

Analysis for a XXX Website

This case study provides the effective usage of free online tools available to test the XXX Website

Evaluator	Standards	Errors	Warning
HTML	W3C	123	107
Link Checker	W3C	1	110
CSS	W3C	49	203
HTML	WDG	30	21

Not an issue: Though these errors are reported, it doesn’t affect the core functionality.

Nice to have: The ease of displaying in various browsers, PDA, Pagers etc.

* All the results reported were w.r.t the XXX Website

Accessibility analysis using WAVE

Web pages	No of Errors
Home page	29
Collection page	55
Product page	46
Shopping Cart	27
Sign in - check out	24
Recipient page	36
Billing page	48
Review order page	79

Not an issue: Within a given system configuration we will not have any issues.

Nice to Have: The web page is assured to meet WCAG standards and also accessible at any situation. (The errors reported are “Alt text” missing for the images, links, form labels. The error will be critical to access, if the page is not fully loaded.)

* Refer appendix for screen shot of WAVE Tool

Browser Compatibility

Browser	Versions	Browser	Versions
Avant	11.7	K-Meleon	(1.1.4), (1.5.0)
Chrome	(0.2.149.30), (1.0.154.65), (0.3.154.9), (0.4.154.25)	Konqueror	4.2
Epiphany	2.22	Minefield	(3.1), (3.2), (3.6)
Firefox	(0.8), (0.9), (1.0.8), (1.5),(2.0.0.4), (3.0.1), (3.1)	MSIE	(5.01), (6.0), (7.0)
Flock	(1.2.6), (2.03)	Navigator	9.0.06
Galeon	2.0.4	Safari	(3.2.1), (4.0)
Iceweasel	3.0.4	SeaMonkey	(1.1.14), (2.0)
Kazehakase	0.5.2	Shiretoko	(3.1), (3.5)
Opera	(8.0), ( 9.25), (10.0), (7.03), (7.11), (7.54), (8.54), (9.26), (9.27), (9.6), (9.62), (9.63), (9.64)

47 browsers showed positive results for the XXX Website. Almost all the widely used browsers were able to access the site

Evaluator	Value
Gunning Fox Index	10.89
Flesh Reading Ease	52.04
Flesh Kincaid Grade	6.92

 

• Gunning Fox index and Flesh Kincaid, the value represents number of schoolings required to read the content. For example if the value is sixteen, the sixteen years of schoolings is required to read the content
• For Flesh Reading Ease, the result is an index number that rates the text on a 100-point scale. The higher the score, the easier it is to understand the document. 60 – 70 is standard value and we stand at only 52.04 (Almost Near but still can reach the Standards)

Web Page Speed Analysis

Test	Units	Remarks	Status
Total HTML	1	The Total HTML should be 1	Pass
Total Objects	93	The Optimum number of object should be <20	Warning
Total Images	69	The images should be lowered for quick loading	Warning
Total CSS	5	The CSS should be lowered for quick loading	Caution
Total Size	735Kb	The optimum size should be < 100Kb	Warning
Total Script	18	The total script should be lowered for quick loading	Warning
HTML Size	67Kb	The Optimum HTML size should be <50Kb	Caution
Image Size	390Kb	The Optimum Image size should be <100Kb	Warning
Script Size	223Kb	The Optimum Script size should be <20Kb	Warning
CSS size	55Kb	The Optimum CSS Size should be <55Kb	Warning
Multi Size	0	The Multimedia size should be < 10Kb	Pass

Nice to Have: The Page size – Images, CSS, HTML can be decreased so that the pages gets loaded faster

Note: This depends on parameters like bandwidth, System Configuration etc..

Merits

The benefits on rectification of these errors are,

Clean code – Any developer could analyze the code easily.

Search Engine friendly – Search engines can track our website with ease

Faster Page Loading – A minor change can help in loading the pages quickly

Standard compliance - Publishing the standard compliance the Client can win the customer confidence

Accessibility – Easy accessibility of website across the various category of people

Once the site is W3C standard compliant, then the site shall have the W3C logo

Conclusion

• The Accessibility can be improvised to WCAG standard and open a new path for Disabled customers
• Writing web pages in accordance with the standards shortens site development time and makes pages easier to maintain. Debugging and troubleshooting become easier, because the code follows a standard
• Cross Browser testing can ensure that the Website can be used by the largest possible audience, with minimal variation in the user-experience
• The Readability Test provides prediction of the reading ease of the document with respect to sentence length and polysyllabic words. The results shall be used with Good writing style guidelines to increase the readability
• Lower page load times create more comfortable visitors. The Website brand perception will be enhanced.

Screen Shots

Web Accessibility Tool bar 2.0

WAVE report

Read More

Fighting the web robots

The internet bots have become major problem. These bots used to register for E-mail addresses that are later used to send unwanted ads, or spam, to e-mail users. CAPTCHA is a standard security technology. The most widely used CAPTCHAs rely on the sophisticated distortion of text images rendering them unrecognizable to the state of the art of pattern recognition techniques, and these text-based schemes have found widespread applications in commercial websites.

Introduction

The term "CAPTCHA" [Completely Automated Public Turing test to tell Computers and Humans ] (based upon the word capture) was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas J. Hopper and John Langford. It is a contrived acronym for "Completely Automated Public Turing test to tell Computers and Humans Apart."

A CAPTCHA system is a means of automatically generating new challenges which:

• Current software is unable to solve accurately.
• Most humans can solve
• Note: The visually disabled who rely on screen reading technology cannot solve a visual CAPTCHA, thus limiting or preventing their access to some sites.

This paper describes how security to a website can be provided using CAPTCHA to prove that they are a real person, and not just a spambot, or some other computerized agent trawling the Web for exploits.

History

In 1997 Alta Vista sought ways to block or discourage the automatic submission of URLs to their search engine. This free "add-URL" service was important to AltaVista since it broadens its search coverage. Yet some users were abusing the service by automating the submission of large number of URLS, in an effort to skew AltaVista's importance ranking algorithms.

Andrei Broder, Chief Scientist of AltaVista, and his colleagues developed a filter. Their method was to generate an image of printed text randomly so that machine vision (OCR) systems cannot read it but humans still can. A U.S. patent was issued in April 2001. In January 2002 Broder stated that the system had been in use for "over a year" and had reduced the number of "spam add-URL" by "over 95%." 

Yahoo's Chat Room Problem

In September 2000, Udi Manber of Yahoo described this "chat room problem" to researchers at CMU: 'bots' were joining on-line chat rooms and irritating the people there, by pointing them to advertising sites. How could all 'bots' be refused entry to chat rooms?

CMU's Prof. Manual Blum, Luis A. von Ahn, and John Langford articulated some desirable properties of a test, including:

• the test's challenges can be automatically generated and graded 
• the test can be taken quickly and easily by human users
• the test will accept virtually all human users with high reliability while rejecting very few
• the test will reject virtually all machine users
• the test will resist automatic attack for many years even as technology advances

CMU's CAPTCHA Research

The CMU team developed a 'hard' GIMPY CAPTCHA which picked English words at random and rendered them as images of printed text under a wide variety of shape deformations and image occlusions, the word images often overlapping. The user was asked to transcribe some number of the words correctly.

A simplified version of GIMPY (EZ GIMPYU), using only one word-image at a time, was installed by Yahoo, and is currently in use in their chat rooms to restrict access to only human users.

Need of CAPTCHA

Why would anyone need to create a test that can tell humans and computers apart?

• Let’s consider a scenario where a program is developed to create several free e-mail accounts.

This can affect millions of people across the world. These e-mail accounts can be used to spam millions of genuine users, send links to advertisements etc. If CAPTCHAs are present while registering for the free e-mail account, then it will be possible to distinguish if the user is a human or a bot. The same use will be found in several other Internet related registrations.

• Even bookings for concerts can be made to the tune of thousands with the help of computer programs.

However this cannot happen in the presence of CAPTCHAs.

Sometimes there is a possibility of the failure of the CAPTCHAs to recognise these bots.

But every CAPTCHA failure is really an advance in artificial intelligence.

• Spammers will post links back to their site in order to increase the number of links and make their site rank higher in search engines. This is called Comment spam and can be reduced through the use of a CAPTCHA.

Examples/Illustrations

Typesof CAPTCHA

• Visual Types

It’s also possible to customize the image (captcha) using our color scheme, image styles (text alignment and decoration) and the text font we like.

Simple:		Dashed:
Inclined:		Inclined & Dashed:
Horz Line:
Mixed:		Mixed & Dashed:
By choosing different colors you can create something of this type:		By choosing different fonts you can create something of this type:

• Audio Types

This is for the visually impaired and most common type of CAPTCHA.

• Problem solving Types

These require you to solve a problem that should be easy for a person but very hard for a computer to solve such as choosing which item in a list is not a bird for example, but the problem with this is that you need to have a large number of questions before it really becomes effective.

Applications

• Registration forms on Web sites often use CAPTCHAs. For example, free Web-based e-mail services like Hotmail, Yahoo! Mail or Gmail allow people to create an e-mail account free of charge. Usually, users must provide some personal information when creating an account, but the services typically don't verify this information. They use CAPTCHAs to try to prevent spammers from using bots to generate hundreds of spam mail accounts.
• Ticket brokers like TicketMaster also use CAPTCHA applications. These applications help prevent ticket scalpers from bombarding the service with massive ticket purchases for big events. Without some sort of filter, it's possible for a scalper to use a bot to place hundreds or thousands of ticket orders in a matter of seconds. Legitimate customers become victims as events sell out minutes after tickets become available. Scalpers then try to sell the tickets above face value. While CAPTCHA applications don't prevent scalping, they do make it more difficult to scalp tickets on a large scale.
• Some Web pages have message boards or contact forms that allow visitors to either post messages to the site or send them directly to the Web administrators. To prevent an avalanche of spam, many of these sites have a CAPTCHA program to filter out the noise. It will help prevent bots from posting messages automatically.
• The most common form of CAPTCHA requires visitors to type in a word or series of letters and numbers that the application has distorted in some way. Some CAPTCHA creators came up with a way to increase the value of such an application: digitizing books. An application called reCAPTCHA harnesses users responses in CAPTCHA fields to verify the contents of a scanned piece of paper. Because computers aren't always able to identify words from a digital scan, humans have to verify what a printed page says. Then it's possible for search engines to search and index the contents of a scanned document.
• Here's how it works: First, the administrator of the reCAPTCHA program digitally scans a book. Then, the reCAPTCHA program selects two words from the digitized image. The application already recognizes one of the words. If the visitor types that word into a field correctly, the application assumes the second word the user types is also correct. That second word goes into a pool of words that the application will present to other users. As each user types in a word, the application compares the word to the original answer. Eventually, the application receives enough responses to verify the word with a high degree of certainty. That word can then go into the verified pool.
• It sounds time consuming, but in this case the CAPTCHA is pulling double duty. It is not only verifying the contents of a digitized book, it's also verifying that the people filling out the form are actually people. In turn, those people gain access to the service they want to use.

Drawbacks

• One of the biggest drawbacks of CAPTCHA is that it relies on visual perception.

Users unable to view a CAPTCHA because of some disability or because they find the words are difficult to read will find CAPTCHAs difficult and also may find it hard to access the websites which make use of these CAPTCHAs for authentication.

Therefore it is suggested that sites using visual CAPTCHAs should also implement audio CAPTCHAs.

• However even with audio and visual CAPTCHAs some users may require help (users with both hearing and visual disabilities).

There have been attempts at creating CAPTCHAs that are more accessible including mathematical questions, general questions etc.

• However, none of these attempts meet both the criteria of being able to be automatically generated and not relying on the type of CAPTCHA being new to the attacker. Therefore, they are not CAPTCHAs and do not provide the protection that true CAPTCHAs provide.

Conclusion

CAPTCHA is a suitable technique to provide security and authenticate real users. However we do not live in a perfect world where all users are capable of handling these CAPTCHAs. There are those with disabilities and who may be troubled by their presence.

Sites with attractive resources and millions of users will always have a need for access control systems that limit widespread abuse. At that level, it is reasonable to employ many concurrent approaches, including audio and visual CAPTCHA, to do so. However, it must be noted that human users will fall through the cracks in these systems, and it will be necessary for sites like these to ensure that users with disabilities will have some human-operated means of interacting with a given resource in a reasonable amount of time.

An explicitly inaccessible access control mechanism should not be promoted as a solution, especially when other systems exist that are not only more accessible, but may be more effective, as well. It is strongly recommended that smaller sites adopt spam filtering and/or heuristic checks in place of CAPTCHA.

Lastly, new approaches should be found where the human users with disabilities can authenticate themselves. A short-term security benefit is not worth threatening a person's autonomy by denying them access to such important data as their finances.

References

[1] http://recaptcha.net/whyrecaptcha.html

[2] http://www.captcha.net/

[3] http://en.wikipedia.org/wiki/Captcha

[4] http://googleblog.blogspot.com/2006/11/audio-captchas-when-visual-images-are.html

[5] http://news.cnet.com/8301-17939_109-10222514-2.html

[6] http://www.answers.com/topic/captcha-1

Appendix

B

Bots…………….Web robots, www robots, bots are software applications that run automated tasks over the Internet.

I

Internet…………..The Internet is a global network of interconnected computers, enabling users to share information along multiple channels.

R

ReCAPTCHA…….An application which harnesses users’ responses in CAPTCHA fields to verify the contents of a scanned piece of paper

Acronyms

C

CAPTCHA……… Completely Automated Public Turing test to tell Computers and Humans

Apart

O

OCR………………Optical Character Recognition

U

URL………………Uniform Resource Locator

Read More

Fighting the web robots

The internet bots have become major problem. These bots used to register for E-mail addresses that are later used to send unwanted ads, or spam, to e-mail users. CAPTCHA is a standard security technology. The most widely used CAPTCHAs rely on the sophisticated distortion of text images rendering them unrecognizable to the state of the art of pattern recognition techniques, and these text-based schemes have found widespread applications in commercial websites.

Introduction

The term "CAPTCHA" [Completely Automated Public Turing test to tell Computers and Humans ] (based upon the word capture) was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas J. Hopper and John Langford. It is a contrived acronym for "Completely Automated Public Turing test to tell Computers and Humans Apart."

A CAPTCHA system is a means of automatically generating new challenges which:

• Current software is unable to solve accurately.
• Most humans can solve
• Note: The visually disabled who rely on screen reading technology cannot solve a visual CAPTCHA, thus limiting or preventing their access to some sites.

This paper describes how security to a website can be provided using CAPTCHA to prove that they are a real person, and not just a spambot, or some other computerized agent trawling the Web for exploits.

History

In 1997 Alta Vista sought ways to block or discourage the automatic submission of URLs to their search engine. This free "add-URL" service was important to AltaVista since it broadens its search coverage. Yet some users were abusing the service by automating the submission of large number of URLS, in an effort to skew AltaVista's importance ranking algorithms.

Andrei Broder, Chief Scientist of AltaVista, and his colleagues developed a filter. Their method was to generate an image of printed text randomly so that machine vision (OCR) systems cannot read it but humans still can. A U.S. patent was issued in April 2001. In January 2002 Broder stated that the system had been in use for "over a year" and had reduced the number of "spam add-URL" by "over 95%." 

Yahoo's Chat Room Problem

In September 2000, Udi Manber of Yahoo described this "chat room problem" to researchers at CMU: 'bots' were joining on-line chat rooms and irritating the people there, by pointing them to advertising sites. How could all 'bots' be refused entry to chat rooms?

CMU's Prof. Manual Blum, Luis A. von Ahn, and John Langford articulated some desirable properties of a test, including:

• the test's challenges can be automatically generated and graded 
• the test can be taken quickly and easily by human users
• the test will accept virtually all human users with high reliability while rejecting very few
• the test will reject virtually all machine users
• the test will resist automatic attack for many years even as technology advances

CMU's CAPTCHA Research

The CMU team developed a 'hard' GIMPY CAPTCHA which picked English words at random and rendered them as images of printed text under a wide variety of shape deformations and image occlusions, the word images often overlapping. The user was asked to transcribe some number of the words correctly.

A simplified version of GIMPY (EZ GIMPYU), using only one word-image at a time, was installed by Yahoo, and is currently in use in their chat rooms to restrict access to only human users.

Need of CAPTCHA

Why would anyone need to create a test that can tell humans and computers apart?

• Let’s consider a scenario where a program is developed to create several free e-mail accounts.

This can affect millions of people across the world. These e-mail accounts can be used to spam millions of genuine users, send links to advertisements etc. If CAPTCHAs are present while registering for the free e-mail account, then it will be possible to distinguish if the user is a human or a bot. The same use will be found in several other Internet related registrations.

• Even bookings for concerts can be made to the tune of thousands with the help of computer programs.

However this cannot happen in the presence of CAPTCHAs.

Sometimes there is a possibility of the failure of the CAPTCHAs to recognise these bots.

But every CAPTCHA failure is really an advance in artificial intelligence.

• Spammers will post links back to their site in order to increase the number of links and make their site rank higher in search engines. This is called Comment spam and can be reduced through the use of a CAPTCHA.

Examples/Illustrations

Typesof CAPTCHA

• Visual Types

It’s also possible to customize the image (captcha) using our color scheme, image styles (text alignment and decoration) and the text font we like.

Simple:		Dashed:
Inclined:		Inclined & Dashed:
Horz Line:
Mixed:		Mixed & Dashed:
By choosing different colors you can create something of this type:		By choosing different fonts you can create something of this type:

• Audio Types

This is for the visually impaired and most common type of CAPTCHA.

• Problem solving Types

These require you to solve a problem that should be easy for a person but very hard for a computer to solve such as choosing which item in a list is not a bird for example, but the problem with this is that you need to have a large number of questions before it really becomes effective.

Applications

• Registration forms on Web sites often use CAPTCHAs. For example, free Web-based e-mail services like Hotmail, Yahoo! Mail or Gmail allow people to create an e-mail account free of charge. Usually, users must provide some personal information when creating an account, but the services typically don't verify this information. They use CAPTCHAs to try to prevent spammers from using bots to generate hundreds of spam mail accounts.
• Ticket brokers like TicketMaster also use CAPTCHA applications. These applications help prevent ticket scalpers from bombarding the service with massive ticket purchases for big events. Without some sort of filter, it's possible for a scalper to use a bot to place hundreds or thousands of ticket orders in a matter of seconds. Legitimate customers become victims as events sell out minutes after tickets become available. Scalpers then try to sell the tickets above face value. While CAPTCHA applications don't prevent scalping, they do make it more difficult to scalp tickets on a large scale.
• Some Web pages have message boards or contact forms that allow visitors to either post messages to the site or send them directly to the Web administrators. To prevent an avalanche of spam, many of these sites have a CAPTCHA program to filter out the noise. It will help prevent bots from posting messages automatically.
• The most common form of CAPTCHA requires visitors to type in a word or series of letters and numbers that the application has distorted in some way. Some CAPTCHA creators came up with a way to increase the value of such an application: digitizing books. An application called reCAPTCHA harnesses users responses in CAPTCHA fields to verify the contents of a scanned piece of paper. Because computers aren't always able to identify words from a digital scan, humans have to verify what a printed page says. Then it's possible for search engines to search and index the contents of a scanned document.
• Here's how it works: First, the administrator of the reCAPTCHA program digitally scans a book. Then, the reCAPTCHA program selects two words from the digitized image. The application already recognizes one of the words. If the visitor types that word into a field correctly, the application assumes the second word the user types is also correct. That second word goes into a pool of words that the application will present to other users. As each user types in a word, the application compares the word to the original answer. Eventually, the application receives enough responses to verify the word with a high degree of certainty. That word can then go into the verified pool.
• It sounds time consuming, but in this case the CAPTCHA is pulling double duty. It is not only verifying the contents of a digitized book, it's also verifying that the people filling out the form are actually people. In turn, those people gain access to the service they want to use.

Drawbacks

• One of the biggest drawbacks of CAPTCHA is that it relies on visual perception.

Users unable to view a CAPTCHA because of some disability or because they find the words are difficult to read will find CAPTCHAs difficult and also may find it hard to access the websites which make use of these CAPTCHAs for authentication.

Therefore it is suggested that sites using visual CAPTCHAs should also implement audio CAPTCHAs.

• However even with audio and visual CAPTCHAs some users may require help (users with both hearing and visual disabilities).

There have been attempts at creating CAPTCHAs that are more accessible including mathematical questions, general questions etc.

• However, none of these attempts meet both the criteria of being able to be automatically generated and not relying on the type of CAPTCHA being new to the attacker. Therefore, they are not CAPTCHAs and do not provide the protection that true CAPTCHAs provide.

Conclusion

CAPTCHA is a suitable technique to provide security and authenticate real users. However we do not live in a perfect world where all users are capable of handling these CAPTCHAs. There are those with disabilities and who may be troubled by their presence.

Sites with attractive resources and millions of users will always have a need for access control systems that limit widespread abuse. At that level, it is reasonable to employ many concurrent approaches, including audio and visual CAPTCHA, to do so. However, it must be noted that human users will fall through the cracks in these systems, and it will be necessary for sites like these to ensure that users with disabilities will have some human-operated means of interacting with a given resource in a reasonable amount of time.

An explicitly inaccessible access control mechanism should not be promoted as a solution, especially when other systems exist that are not only more accessible, but may be more effective, as well. It is strongly recommended that smaller sites adopt spam filtering and/or heuristic checks in place of CAPTCHA.

Lastly, new approaches should be found where the human users with disabilities can authenticate themselves. A short-term security benefit is not worth threatening a person's autonomy by denying them access to such important data as their finances.

References

[1] http://recaptcha.net/whyrecaptcha.html

[2] http://www.captcha.net/

[3] http://en.wikipedia.org/wiki/Captcha

[4] http://googleblog.blogspot.com/2006/11/audio-captchas-when-visual-images-are.html

[5] http://news.cnet.com/8301-17939_109-10222514-2.html

[6] http://www.answers.com/topic/captcha-1

Appendix

B

Bots…………….Web robots, www robots, bots are software applications that run automated tasks over the Internet.

I

Internet…………..The Internet is a global network of interconnected computers, enabling users to share information along multiple channels.

R

ReCAPTCHA…….An application which harnesses users’ responses in CAPTCHA fields to verify the contents of a scanned piece of paper

Acronyms

C

CAPTCHA……… Completely Automated Public Turing test to tell Computers and Humans

Apart

O

OCR………………Optical Character Recognition

U

URL………………Uniform Resource Locator

Read More

What and Why Web Analytics ?

Introduction

Web analytics is collecting and reporting internet website data to analyze the results for better understanding and optimizing web site usage.

What is Web analytics?

Web analytics is a study which explains the online behavior and number of website visitors in order to increase the organization scope. It plays vital role in taking strategic and marketing decisions. It is no surprise business of web analytics has grown equally with business use of the Internet. To be clear and precise, Web analytics are nothing but tools and methodologies used to enable organizations to track the number of people who view their site and then use this to measure the success of their online strategy.

Why Web analytics?

In the past decade, the Internet has transformed marketing in such a way every one using the web depending on web traffic analytics to increase their revenue and profitability. The web is a great medium to market and sell product and services. But if you don’t understand the behavior of your website visitors in sufficient detail, in another words analytics behind it, your business is going no where.

Two categories

Off-site web analytics

There are two categories; off-site and onsite web analytics. Off-site web analytics measures the websites potential audience, share of voice (visibility) and comments that happening irrespective of whether you own or maintain a website.

On-site web analytics

On-site web analytics measures the visitors journey with respect to landing pages (It’s just a lead capture page that appears when a potential customer clicks on an advertisement or a search-engine result link.) which encourage people to make a purchase. It measures the performance of your website in a commercial context. This data is typically compared against key performance indicators used to improve website efficiency and marketing audience response.

There are many vendors that provide web analytic tools that span both categories.

Techniques to collect the data

There are two common techniques to collect the data. The first method, log file analysis, refers to data collected by web server independent of your web browser .This technique is also known as server side collection The second method, page tagging, collects the visitors data through their web browser. This is usually captured by JavaScript on each page to notify a third-party server when a page is rendered by a web browser. This is also known as client side data collection and used by most of the outsourced, hosted vendor solutions. Both collect data that can be processed to produce web traffic reports.

Below table gives the advantages and disadvantages of both.

Page Tagging Advantages	Log file analysis Advantages
Breaks through proxy and caching servers – provides more accurate session tracking. Tracks client side events JavaScript, Flash ,Web2.0 Capture client side e-commerce data –server side access can be problematic Collects and process visitor data in near real time. Allows program updates performed by your vendor. Allows data storage and archiving to be performed by your vendor.	Historical data can be reprocessed No fire wall issues. Can track band width and completed downloads and can differentiate between completed and partial downloads. Tracks search engine spiders and robots by default. Tracks mobile visitors by default.
Page Tagging Disadvantages	Log file analysis Disadvantages
Set up errors lead to data loss – if you make a mistake with your tags, data is lost and you can’t go back and reanalyze. Fire walls can mangle or restrict tags Can not track band width or completed downloads. Tags are set when the page or file is requested not when the download is complete. Cannot track search engine spiders, robots ignore page tags?	Proxy and cache inaccuracies. If a web page is cached, no record is logged on your web server. No event tracking. No JavaScript, Flash, Web 2.0 is tracking Requires program updates perform by your own team Requires storage and archiving to be performed by your own team Robots multiply visits

Alternatives and choosing the right solution

Cookies and packet sniffers are some of the alternatives. Which solution is right to implement depends on the amount of technical expertise within the organization, the vendor chosen, the amount of activity seen on the web sites, the depth and type of information sought, and the number of distinct web sites needing statistics.

Common problems in Web analytics

Data collection issues affecting log files

One IP address registers as one person

Generally a log file solution tracks visitor sessions by attributing all hits from the same IP address and web browser signature to one person. This becomes a problem when Internet service providers ISPs assign different IP addresses throughout the session. For example, a typical home PC 10.5 different IP addresses per month, In this case those 10 visits from single PC considered as 10 visitors. As a result visitor numbers are often overly calculated. This limitation can be overcome by using cookies.

Cached pages are counted once

Client side caching is where a visitors computer stores a web page they have visited. The next time they look at that page, it will be served locally from their computer. This means that the site visit will not be recorded at the web server. Server-side caching is made possible by “web accelerator” technology. This caches a copy of a web site to speed up delivery. It means that all subsequent requests a visitor makes to view that page are also served from the cache and not the site itself, again affecting visitor tracking.

Robots multiply figures

Robots, also known as Spiders or web crawlers, are most often used by search engines to fetch and index pages. These affect web analytics because a log file solution will also show all data for robot activity on your website even though they are not real visitors. When counting visitor numbers, robots can make up a significant proportion of your page view traffic.

Data collection issues affecting page tags

Setup error cause missed tags

The setup of page tags causes a number of issues when trying to track visitors to a site. Where web servers automatically log everything, a page tag solution relies on the webmaster to add hidden tag codes to each page. Pages can get missed, even with automated page tagging or content management systems. In one case, a corporate business to business site was found to have 70 percent of its pages missing tags. Missing tags equals no data for those page views.

JavaScript errors halt page loading

JavaScript page tags work well provided JavaScript is enabled on the visitor’s browser. Only about 1-3 percent of Internet users have disabled JavaScript on their browsers. How ever the inconsistent use of JavaScript code on web pages can cause a bigger problem. Any errors in other JavaScript on the same page will immediately halt the browser scripting engine at that point so page tags placed below it will not execute. JavaScript page tags work well provided JavaScript is enabled on the visitor’s browser.

Firewall block page tags

Generally corporate and personal firewalls that can prevent page tag solutions from sending data to collecting servers. In addition firewalls can also be set up to reject or delete cookies automatically. Some web analytics vendors can revert to using the visitors IP address for tracking in these instances. Users own and share multiple computers and latencies (time takes for a visitor to become a customer) also a big hurdle in collecting the data accurately.

Here are 10 recommendations to improve web analytics quality

1. It is important to choose the best method of data collection based on your business needs and resources.

2. If you choose to select a tool for data collection make sure that tool uses first party cookies.

3. Incase if you don’t get first party cookie information from a user ignore that user rather than considering it as visitor.

4. Keep a separate track on robots and server performance monitors else you can remove them considering them as a non-human activity.

5. Tracking everything is key to the success of getting as much as information we needed. Don’t limit tracking to certain landing pages, track your entire web sites activity, including file downloads and internal search terms and outbound links.

6. In order to ensure page tag completeness audit your website regularly. This will result in avoiding missing tags such as forgotten, corrupted and deleted.

7. Display clear and easy-to-read policy to establish confidence with in the customers for better understanding of how they are being tracked.

8. Don’t come to conclusions on very recent data that is one hours old. Most of the time it is inaccurate.

9. Test redirection of URLs to guarantee they maintain tracking parameters.

10. Make a clear differentiation by using tracking URLs between all paid online campaigns and non paid resources.

The above suggestions will you appreciate the errors often made when collecting web analytics data. Insight will make all the difference. There is so much room for errors so web analytics is not 100 percent accurate, and taking web analytics reports at face value can be misleading. But this knowledge will maximize the accuracy of your data and it is a critical approach for success.

Read More