Browser-agnostic color verification using WebAii & Automation Design Canvas

Color style verification in WebAii and Automation Design Canvas is one of those nice-to-have features that saves testers some cycles especially when trying to make an automated test run cross browser (specifically in IE and Firefox). If the application you are testing uses style sheets - which most of today's rich web applications do – there is a good chance you might have to verify a style on a specific tag or that a specific tag has changed color based on its cascaded style sheet. WebAii and Automation Design Canvas offer specific functionality to help address the color formatting for computed styles.

Let's first take at look at why computed styles for color verification causes problems for testers:

The issue is related to how IE and Firefox return colors when it comes to computed styles:

In IE, computed styles accessed using (element.currentStyle) return color (i.e. backgroundColor, foreColor) using the following format : #xxyyzz

FireFox on the other hand always returns the color in the following format: rgb(x,y,x)

Typically , when writing automated tests for such a scenario (that you wish to run on both IE and Firefox), you end-up having to fork your code based on each browser type and do the validation using the specified format.

Well not anymore…., let's take a look at what WebAii 1.1 offers in terms of support in this scenario:

WebAii 1.1 Framework

There are several built-in features in WebAii framework that address this scenario:

1. HtmlStyle object:
   

   If you notice, our HtmlControl.GetComputedStyle() returns an HtmlStyle object, not a string value. The GetComputedStyle() function retrieves the style from the browser DOM and places it in the HtmlStyle object for further processing. If you wish to get the value immediately, you always call the HtmlControl.GetComputedStyleValue() which gets the actual style value as a string.
   

   The HtmlStyle object offers the following facilities to make style validation simple and more straigh-forward for colors:
   

• HtmlStyle.IsColor(): This call checks to see if the style returned is a color or not. If for any reason you have code that generically processes styles, you can use this to distinguish between color styles and other styles
  
• HtmlStyle.ToColor(): This method is very powerful and is where we abstract out the formatting between IE and Firefox. This method simple parses the value out for you and returns a System.Drawing.Color object that you can then use to validate against another System.Drawing.Color you might want to check against.
  

For example:

The following line of code can be used to validate that the background computed style color for a div is red. This line of code works for both IE and Firefox:

Assert.AreEqual<Color>(Color.Red, myDiv.GetComputedStyle("backgroundColor").ToColor());

1. AssertStyle() extensions.
   

   In WebAii 1.1, we introduced verification extensions to help facilitate HTML specific validation. The AssertStyle() class takes advantage of the above functionality to abstract out the browser formatting specifics and make color verification completely seamless.
    

   For example:
    

   The following validation will assert that the backgroundColor style on the Div tag is red regardless of the browser used.
    

   myDiv.AssertStyle().ColorAndBackground(HtmlStyleColorAndBackground.BackgroundColor, "Red");
   

Automation Design Canvas 1.0

In Automation Design Canvas 1.0 (ADC), we consume the above functionality inside the style verification sentences and we make it completely straight forward to craft and execute color verification. In ADC, we internally persist colors in #XXYYXX format and always covert (named/rgb/#xxyyzz) colors to the #XXYYZZ format. Using that format comparisons are done. Here is how a color style verification looks like in ADC 1.0:

For example, if you invoke verification to validate the forecolor of the following text on msn.com home page:

The verification sentence builder will automatically resolve the color to its #XXYYZZ format as shown below.

Definietly a craft and forget color validation. J.

Automation Design Canvas 1.0 : Test Failure Resolution

One of the key features that we are proud of in Automation Design Canvas 1.0, is the designer's ability to help testers maintain and fix their automated tests. Automated testing is not only about how fast you can record a test but also how fast you can analyze and resolve a test failure. Some of the key features included in Automation Design Canvas 1.0 are:

1. Visual State Capturing: As you record each step, the designer captures the browser state at that point in time with the target element highlighted on the surface. Each visual step is then added to the story board so that you can visually go back to it. Each visual step is also tied to the actual step, so as you re-order your steps and move them around or delete them, your visual story board of your test is always up to date with your latest changes. The goal of this feature is to help you understand where elements were at the time of recording and what the entire page looks like at that moment in time. In some cases, these visual captures can help you pin-point a failure by simply looking at the captured images.
   
2. DOM State Capturing: One thing we realized too is that capturing the visual state is not enough. In many cases, the failures are due to issues like ID changes or attribute value changes on HTML elements and such failures rarely exhibit any visual change. You actually need the DOM state at that moment in time to help you understand what changed in that DOM.
   
3. DOM Tree Diff View: When loading a failure in the Automation Design Canvas, the DOM Tree tool window enables a "DOM Diff" view that you can be use to help understand the differences between Recorded & Executed DOMs. Such a side-by-side view is an enormous help in allowing you to inspect property or DOM structure changes between when the step was recorded vs its execution.
   
4. DOM Element Index View: Each element in the DOM Tree is prefixed with its TagIndex. Combined with the Diff View discussed about, this gives you a fast path to detected and match two DOM nodes between the recorded and executed DOM trees.
   
5. Live Validation: For both Element Identification & Verification Rules. As you are investigating your failures, you can easily edit the failing rule for identification or verification and validate it against the executed DOM.
   

To help us demonstrate these features, lets take a simple example of a common failure for web automation: Element Id Changes. Let's start by taking this HTML Page:

As you can see, the input button has a "btn1" id associated with it. Let's load this page in Automation Design Canvas 1.0 and record the button click. The Automation Design Canvas will determine that the best method to identify this element on the page is using its id (btn1) given that the ID is currenly set on the page and is unique. For more information about how Automation Design Canvas builds identifications for elements including how to customize them, check out our help topic here.

Here is what the FindParam editor looks like:

If you run the test, obviously everything should pass…

Let's now say a few weeks later, the developer of this application decides to change the button's ID from "Btn1" to "Btn2". Let's go ahead and make that change in the page and run the test again:

Now that we are at this stage, let's see how Automation Design Canvas, can help us resolve this issue. To debug a failure in Automation Design Canvas, we will simply double click on the red x right next to the failing step. In design canvas, you get debug capability at the step level. When clicking on the red x, the Automation Design Canvas will open up the Debug UI for you. Let's take a close look at what that UI looks like and all its pieces:

If you open up the DOM Explorer at this point, you will notice that it also has a drop-down similar to the one listed above in the "Actions" section:

The first thing that comes to mind when debugging such a failure is what was the state of the DOM when I recorded my test.

Why can't my test find the "btn1" id. [As a tester, you might not be aware that this ID has changed]. Let's switch the Target DOM to "Recorded" and then click on the "Validate" button to see if I can still find the element in the DOM when I recorded it.

As you can see, the designer was able to find this element against the DOM that was captured at the time of recording. If you actually pull up the DOM Explorer and switch it to "Recorded", you will see that element highlighted in the tree:

We now atleast know that the FindParam used to identify this element is correct and that the failure must be due to a change in the page, not a problem in how we are identifying it. The next question to answer then, what changed and what is the new value for that change. To help identify that, let's switch the "View" of the "Recorded" dom to "TagName" so that we get a flat list of each tag with its index that looks like this:

Next we are going to compare this element with the actual "Execution" dom that the test failed again. To do so, let's simply switch the "Displayed DOM" to (both). And switch also the "Execution" "View" to "TagName". The DOM Explorer now displays both recorded and executed DOM side by side:

Now that we have both DOMs side by side, we can easily navigate the Execution DOM to the INPUT tag with index "1".

Ah!

Now we see that this element's id is btn2 not btn1. At this point there are two ways to proceed:

1. This is a bug in the product and should be reported.
   
2. This is an acceptable change and we simply need to update our automated test.
   

To update our automated test, simply go to the editor UI shown above and change the btn1 to btn2. Once you make that change, you might want to validate it against the execution DOM. Simply select "Execution" in the Target DOM drop down in the editor and click on the "Validate" button. The element should validate correctly and you are good to go at this point.

Click OK on the debug UI and you should be able to run the test again and it should pass.

Note: As soon as you click OK, the old recorded DOM is discarded for the Execution DOM since this is the DOM that the element has validated successfully against.

Powerful DOM Search & Validation using WebAii 1.1 & LINQ

One of the most powerful new (and probably least discovered) features of WebAii 1.1, is its expanded search support using LINQ. LINQ is Microsoft's latest technology introduced in .NET 3.5 that offers a highly rich and integrated query language inside .NET languages like C# or VB.NET.

It was relatively easy for us to realize how we can leverage the power of LINQ in WebAii. LINQ's most common use is to enable a rich syntax to query structured information and we thought it would serve a great purpose in also querying the DOM. Given that WebAii offers a 100% managed HTML DOM to the testcase, users can harness the full power of LINQ with their DOM Searches.

We first started by adding support to do LINQ queries on the Element object. So we introduced the Find.AllElements() that allows you to query the entire DOM tree using a LINQ expression.

For example:

Such powerful searchs makes it a lot easier and concise to do rich searches. Although this was a great enhancement to our search routine, we felt there is still more to be done. Most users are actually interested in querying a certain type of Html elements and not all of them. For example, if they want to query all tables, they will need to add another clause in the Where() clause that says element.tag == "table". In addition that, we realized that some of these searches might also want to leverage the strongly-typed HtmlControls suite that we have in WebAii. The HtmlControls suite offers a lot of rich context specific information regarding a specific Html tag. (i.e. Table's row count or even cell count). So if you are, for example, trying to use the Find.AllELements() to validate that all tables on a page have alteast 5 columns, you might have to write some pretty complex 'where' clause that defeats the purpose of making search easier using LINQ.

With that in mind, we decided that we definietly need to enable html filtering as part of our search API. We also thought that it would be great if we can make such filter based on a strongly typed HtmlControl type. For that purpose, we introduced the Find.AllControls<T>() function. This function is very powerful. I will use the rest of this blog post to go over some samples that use it.

Let's start with something simple. Let's say you simply want to get all the Images on a certain web page. With this API, you can simply do:

As you can see the Find.AllControls<T> already has all the information you need to get you the right set of images on the page. You can then loop through those images to do any validation or actions on them.

Let's take another example that leverages the rich API on our HtmlControl. Let's say you want to validate that all images on the www.gettyimages.com home page are actually loaded properly. In WebAii 1.1, we introduced the HtmlImage.IsLoaded() function that queries the DOM to validate that the image has actually loaded in the browser and is not a broken image. You can perform this validation in one line of code like this:

Very Nice, eh? Let's have some more fun with this API. Given that we are in elections season, let's say we want to go to Google's Elections News website and highlight all links that contain "Obama" with blue and "McCain" with Red. [Sounds like a CNN application to see who is getting more press J]. Here is the code that does this for us using WebAii 1.1:

If you have WebAii running, place the above code in a TestMethod and run it, looks pretty cool.

Ok, last, let's take a bit more complex validation scenario. Let's say we have a testcase that wants to validate that an ASP.NET DataGrid when put in edit mode, changes all the cells in Column 1 from labels to actual TextBox. How can we validate that?

We basically need to validate that:

1. All column 1 cells have one TextBox.
   
2. All column1 cells have only that textbox in that cell.
   

Here is the code using LINQ and Find.AllControls<> that does this for us:

That's it for now. We would love to hear your feedback about this feature or any other features in WebAii. We are in the process of revamping our online documentation to be more up to date with the WebAii 1.1 release. We are also starting an new effort to start blogging more about WebAii features and our Automation Design Canvas!!

Enjoy

Automation Design Canvas 1.0 Released!

Greetings!

ArtOfTest, Inc. is happy to announce the release of Automation Design Canvas 1.0! Automation Design Canvas is the test automation answer to today's rich web experiences and Ajax applications. It is available as a plug-in right inside Visual Studio Professional or Team Suite.

Since the release of RC0 we have focused solely on the quality and stability of the product to offer you a solid V1.0! We think we have an award winning product with lots of innovations.

Watch Quick Demo

Automation Design Canvas enables you to:

Build test automation in minutes vs hours

Shave weeks off your release schedules. No code to write or script to craft! Automation Design Canvas is built from the ground up to support rich Web 2.0 applications. It utilizes an advanced designer surface that makes test automation as easy as point and click, allowing for seamless crafting of verifications and syncrhonization for AJAX Pages.

Simple but powerful

A quick start guide is all you need to get started building test automation like a pro. The designer is highly customizable to allow advanced users to alter how recording is done. Recorded scripts are also extendible using an optional code behind model that allows developers to fully customize recorded tests whenever needed using WebAii Framework's rich .NET API. Using C# or VB.NET!

Built with maintenance in mind

Test automation needs to be constantly updated as the product being tested or the test environment changes. We have built-in features that give you the ability to rapidly resolve test failures and easily update your tests as well as track product changes through out the life time of the script using a DOM state archiving feature that can offer you DOM diffs.

Visual Studio Team System Integration

Design Canvas™ 1.0 is fully integrated in Visual Studio 2008 Professional & Team Suite. We add one additional test type to what Visual Studio offers. It integrates fully into the life cycle management of tests that VS offers including test case management, test execution, source control, remote execution and reporting using team foundation server. Futhermore, Design Canvas enables you to convert recorded scripts to VS Stress/Load tests.

Download Trial

Full features listing: http://www.artoftest.com/webaiidcproduct.aspx

How-To Videos: http://www.artoftest.com/webaiidcproduct.aspx?QS

WebAii 1.1 Released!

In conjunction with this release, WebAii 1.1 has also been released. Our next version of WebAii will be our 2.0 community preview which will include our Silverlight support that we have been working hard on.

ArtOfTest Inc.

Automation Design Canvas 1.0 (RC0) Released!

We are happy to unveil our RC0 of design canvas. RC0 represents the full feature set that we are planning for V1.0 release. RC0 also incorporates tons of feedback from our customers that have downloaded and used our beta. From this point on, we will only be taking bug fixes or major/adoption blocker features. We believe we have a very compelling and complete solution for dynamic web automation in this release. If you downloaded our beta we encourage you to upgrade to RC0. RC0 contains tons of bug fixes and additional features here are some of the major ones:

1. Dialogs Support: This includes both Html Pop-ups and Win32 Dialogs like alerts, logons or Download dialogs (For both IE/Firefox). We also included a "Generic" dialog handler for custom Win32 dialogs that enable you to define your own dialog titles, content search and how you want it handled.


2. Visual Studio Integration:
1. MSTest Execution: Execution using MSTest is now supported.
2. Create TestLists: We had a bug in that space. Now you can create TestLists, save them and execute them.
3. Data:
1. VS Data Sources: You can now drive your test using one of Visual Studio's supported data source like CSV files, XML Files and any DataBase. Select the test in "Test View", and click on "Data Connection String" in the properties window.
2. Data Range Constraints: Once your data is bound to a data source (Regardless if it is the built in data grid or a VS external source), you can define an iteration range to execute against. (i.e. 1:4, runs iteration 1 through 4)
3. Per Iteration Test Results: The Test Explorer now allows you to view test results on a per iteration. Pick an iteration from the drop down and the Test Explorer will be populated with the results for that specific iteration.
4. XML Based Elements Storage: In Beta our elements storage (i.e Elements.WebAiiElements file) was binary based. Now it is XML based and human readable.
5. Editing Elements in the Element's Explorer: In Beta that was restricted. Now you can easily update the names of Pages/Frames/TestRegions or specific elements with a Friendly Name that can be used to easily identify the element. These names will be used for code generation when you choose to convert tests to code.
6. Enhanced Element Search Alogrithm: We now use a more robust element search mechanism that tries to identify elements using single attributes then attribute combinations (included text content) before defaulting to tag index. Our testing showed how our algorithm produces highly reliable test automation. The algorithm is completely customizable using the UserSettings->Element Identification tab.
7. Page Url Configuration: The Page's Urls captured can now be configured/changed to customize how these pages are identified. For example, you can change a page to be identified by its Title vs its full url. Or maybe you want the page to be identified using the Base Url only or the Relative path + the query string. All of that is easily customizable per page.
8. SetText:
1. Automatic Password Detection: Automatically detects passwords and masks them.
2. SimulateRealTyping: You can optionally turn that on for that step and the text will be typed one letter at a time using a settable speed. This is great for automating scenarios like auto-complete text boxes.
9. WaitOnElements: All steps now have the option to wait on their elements to exist before executing the test. This is turned on by default. With this feature many of your dynamic applications no longer need an explicit Wait for Exists. You can optionally turn that feature off on a per step basis if needed. Each step also has a WaitOnElementsTimout that can be set.
10. Annotation Step Type: You can now add custom annotation messages within your steps with a configurable display time.
11. Capture Step Type: Add browser captures or desktop captures as part of your test to visually capture browser or desktop states during your test execution.
12. Simulate Real Click: You can now optionally use DOM Click (Default) or SimulateRealClick which will move the mouse to that element and click it like a real user.
13. Browser Constraints: You can now configure specific steps to run only on specific browser types. (i.e. Run this step only on Firefox or IE or both)
14. Pause: You can optionally pause your execution right before or after a specific step. This will show modal dialog that halts execution and allows you to inspect the state of the test right at that point. This is configurable on each step.
15. Browser agnostic Style-Color Validation: The designer now uses a browser agnostic style validation mechanism that abstracts out the different color persisting formats between IE and Firefox (i.e. rgb(r,g,b) vs #xxxxxx or named colors)
16. Tons of bugs fixes specifically the IE Blank Page issue: We fixed the IE extension to work seamlessly now. Keep in mind that if you are running:
1. Vista : You still need to disable "Protection Mode"
2. Windows 2003/2008 Server: Uninstall the "Internet Explorer Enhanced Security Configuration"
   
   

We really encourage you to try this updated build. If you have any feedback please don't hesitate to email us at CONTACT _AT_ ARTOFTEST.COM.

Note to WebAii Framework Users: The designers install RC2 of the WebAii Framework. This build includes tons of bug fixes and feature enhancements described here:

Enjoy!

ArtOfTest Team.

WebAii Design Canvas 1.0 Beta - Now supported on all VS Team System SKUs + VS Pro

We've had several customers raise the issue of our setup blocking if "VS 2008 Team System - Test Edition" is not installed. Many were using the Developer Edition or even VS Pro.

After reviewing all the VS 2008 APIs that Pro and the rest of the VS Team System editions expose, we confirmed that eveything we need for the designer is supported unlike VS 2005. With that in mind, we simply removed all the setup restrictions and the designer now works in full functionality on VS Professional and any of the VS Team System SKUs. The only feature that requires the Test Edition is the "WebAii Test to VS WebTest" conversion since only VS Test Edition has the WebTest type.

If you are an existing customer that already registered and downloaded the designer and wish to get the setup without the Test Edition block, simply re-download the designer setup from the link you got from us. It should be updated with a new setup without that block.

Enjoy!

Please let us know if you have any questions or concerns.

ArtOfTest, Inc.

Announcing WebAii Automation Design Canvas 1.0 (Beta)!!

We are very happy to announce the availability of our longly awaited designer\recorder. The WebAii Automation Design Canvas is the solution to many of today's tough automation challenges faced by testers and developers trying to automate web 2.0 apps.

The designer features have been meticulously designed and built after listening and learning from the industry and our WebAii customers. We've talked to many customers and we've heard their biggest pain points when it comes to web automation. We've also analyzed many webaii hand crafted unit tests to see where the repetitiveness is and what causes the tests to be fragile and prone to failures. We've also look at the most common time consuming tasks for test failure resolution and tried to address these scenarios directly in our designer. We not only wanted the designer to help you build automation faster but also help build better automation.We don't claim we have solved every problem but we believe the designer is a giant leap in advancing the test automation technologies in the market. Say farewell to the dark ages of web automation tools and say hello to the future. We really believe this tool will make software testing easier! Hopefully you will appreciate it as much as we do....

More info can be found here:

You can register and download the product from here: http://www.artoftest.com/betaregistration.aspx
Designer features in details: http://www.artoftest.com/webaiidcproduct.aspx
See our quick start videos here: http://www.artoftest.com/WebAiidcProduct.aspx?QS

We really want to hear your feedback regarding our beta. If you feel there is a missing feature or an enhancement you would like to see in the designer, please don’t hesitate to contact us. http://www.artoftest.com/contact.aspx

Thanks,
ArtOfTest Team