Wednesday, November 28, 2007

WebAii 1.0 Released!!

We are happy to announce the release of WebAii 1.0.

The most significant change between RC2 and WebAii 1.0 is the update in documentation and samples. Each sample solution that ships with WebAii now has over 80 unit tests that demonstrate the power of WebAii. If you are a current user of WebAii, we appreciate your feedback through out the beta and RC periods. You helped us provide a solid 1.0!!

WebAii is currently being adopted by many customers including some of the most noted UI component providers like Infragistics, Inc, Autodesk, Inc as well as well known websites like GettyImages.com, Payscale.com and much more!! We are planning to have some case studies available on our website soon.

So What's Next?

  • Automation Designer\Recorder: This is being polished for a public beta early next year. We are currently conducting a set of preview demos with customers that expressed interest in the designer. If you are currently using WebAii and are interested in a designer\recorder, please do contact us at contact_at_artoftest.com. We can setup a demo for you to give us your direct feedback. We are pretty confident that the feature set we built in the recorder will impress you!
  • Silverlight Automation Technology Preview: We plan to provide a WebAii version to preview the automation support we have built for Silverlight! Our goal is seamless automation across HTML and Silverlight within one test case.
  • WebAii VNext: We are prioritizing the current wish list to start working on for V1.*
  • WebSite: A new website look is coming soon....

ArtOfTest, Inc.

Tuesday, August 14, 2007

New in RC0 - The Annotator

In RC0 we also built a cool little feature that was not really planned. Although this feature is not critical to performing automated testing, it can help many automators follow their test execution visually and figure out what their test code is doing step-by-step.

The Annotator can easily be enabled by setting the AnnotateExecution flag to true. This flag is part of WebAii's Settings object so it can be set both in app.config or programmatically using the Settings object. Once this flag is set, each automation action will be annotated on your browser surface. For example, the following two lines:


            // Set Google's search text to 'ArtOfTest'


            Find.ByName<HtmlInputText>("q").Text = "ArtOfTest";


 


            // Click Google's search button


            Find.ByName<HtmlInputSubmit>("btnG").Click();



Will annotate as follows:



If you wish to output your own annotations per Html element or simply output messages to the browser you can easily do that using the Actions object. The following lines:


            // Output a custom message by the search textbox.


            Actions.AnnotateElement(Find.ByName("q"), "This is the google search text box");


 


            // Output a general message to the browser window.


            Actions.AnnotateMessage("Now running Find.Byxx tests");



Output annotations as follows:



And last but not least, this feature won't be complete if the developers didn't get the option to format their own annotation styles and colors :). You can easily customize the colors/fonts to your liking by using the Settings off each annotator object.

For example I can customize my message as follows:


            ActiveBrowser.Annotator.Settings.FontemSize = 12;


            ActiveBrowser.Annotator.Settings.FontStyle = System.Drawing.FontStyle.Bold;


            ActiveBrowser.Annotator.Settings.BackColor = System.Drawing.Color.AliceBlue;


            ActiveBrowser.Annotator.Settings.Color = System.Drawing.Color.DarkKhaki;




Which will output:


If anyone has feedback on what else they would like to see added to this feature, feel free to send us a note or leave us a comment on our blog.

HINT: You should use this feature while setting the ExecutionDelay off the (Settings) also to some value so that you can slow down the execution of tests and follow the execution easier. Place the following two lines of code at the beginning of your tests to get annotation going.



            Manager.Settings.AnnotateExecution = true;


            Manager.Settings.ExecutionDelay = 500;


Thursday, August 2, 2007

New features in RC0

We've been heads down trying to get V1.0 out of the door. We apologize for not being active on the blog...

We've had great feedback from customers regarding issues they want to see fixed or improvements in WebAii. We've been trying to address these issues as they come along and we will be updating the RC builds more frequently to get customers these fixes if any are blocking them.

With RC0, we introduced few additional features that customers have been asking for. Specifically: HtmlControl suite that abstracts out actions/verifications. So you can do: button.Click() instead of Actions.Click(Element). or Assert.IsTrue(textBox.Text.Equals("foo") instead of Element.GetAttribute("value").Equals(""). In addition to many new features to make the abstraction even more powerful.

Here is a taste of what you can do with the HtmlControls suite in RC0:

Finding Controls


// Find an HTML Button with id htmlbutton and click it


Find.ById<HtmlButton>("htmlbutton").Click();



// Find the first table on the page.


HtmlTable outertable = Find.ByTagIndex<HtmlTable>("table", 0);


Assert.IsTrue(outertable.Rows.Count == 3);




A Find object scoped for searching only within the container html control.


// Find the first table inside the outer table


//


// Note: HtmlContainerControls have a Find object


// associated with them that scopes all the Find.Byxx searches


// to elements contained within them only.


// So even if you have multiple controls with similar contained


// elements, this will help avoid any conflicts.


// Also, note how we are referencing the innertable using index 0


// since it is the first table inside our outer table.


//


HtmlTable innerTable = outertable.Find.ByTagIndex<HtmlTable>("table", 0);


HtmlTableCell cell = innerTable.Find.TableCell("TD21");


Assert.IsTrue(cell.Text.Equals("TD21"));




Navigate up the control tree


// Find the table that contains this cell.


// Navigate up until you find the first html table.


HtmlTable table = cell.Parent<HtmlTable>();




Find all controls that meet certain criteria. Just like the Find.AllByxx


// Find all HtmlInputText control with src containing partial value foo


IList<HtmlInputText> txtCtrls = Find.AllByAttributes<HtmlInputText>("src=~foo");




Automate actions on the control


//


// CLICKING


//


// Click using the DOM


Find.ById<HtmlInputButton>("button1").Click();


//


// Click using a pure desktop mouse click


Find.ById<HtmlInputButton>("button1").MouseClick();


//


// Check a checkbox and invoke the onclick event.


HtmlInputCheckBox ck = Find.ById<HtmlInputCheckBox>("checkbox1");


ck.Check(true, true);


//


// Query the checked state


Assert.IsTrue(ck.Checked);


//


// Invoke event on the control.


HtmlAnchor link = Find.ByAttributes<HtmlAnchor>("href=~google");


// Invoke any events on the control


link.InvokeEvent(ScriptEventType.OnFocus);


//


// You can capture any element on the page using the .Capture()


// Will be stored to the Log.LogLocation


Find.ById<HtmlInputImage>("image1").Capture("myfile");


//


// SELECTING


//


HtmlSelect select = Find.ById<HtmlSelect>("select1");


Assert.IsTrue(select.SelectedOption.Value.Equals("Option1Text"));


Assert.IsTrue(select.SelectedOption.Text.Equals("Option1"));


//


//


// and much more...




Get any control property directly from the DOM including Read-Only properties. You can also set any property


// Get whether a checkbox is enabled or disabled.


HtmlInputCheckBox cks = Find.ById<HtmlInputCheckBox>("checkbox1");


bool disabled = cks.GetValue<bool>("disabled");



// Disable it


cks.SetValue<bool>("disabled", true);



Assert.IsTrue(cks.GetValue<bool>("disabled"));




Hope you enjoy using these new features. Send us some feedback on what else you would like to see in the framework that would make your testing more productive..

BTW - A recorder is in the works...

Monday, June 18, 2007

Automating Safari on Windows

Apple's recent announcement of Safari Beta3 and its support for running on Windows drew a lot of interest and debate regarding Apple's strategy for supporting Windows. For us, we looked at it as an opportunity to extend our WebAii's browser support to Safari. :)

Since the announcement we've been working day and night to get a Windows host running the Safari rendering engine (WebKit) to run on Windows. Our goal was to use the same exact WebKit.dll that comes installed on the machine when you install Safari to allow our host to have the same rendering behavior as the Safari browser that Apple ships.

I'm happy to say that late last night we resolved our last blocking issue and now are able to host WebKit using purely .NET (with few PInvoke calls ;-)) in a windows form. We are now in the process of building the automation support so users can actually launch the Safari host from WebAii and perform their automation just like IE or Firefox today!

This is a Windows Form hosting the same exact rendering engine that comes with Safari:



Stay tuned to our blog for future updates.
We'll continue the "Automating Silverlight" PART II series next week!

Wednesday, May 30, 2007

Automated Testing of Silverlight Applications – PART I

PART II

Update (11/10/09): Record/Playback tool for Silverlight automation release.

One of the latest hot web development technologies on the market is Microsoft’s Silverlight framework. When Microsoft introduced WPF/e (known now as Silverlight Beta), the focus was on building rich media applications - similar to what you can do today with Flash - using a subset of their latest client development technology known as the Windows Presentation Foundation (aka WPF). Earlier this month, Microsoft introduced Silverlight Alpha which expanded the scope of Silverlight by introducing a light-weight CLR that allows developers to write managed code applications and host these applications inside the browser. To allow Silverlight applications to integrate well with current HTML/Ajax applications, Silverlight also enabled easy access to the HTML document DOM hosting these applications. Pretty cool!

As many in the development community ponder the feature set of this technology and how it can be leveraged in future development projects, us and
others in the software testing community are investigating how automated testing would work in Silverlight world! There are quite a bit of challenges to overcome especially with it only being an Alpha! But again, it never hurts to get a head start on looking at this technology from a testing prospective and providing Microsoft with feedback on how they can make our lives a bit easier when it comes to automated testing :).

When thinking about Silverlight applications and automated testing, there are a couple of different approaches that come to mind:



1. Isolated API/Unit Testing: This can be efficiently executed by running test cases inside Silverlight in a light host similar to what Jamie did with TestDriven.NET or what Visual Studio Team Test might provide in future versions. [Think of Visual Studio Team Test's ASP.NET testing support today where they allow tests to have access to the ASP.NET HttpContext. They would simply enable a Silverlight configuration where they provide a light-weight Silverlight host that tests can run in.]

This approach works well when:

  • UnitTesting the business logic of your Silverlight application.
  • When you have a pure windowless Silverlight application that standsalone and has very limited interaction with the HTML on the page.

This approach might not be suiteable for:

  • Heavy UI applications that perform heavy interactions with other HTML elements on the page. Especially when HTML elements trigger events in Silverlight Apps or when using Ajax on your pages.
  • Automating usage scenarios or end-to-end scenarios that require a browser host.
  • Testing your application in different browsers (IE/Firefox/Safari)

2. Browser Testing: Basically automating the browser by accessing the HTML DOM of the page including the XAML DOM of the hosted Silverlight app(s) and being able to execute actions and query states of these two DOM trees in a consistent and interchangeable manner. This approach is suitable for the scenarios described above that are not fit for Isolated API/UnitTesting.

Here at ArtOfTest, we’ve spent some time prototyping a few solutions that we think can help solve some of the challenges introduced by Silverlight. We are currently focused on approach #2 above since we think the progression of leveraging Silverlight is going to be through integration of small Silverlight apps within existing HTML/Ajax applications.

So, how are we going about supporting Silverlight?

Well, our web automation infrastructure WebAii currently provides a rich infrastructure for automating HTML based web applications including rich support for Ajax. One of our goals is to extend WebAii’s rich support to Silverlight. We want to extend our current API to be used interchangeably between HTML elements and XAML elements. This API should allow both action invokation and data retrieval for both element types (XAML & HTML). If we can accomplish that we can offer customers:

  • One consistent API for automating the HTML DOM and the XAML DOM.
  • Eliminate the need to learn a new automation framework for Silverlight and the need to integrate yet another new library into testing environments.
  • Enable a single test case to automate and interact with both the HTML & XAML elements on the same page to allow testing of the interaction points that mix HTML/XAML applications might have.
  • No breaking changes to existing testcases that are currently using WebAii. You can simply upgrade the library and then slowly update existing tests to include some Silverlight scenarios or start writing new automated tests for Silverlight within your existing testbed.

In my next post I’ll be digging into how WebAii’s Silverlight support will look like and discuss some of the current automation limitations with Silverlight and what Microsoft can do to help improve its testability.

Stay tuned!


PART II

Tuesday, May 22, 2007

Light-Weight ASP.NET Unit Testing - Go Browser-less, Server-less!

ASP.NET comes with great set of features especially for extensibility. Web Hosting is one of those features that is extremely poweful but not widely used since it's not really common for web developers to need to host their own ASP.NET processing server.

When it comes to testing, there is great power and agility gained with having a host processing requests within the test process. It allows us to eliminate the need for a browser and a webserver. It allows for an extemely quick and light-weight execution of tests that is suited for scenarios such as:

  • Running Build Verification Tests (aka BVTs): These tests commonly run as part of a continuous integration server process and they are very basic tests that ensure basic functionality and cover a broad set of features but do not go deep into each feature. BVTs can also be used during the dev check-ins process to ensure certain code quality at check-in. Having a quick turn around on these tests is important so they don't become a bottle neck for product building and development check-in.
  • UnitTesting business logic behind UI elements: If you want to write unit tests for your business logic that is tightly coupled with certain UI elements or UI logic.
  • Any other testing that does not require client javascript execution or a browser.

The Plasma project on codeplex attempts to accomplish that but is still at a very early stage in development. With WebAii Beta2 refresh, we updated WebAii to include a similar Asp.Net host and integrated it with our DOM parser, element identification engine and Actions interface. We enabled the scenario of executing ASP.NET Unit Tests using a light weight process without a need for a browser or a webserver.

The Actions object implements all the common actions like Click(), SetText(), SelectDropDown()..etc. which is the same interface you would use to automate IE or Firefox; so there is no new interface to learn or setups to perform. Simply flip a switch in your settings and you are automating without a browser or a server.

So what do you need to do to enable automation with WebAii using an in-process host? Simply set the defaultBrowser config setting to AspNetHost or set it in your code if you are not using a config:




<WebAii.Settings


defaultBrowser="AspNetHost"


logLocation="c:\Log\"


webAppPhysicalPath="C:\Aot\webtest\artoftest.testing\pages"


enableScriptLogging="false"


>



Or set it in your test code:

settings.DefaultBrowser = BrowserType.AspNetHost;

And Voila! That is about it. You can automate your tests using the same Find.Byxx or Actions..xx and with no browser/server needed.

What about debugging?

With browser-less automation, you get the quick, snappy execution but you lose the visual rendering of your pages. But that is no longer the case with WebAii. If you want to visually view the responses coming back from the in-process server, simply set the setting:
enableUILessRequestViewing to true and you will get an IE instance that simply renders the response visually. You can set break-points in your code if the execution is going too fast to notice specific pages.

If you want to learn more about this feature, check out our documentation on the in-process host here. You can also download our complete samples here which include a sample using AspNetHost.

Wednesday, April 4, 2007

WebAii Beta2 Released!

We are happy to announce the release of WebAii Beta2. Beta2 offers over 20 new features in addition to what was included in Beta1.

Some of the exciting new features in Beta2:

JavaScript Testing Support:
o Invoke JavaScript functions directly from .NET test code.
o Invoke JavaScript element events. (i.e. onblur).
o Log directly from your JavaScript functions.

Richer Element Search API
o Find elements using their markup content (innerText, innerMarkup, regex).
o Quick and easy notations to craft complex searches.
o Isolate application details from test logic using serialization support.

Additional Ajax Support
o Wait for 1-N element changes using one call.

NUnit First-Class Support
o Visual Studio NUnit item template.
o NUnit base test class.

HTML Pop-up & Dialogs Support
o Access and automate your HTML pop-ups.
o Built-in support for common browser dialogs (Alerts, FileUploads).
o Extensible framework for building your own custom dialog support.

Design ASP.NET Application Testability directly in VS Designer.
o New ASP.NET TestRegion custom control to use in Visual Studio’s designer to enable you to visually build TestRegions into your web application.

Unit Test Web Extensions
o 2 new test fixtures for defining and sharing elements and dialog handlings across unit tests.

Complete Code Samples & Updated Documentation / WebSite
o 50 unit tests that come in 4 flavors (C#/VB.NET for NUnit or VS Team Test)

And much more at www.artoftest.com

Feel free to send us any feedback!