Testing Tools For C#
Test Design Tools Tools that help you decide what tests need to be executed. Test data and test case generators. Total 15 tools. GUI Test Drivers Tools that automate execution of tests for products with graphical user interfaces. Client/server test automation tools, including load testers, also go here. Total 28 tools. Load and Performance Tools.
C# Integration Testing Tools
-->Note
This article applies to Visual Studio 2015. If you're looking for the latest Visual Studio documentation, use the version selector at the top left. We recommend upgrading to Visual Studio 2019. Download it here
Automated tests that drive your application through its user interface (UI) are known as coded UI tests (CUITs). These tests include functional testing of the UI controls. They let you verify that the whole application, including its user interface, is functioning correctly. Coded UI Tests are particularly useful where there is validation or other logic in the user interface, for example in a web page. They are also frequently used to automate an existing manual test.
As shown in the following illustration, a typical development experience might be one where, initially, you simply build your application (F5) and click through the UI controls to verify that things are working correctly. You then might decide to create a coded test so that you don’t need to continue to test the application manually. Depending on the particular functionality being tested in your application, you can write code for either a functional test, or for an integration test that might or might not include testing at the UI level. If you simply want to directly access some business logic, you might code a unit test. However, under certain circumstances, it can be beneficial to include testing of the various UI controls in your application. A coded UI test can automate the initial (F5) scenario, verifying that code churn does not impact the functionality of your application.
Creating a coded UI test is easy. You simply perform the test manually while the CUIT Test Builder runs in the background. You can also specify what values should appear in specific fields. The CUIT Test Builder records your actions and generates code from them. After the test is created, you can edit it in a specialized editor that lets you modify the sequence of actions.
Alternatively, if you have a test case that was recorded in Microsoft Test Manager, you can generate code from that. For more information, see Record and play back manual tests.
The specialized CUIT Test Builder and editor make it easy to create and edit coded UI tests even if your main skills are concentrated in testing rather than coding. But if you are a developer and you want to extend the test in a more advanced way, the code is structured so that it is straightforward to copy and adapt. For example, you might record a test to buy something at a website, and then edit the generated code to add a loop that buys many items.
Requirements
Visual Studio Enterprise
Sun tv serials today episode. For more information about which platforms and configurations are supported by coded UI tests, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings.
In this topic
Creating Coded UI Tests
Create a Coded UI Test project.
Coded UI tests must be contained in a coded UI test project. If you don’t already have a coded UI test project, create one. In Solution Explorer, on the shortcut menu of the solution, choose Add, New Project and then select either Visual Basic or Visual C#. Next, choose Test, Coded UI Test.
I don't see the Coded UI Test project templates.
You might be using a version of Visual Studio that does not support coded UI tests. To create coded UI tests, you must use Visual Studio Enterprise.
Add a coded UI test file.
If you just created a Coded UI project, the first CUIT file is added automatically. To add another test file, open the shortcut menu on the coded UI test project, point to Add, and then choose Coded UI Test.
In the Generate Code for Coded UI Test dialog box, choose Record actions, edit UI map or add assertions.
The Coded UI Test Builder appears and Visual Studio is minimized.
Record a sequence of actions.
To start recording, choose the Record icon. Perform the actions that you want to test in your application, including starting the application if that is required.
For example, if you are testing a web application, you might start a browser, navigate to the web site, and log in to the application.
To pause recording, for example if you have to deal with incoming mail, choose Pause.
Warning
All actions performed on the desktop will be recorded. Pause the recording if you are performing actions that may lead to sensitive data being included in the recording.
To delete actions that you recorded by mistake, choose Edit Actions.
To generate code that will replicate your actions, choose the Generate Code icon and type a name and description for your coded UI test method.
Verify the values in UI fields such as text boxes.
Choose Add Assertions in the Coded UI Test Builder, and then choose a UI control in your running application. In the list of properties that appears, select a property, for example, Text in a text box. On the shortcut menu, choose Add Assertion. In the dialog box, select the comparison operator, the comparison value, and the error message.
Close the assertion window and choose Generate Code.
Tip
Alternate between recording actions and verifying values. Generate code at the end of each sequence of actions or verifications. If you want, you will be able to insert new actions and verifications later.
For more details, see Validating Properties of Controls.
View the generated test code.
To view the generated code, close the UI Test Builder window. In the code, you can see the names that you gave to each step. The code is in the CUIT file that you created:
Add more actions and assertions.
Place the cursor at the appropriate point in the test method and then, on the shortcut menu, choose Generate Code for Coded UI Test. New code will be inserted at that point.
Edit the detail of the test actions and the assertions.
Open UIMap.uitest. This file opens in the Coded UI Test Editor, where you can edit any sequence of actions that you recorded as well as edit your assertions.
For more information, see Editing Coded UI Tests Using the Coded UI Test Editor.
Run the test.
Use Test Explorer, or open the shortcut menu in the test method, and then choose Run Tests. For more information about how to run tests, see Run unit tests with Test Explorer and Additional options for running coded UI tests in the What’s next? section at the end of this topic.
The remaining sections in this topic provide more detail about the steps in this procedure.
For a more detailed example, see Walkthrough: Creating, Editing and Maintaining a Coded UI Test. In the walkthrough, you will create a simple Windows Presentation Foundation (WPF) application to demonstrate how to create, edit, and maintain a coded UI test. The walkthrough provides solutions for correcting tests that have been broken by various timing issues and control refactoring.
Starting and stopping the application under test
I don’t want to start and stop my application, browser, or database separately for each test. How do I avoid that?
If you do not want to record the actions to start your application under test, you must start your application before you choose the Record icon.
At the end of a test, the process in which the test runs is terminated. If you started your application in the test, the application usually closes. If you do not want the test to close your application when it exits, you must add a .runsettings file to your solution and use the
KeepExecutorAliveAfterLegacyRunoption. For more information, see Configure unit tests by using a .runsettings file.You can add a test initialize method, identified by a [TestInitialize] attribute, which runs code at the start of each test method. For example, you could start the application from the TestInitialize method.
You can add a test cleanup method, identified by a [TestCleanup] attribute, that runs code at the end of each test method. For example, the method to close the application could be called from the TestCleanup method.
Validating the properties of UI controls
You can use the Coded UI Test Builder to add a user interface (UI) control to the UIMap for your test, or to generate code for a validation method that uses an assertion for a UI control.
To generate assertions for your UI controls, choose the Add Assertions tool in the Coded UI Test Builder and drag it to the control on the application under test that you want to verify is correct. When the box outlines your control, release the mouse. The control class code is immediately created in the UIMap.Designer.cs file.
The properties for this control are now listed in the Add Assertions dialog box.
Another way of navigating to a particular control is to choose the arrow (<<) to expand the view for the UI Control Map. To find a parent, sibling, or child control, you can click anywhere on the map and use the arrow keys to move around the tree.
I don’t see any properties when I select a control in my application, or I don’t see the control in the UI Control Map.
In the application code, the control that you want to verify must have a unique ID, such as an HTML ID attribute, or a WPF UId. You might need to update the application code to add these IDs.
Next, open the shortcut menu on the property for the UI control that you want to verify, and then point to Add Assertion. In the Add Assertion dialog box, select the Comparator for your assertion, for example AreEqual, and type the value for your assertion in Comparison Value.
When you have added all your assertions for your test, choose OK.
To generate the code for your assertions and add the control to the UI map, choose the Generate Code icon. Type a name for your coded UI test method and a description for the method, which will be added as comments for the method. Choose Add and Generate. Next, choose the Close icon to close the Coded UI Test Builder. This generates code similar to the following code. For example, if the name you entered is
AssertForAddTwoNumbers, the code will look like this example:Adds a call to the assert method AssertForAddTwoNumbers to the test method in your coded UI test file:
You can edit this file to change the order of the steps and assertions, or to create new test methods. To add more code, place the cursor on the test method and on the shortcut menu choose Generate Code for Coded UI Test.
Adds a method called
AssertForAddTwoNumbersto your UI map (UIMap.uitest). This file opens in the Coded UI Test Editor, where you can edit the assertions.For more information, see Editing Coded UI Tests Using the Coded UI Test Editor.
You can also view the generated code of the assertion method in UIMap.Designer.cs. However, you should not edit this file. If you want to make an adapted version of the code, copy the methods to another file such as UIMap.cs, rename the methods, and edit them there.
The control I want to select loses focus and disappears when I try to select the Add Assertions tool from the Coded UI Test Builder. How do I select the control?Selecting a hidden control using the keyboard
Sometimes, when adding controls and validating their properties, you might have to use the keyboard. For example, when you try to record a coded UI test that uses a context menu control, the list of menu items in the control will lose focus and disappear when you try to select the Add Assertions tool from the Coded UI Test Builder. This is demonstrated in the following illustration, where the context menu in Internet Explorer will lose focus and disappear if you try to select it with the Add Assertions tool.
To use the keyboard to select a UI control, hover over the control with the mouse. Then hold down the Ctrl key and the I key at the same time. Release the keys. The control is recorded by the Coded UT Test Builder.
Warning
If you use Microsoft Lync, you must close Lync before you start the Coded UI Test Builder. Microsoft Lync interferes with the Ctrl+I keyboard shortcut.
I can’t record a mouse hover on a control. Is there a way around this?Manually recording mouse hovers
Under some circumstances, a particular control that’s being used in a coded UI test might require you to use the keyboard to manually record mouse hover events. For example, when you test a Windows Form or a Windows Presentation Foundation (WPF) application, there might be custom code. Or, there might be special behavior defined for hovering over a control, such as a tree node expanding when a user hovers over it. To test circumstances like these, you have to manually notify the Coded UI Test Builder that you are hovering over the control by pressing predefined keyboard keys.
When you perform your coded UI test, hover over the control. Then press and hold Ctrl, while you press and hold the Shift and R keys on your keyboard. Release the keys. A mouse hover event is recorded by the Coded UT Test Builder.
After you generate the test method, code similar to the following example will be added to the UIMap.Desinger.cs file:
The key assignment for capturing mouse hover events is being used elsewhere in my environment. Can I change the default key assignment?Configuring mouse hover keyboard assignments
If necessary, the default keyboard assignment of Ctrl+Shift+R that is used to apply mouse hovering events in your coded UI tests can be configured to use different keys.
Warning
You should not have to change the keyboard assignments for mouse hover events under ordinary circumstances. Use caution when reassigning the keyboard assignment. Your choice might already be in use elsewhere within Visual Studio or the application being tested.
To change the keyboard assignments, you must modify the following configuration file:
<drive letter:>Program Files (x86)Microsoft Visual Studio 11.0Common7IDECodedUITestBuilder.exe.config
In the configuration file, change the values for the HoverKeyModifier and HoverKey keys to modify the keyboard assignments:
I’m having issues with recording mouse hovers on a website. Is there a fix for this, too?Setting implicit mouse hovers for the web browser
In many websites, when you hover over a particular control, it expands to show additional details. Generally, these look like menus in desktop applications. Because this is a common pattern, coded UI tests enable implicit hovers for Web browsing. For example, if you record hovers in Internet Explorer, an event is fired. These events can lead to redundant hovers getting recorded. Because of this, implicit hovers are recorded with ContinueOnError set to true in the UI test configuration file. This allows playback to continue if a hover event fails.
To enable the recording of implicit hovers in a Web browser, open the configuration file:
<drive letter:>Program Files (x86)Microsoft Visual Studio 11.0Common7IDECodedUITestBuilder.exe.config
Verify that the configuration file has the key RecordImplicitiHovers set to a to a value of true as shown in the following sample:
Customizing your coded UI test
After you’ve created your coded UI test, you can edit it by using any of the following tools in Visual Studio:
Coded UI Test Builder: Use the Coded UI Test Builder to add additional controls and validation to your tests. See the section Adding controls and validating their properties in this topic.
Coded UI Test Editor: The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods. You can also edit UI actions and their associated controls in the UI control map. For more information, see Editing Coded UI Tests Using the Coded UI Test Editor.
Code Editor:
Manually add code for the controls in your test as described in the Coding UI control actions and properties section in this topic.
After you create a coded UI test, you can modify it to be);
Accessing Properties of UI Test Control
To get and set UI control specific property values, you can directly get or set the values the properties of a control, or you can use the UITestControl.GetProperty and UITestControl.SetProperty methods with the name of the specific property that you want you get or set.
GetProperty returns an object, which can then be cast to the appropriate Type. SetProperty accepts an object for the value of the property.
To get or set properties from UI test controls directly
With controls that derive from T:Microsoft.VisualStudio.TestTools.UITesting.UITestControl, such as T:Microsoft.VisualStudio.TestTools.UITesting.HtmlControls.HtmlList or T:Microsoft.VisualStudio.TestTools.UITesting.WinControls.WinComboBox, you can get or set their property values directly, as follows:
To get properties from UI test controls
To get a property value from a control, use GetProperty.
To specify the property of the control to get, use the appropriate string from the
PropertyNamesclass in each control as the parameter to GetProperty.GetProperty returns the appropriate data type, but this return value is cast as an Object. The return Object must then be cast as the appropriate type.
Example:
int i = (int)GetProperty(myHtmlList.PropertyNames.ItemCount);
To set properties for UI test controls
To set a property in a control, use SetProperty.
To specify the property of the control to set, use the appropriate string from the
PropertyNamesclass as the first parameter to SetProperty, with the property value as the second parameter.Example:
SetProperty(myWinCheckBox.PropertyNames.Checked, true);
Debugging
You can analyze Coded UI tests using coded UI test logs. Coded UI test logs filter and record important information about your coded UI test runs. The format of the logs lets you debug issues quickly. For more information, see Analyzing Coded UI Tests Using Coded UI Test Logs.
What’s next?
Additional options for running coded UI tests: You can run coded UI tests directly from Visual Studio, as described earlier in this topic. Additionally, you can run automated UI tests from Microsoft Test Manager, or from Team Foundation Build. When coded UI tests are automated, they have to interact with the desktop when you run them, unlike other automated tests.
Adding support for custom controls: The coded UI testing framework does not support every possible UI and might not support the UI you want to test. For example, you cannot immediately create a coded UI test of the UI for Microsoft Excel. However, you can create an extension to the coded UI testing framework that will support a custom control.
Coded UI Tests are often used to automate manual tests. For additional guidance, see Testing for Continuous Delivery with Visual Studio 2012 – Chapter 5: Automating System Tests. For more information about manual tests, see [retired] Creating Manual Test Cases Using Microsoft Test Manager. For more information about automated system tests, see Creating Automated Tests Using Microsoft Test Manager.
External Resources
Guidance
FAQ
Forum
See also
This is a list of Automated Testing Frameworks for .NET related to methodologies and types of tests:
- Test-Driven Development (TDD)
- Behavior-Driven Development (BDD)
- Specification by Example (SBE)
- Acceptance Test-Driven Development (ATDD)
- Property-Based Testing (PBT)
- Unit / Integration / Acceptance / Specification / etc. Tests
Please feel free to suggest changes and/or new tools/frameworks.
Key:
- Bold — Most Popular
| Framework | Licensing | Comment |
|---|---|---|
| csUnit | Free and open source | Discontinued |
| EMTF | Free and open source | - Known as Embeddable Micro Test Framework - Discontinued |
| Expecto | Free and open source | F# |
| FsUnit | Free and open source | F# |
| Fuchu | Free and open source | F# / C# / VB.NET |
| MbUnit | Free and open source | Discontinued |
| MSTest | Free and open source | - Also known as Microsoft Test Framework - Part of Visual Studio |
| NUnit | Free and open source | |
| NUnitLite | Free and open source | Discontinued |
| Roaster | Free and open source | Discontinued |
| Unquote | Free and open source | F# |
| xUnit.net | Free and open source |
| Framework | Comment |
|---|---|
| FakeItEasy | |
| Foq | |
| JustMock | |
| JustMock Lite | |
| Microsoft Fakes | Previously known as Microsoft Moles |
| Moq | |
| NMock | |
| NSubstitute | |
| NUnit.Mocks | Discontinued |
| Rhino Mocks | Discontinued |
| Typemock Isolator |
| Framework | Comment |
|---|---|
| ApprovalTests | |
| Avignon | |
| BDDfy | |
| Concordion.NET | |
| Cucumber | |
| Cuke4Nuke | Discontinued |
| FitNesse | |
| Gauge | |
| HonestCode | |
| LightBDD | |
| Machine.Specifications | Also known as MSpec |
| NaturalSpec | Discontinued |
| NBehave | |
| NDecision | Discontinued |
| NSpec | |
| Robot Framework | |
| Spec4Net | Discontinued |
| SpecFlow | |
| SpecsFor | |
| Specter | |
| StoryQ | Discontinued |
| StoryTeller | |
| SubSpec | |
| System.Spec | |
| TickSpec | |
| xBehave.net |
| Framework | Comment |
|---|---|
| Atata | |
| Canopy | F# |
| Coypu | |
| FluentAutomation | Discontinued |
| IeUnit | Discontinued |
| Ivonna | Discontinued |
| Netling | Load tests for web |
| NUnitAsp | Discontinued |
| Puppeteer Sharp | |
| Selenium | |
| Test.Automation | |
| TestLeft | |
| TestStack.Seleno | |
| WatiN | Discontinued |
| Framework | Comment |
|---|---|
| FakeHttpContext | |
| Flurl | |
| MockHttp | |
| MockingBird | Discontinued |
| MyTested.AspNetCore.Mvc | |
| Stubbery | API stubs |
| Framework | Comment |
|---|---|
| Microsoft Coded UI | Part of Visual Studio |
| NUnitForms | |
| TestStack.White |
| Framework | Comment |
|---|---|
| DbUnit.NET | Discontinued |
| NDbUnit | |
| Respawn | A small utility to help in resetting test databases to a clean state |
| Framework | Comment |
|---|---|
| FluentAssertions.Extensions | |
| Microsoft CHESS | |
| ThreadTester | |
| Typemock Racer | Discontinued |
| Framework | Comment |
|---|---|
| .NET Memory Profiler | Also known as MemProfiler |
| dotMemory Unit |
| Framework | Comment |
|---|---|
| CREAM | Also known as CREAtor of Mutants |
| Fettle | |
| Nester | |
| NinjaTurtles | |
| PIT | |
| VisualMutator |
| Framework | Comment |
|---|---|
| Microsoft IntelliTest | Part of Visual Studio Previously known as: - Microsoft Code Digger - Microsoft Pex - Microsoft Smart Unit Tests |
| Randoop.NET |
| Framework | Comment |
|---|---|
| FsCheck | Port of QuickCheck |
| Framework | Comment |
|---|---|
| AxoCover | Bases on OpenCover |
| Coverlet | |
| dotCover | |
| NCover | |
| NCrunch | |
| NDepend | |
| OpenCover | |
| PartCover | Discontinued |
| Semantic Designs C# Test Coverage Tool | |
| Software Verify .NET Coverage Validator | |
| Squish Coco | |
| TestMatrix | |
| Typemock Isolator Coverage | |
| Visual Studio Code Coverage |
| Framework | Comment |
|---|---|
| Continuous Testing for Visual Studio 2010 | Discontinued |
| ContinuousTests | Formerly Mighty Moose |
| dotCover | |
| Giles | |
| Live Unit Testing | Part of Visual Studio |
| NCrunch | |
| Parasoft dotTEST | |
| Typemock Isolator Smart Runner |
| Framework | Comment |
|---|---|
| Fluent Assertions | |
| NFluent | |
| SharpTestEx | Discontinued |
| Should Assertion Library | Discontinued |
| Shouldly |
| Framework | Comment |
|---|---|
| Any-.Net | Discontinued |
| AutoFixture | |
| AutoPoco | Discontinued |
| Bogus | |
| Fabricator | Discontinued |
| Faker.Net | |
| GenFu | |
| Hydrator | Discontinued |
| NBuilder | |
| NTestDataBuilder | - Now TestStack.Dossier - Discontinued |
| TestData | |
| TestDataFactory | Discontinued |
| TestDataGenerator | Discontinued |
| TestStack.Dossier |
| Framework | Comment |
|---|---|
| AutoMoq | Auto mocking provider for Moq |
| ConventionTests | Library that makes it easy to build validation rules for convention validation tests |
| Fixie | Convention for tests |
| FluentMvcTesting | Type-safe tests against ASP.NET MVC Controllers |
| SparkyTestHelpers | Unit test helpers for config files, ASP.NET MVC, and Moq among others |
| XMLUnit | Unit testing XML |
| Framework | Comment |
|---|---|
| Gallio Automation Platform | Discontinued |
| ErrorUnit | Debug C# application by automatically creating C# Unit Tests in Visual Studio that recreate the situation leading up to the error |
| KREM | Automation and test framework. Integration, regression, spec testing. Well suitable for embedded. Written in Python, but support external scripts, etc. |
| NBi | Framework to test Business Intelligence |
| Quality Gate One Studio | Combinatorial and Model-Based Testing |
| TestFlask | Recording and mock replay framework with the ability to generate unit tests for recorded scenarios. It also provides some tools to ease scenario testing inside ASP.NET MVC apps. |
| Framework | Comment |
|---|---|
| nMate | - Unit test code generation - Discontinued |
| QuickUnit Unit Test Designer | |
| ReSharper | |
| TestDriven.Net | |
| Unit Test Boilerplate Generator | |
| Unit Test Generator | Discontinued |
| Visual T# | Discontinued |
| WiseTester - Overcome Unit Test Failures |