Tag Archives: SpecFlow

Reqnroll v3 migration: Dealing with your custom [Given], [When], [Then] attributes

In a previous post, I showed how to make your step-definition code more maintainable in SpecFlow. If you migrate from SpecFlow/Reqnroll v2 to Reqnroll v3 then this post is for you!

TL;DR: Updated attribute for Reqnroll v3

Assuming you’re on .NET 8+ this class works for you. If you’re on an earlier version, just replace the collection expression with your preferred initializer.

public class GivenWhenAttribute : StepDefinitionBaseAttribute
{
    static readonly StepDefinitionType[] types = [
        StepDefinitionType.Given, 
        StepDefinitionType.When
    ];
    public GivenWhenAttribute() : this(null!) { }
    public GivenWhenAttribute(string expression) : base(expression, types) { }
}

Challenge: Culture no longer exists

The old SpecFlow-era overload set a Culture member via a constructor. In Reqnroll v3, that member is gone, so we remove this overload:

public GivenWhenAttribute(string regex, string culture) 
  : this(regex) { Culture = culture; }

Challenge: step definition not discovered

After solving the Culture member, everything compiled fine, but binding discovery failed with this exception:

Message: 
Reqnroll.MissingStepDefinitionException : No matching step definition found for one or more steps.
Stack Trace: 
ErrorProvider.ThrowPendingError(ScenarioExecutionStatus testStatus, String message)
TestExecutionEngine.OnAfterLastStepAsync()
TestRunner.CollectScenarioErrorsAsync()
ReproductionFeature.ScenarioCleanupAsync()
ReproductionFeature.CantFindStep() line 8
...

Reqnroll discovers step definitions via reflection. It looks for methods having an attribute derived from StepDefinitionBaseAttribute

In V3 the parameter names of the constructors on your attribute must match Reqnroll’s expected names. If they don’t, Reqnroll does not recognize the attribute and discovery fails. Just changing the parameter name from regex to expression solves this problem.

Reuse a SpecFlow step definition for Given and When

Have you ever wanted to use a single step definition like I create file '(.*)' from Given and When contexts in a feature file like this?

Scenario: CreateFile
    When I create file 'hello.txt'
    Then ...
Scenario DeleteFile
    Given I create  file 'hello.txt'
    When I delete 'hello.txt'

Here’s how to have 1 method that implements both the Given and the When I create file '...'

💡 Prefer the newer Reqnroll v3 version?

This post explains the original SpecFlow approach. If you’re using Reqnroll v3 read the updated version instead.

First, create a class like this:

public class GivenWhenAttribute : StepDefinitionBaseAttribute
{
    readonly static StepDefinitionType[] types = new[] { StepDefinitionType.Given, StepDefinitionType.When };
    public GivenWhen() : this(null)  {  }
    public GivenWhen(string regex) : base(regex, types ) { }
    public GivenWhen(string regex, string culture) : this(regex) { Culture = culture; }
}

Then use it in the [Binding] classes like this:

[GivenWhen("I create file '(.*)'")]
public void CreateFile(String Name) { File.Create(Name); }

Voila! 1 step definition method now works for Given and When steps in the feature file.

Automatic type conversion of parameters in SpecFlow

In a previous post I showed that SpecFlow can change values of parameters. This mechanism is not just limited to transforming content of string values. It can also convert the literal string in your feature file to some complex object for your step-definition code.

Let say in your feature file you want to write steps like this:

Scenario: MyScenario
    When I print this list 'A,B,C,D'

Then you might be tempted to convert the string 'A,B,C,D' to a list like this 

[Binding]
public class MyBindings
{
    [StepDefinition(@"I print this list '([^']*)'") ]
    public void PrintList(string input)
    {
        var items = input.Split(',')
        foreach(var item in items)
        {
            Console.WriteLine(item)
        }
    }
}

Don’t do this…it causes the same problems as mentioned in the previous post.

Instead SpecFlow’s Step Argument Conversion lets us simplify our step definition code to this:

[Binding]
public class MyBindings
{
    [StepDefinition(@"I print this list '([^']*)'") ]
    public void PrintList(IEnumerable<string> items) 
    {
        foreach(var item in items)
        {
            Console.WriteLine(item)
        }
    }
   
    [StepArgumentTransformation]
    public IEnumerable<string> TransformStringToList(string input)
    {
        return input.Split(',');
    }
}

Automatic parsing of parameters in SpecFlow

Most implementations of cucumber can automatically replace parameter values with some run time value. Let say in your feature file you want to write steps like this:

Scenario: MyScenario
    When I print 'Hello from test case {test}' 
    When I save the value '{test}' to my database

Then you might be tempted to write these bindings (dont do this…keep reading!)

[Binding]
public class MyBindings
{
    [StepDefinition(@"I print '([^']*)'") ]
    public void PrintMessage(string Message)
    {
        string Name = ....
        Message = Message.Replace("{test}",Name)
        Console.WriteLine(Message);
    }

    [StepDefinition(@"I save '([^']*)' to my database") ]
    public void SaveToDatabase(string Message)
    {
        string Name = ....
        Message = Message.Replace("{test}",Name)
        Console.WriteLine(Message);
    }
}

This way of writing step definitions becomes a real problem as the size of your automation increases:

  1. we have to repeat the same logic in every step definition.
  2. Its easy to forget to repeat that code.
  3. New placeholders need you to find and update all places where this kind of parsing is done.
  4. Its code that’s needed but not relevant for the objective of the step definitions.

Luckily SpecFlow’s Step Argument Conversion helps us! We can simplify our binding code to this:

[Binding]
public class MyBindings
{
    [StepDefinition(@"I print '([^']*)'") ]
    public void PrintMessage(string Message) => Console.WriteLine(Message);

    [StepDefinition(@"I save '([^']*)' to my database") ]
    public void SaveToDatabase(string Message) => Console.WriteLine(Message);
    
    [StepArgumentTransformation]
    public string TransformParsedString(string input)
    {
        string Name = ...
        return input.Replace("{test}",Name);
    }
}

Other implementations of cucumber also implement this principle. Its frequently referred to using these buzzwords:

  1. Transforms / Transformations
  2. Parsing / Replacing
  3. StepArgumentTransformation

Running your testcases concurrently

System- and integration tests can take a lot of time to complete. If you want to speed this up then you can choose to run multiple test cases at the same time in parallel.

Unit Testcases in Visual Studio


You can configure Visual Studio to run unit concurrent testcases on your local machine using the following procedure:

  1. Ensure your project is using a .testsettings file
  2. Open it in an XML editor (Notepad++ or visual Studio’s “Open With” command)
  3. Set the parallelTestCount attribute on the Execution entity some some integer value

This will only work for situations where all the following is true:

  • The testcases and related frameworks are thread-safe
  • The testcases are of type unit testcase. It really won’t work for coded-ui tests
  • Its NOT a Data-Driven Unit Test
  • There are 0 diagnostic adapters included in the .testsettings file
  • Your machine has multiple cores

Controller and Agents with Visual Studio


If you have enough Physical or Virtual Machines available, then its always possible to configure a Controller and multiple agents. That will allow you to distribute your testcases over the machines. There’s no limitation on the type of testcases here. The only thing to take into account that running testcases that interact with a User Interface will require specific configuration on part of the agent machines. I use this setup mainly for running a lot of performance-tests against my client’s SharePoint farm.

See Configuring Test Controllers and Test Agents for Load Testing for details.

SpecFlow / SpecRun


SpecRun is a test-runner from the makers of SpecFlow that will allow you to run many concurrent SpecFlow scenarios. Its easy to setup and doesn’t really have any significant limitations. I use a combination of SpecFlow, SpecRun and an on-premise Selenium Grid to run my 250+ automated regression tests.

What to do when SpecRun doesn’t find any testcases to execute

So you’ve installed SpecRun, updated app.config, started a testrun….but SpecRun reports that it found 0 testcases. There’s two fixes for this.

Regenerate the feature files


Usually the *.feature.cs code-behinds aren’t removed when you clean the project. Those files still contain code pertaining to the previous test runner. You need to regenerate the .feature files in the solution before the code-behinds are updated to use the SpecRun test runner.

Clean Visual Studio’s cache for the test runners


Another cause for this unexpected behavior is due due to corruption in Visual Studio’s cache for the test runners. This can be fixed by the following:

  1. Close Visual Studio
  2. navigate to %TEMP%\VisualStudioTestExplorerExtensions\
  3. Delete any SpecRun related folders

Describing large forms and data entry in Cucumber / SpecFlow feature files

Did you know that in .feature files you can use the table notation to specify each field and value in rows instead of columns?

This can make the file way more readable when a step needs a lot of different parameters. Here is an example that shows both ways of creating an invoice:

Feature: Demo

Scenario: TwoDifferentStylesOfSteps
	When I create an invoice with number '111' with '119.00' euro total, '19.00' euro VAT from 'ACME Inc' at '1234XYZ 99' in 'USA'
	When I create the following invoice:
		| Field            | Value      |
		| Number           | 222        |
		| TotalAmount      | 238.00     |
		| VATAmount        | 38.00      |
		| Debtor           | ACME Inc   |
		| DebtorAdresLine1 | 1234XYZ 99 |
		| DebtorCountry    | USA        |
	Then The systems invoice store must look like this:
		| Number | TotalAmount | VatAmount |
		| 111    | 119.00      | 19.00     |
		| 222    | 238.00      | 38.00     |

Here is the corresponding binding code:

namespace TableExample
{
    public class Invoice
    {
        public int Number { get; set; }
        public decimal TotalAmount { get; set; }
        public decimal VatAmount { get; set; }
        public string Debtor { get; set; }
        public string DebtorAdresLine1 { get; set; }
        public string DebtorAdresLine2 { get; set; }
        public string DebtorCountry { get; set; }
    }
  
    [Binding]
    public class TablesSteps
    {
        private IList<Invoice> Invoices;

        public TablesSteps()
        {
            this.Invoices = new List<Invoice>();
        }


        [When(@"I create an invoice with number '(.*)' with '(.*)' euro total, '(.*)' euro VAT from '(.*)' at '(.*)' in '(.*)'")]
        public void WhenICreateAnInvoiceWithNumberWithEuroTotalEuroVATFromAtIn(int Number, 
            decimal TotalAmount,
            decimal VatAmount,
            string Debtor,
            string DebtorLine1,
            string DebtorCountry)
        {
            Invoice TestInvoice = new Invoice();
            TestInvoice.Number = Number;
            TestInvoice.TotalAmount = TotalAmount;
            TestInvoice.VatAmount = VatAmount;
            TestInvoice.Debtor = Debtor;
            TestInvoice.DebtorAdresLine1 = DebtorLine1;
            TestInvoice.DebtorCountry = DebtorCountry;
            this.Invoices.Add(TestInvoice);
        }

        [When(@"I create the following invoice:")]
        public void WhenICreateTheFollowingInvoice(Table table)
        {
            Invoice TestInvoice = table.CreateInstance<Invoice>();
            this.Invoices.Add(TestInvoice);
        }

        [Then(@"The systems invoice store must look like this:")]
        public void ThenTheSystemsInvoiceStoreMustLookLike(Table table)
        {
            var rows = table.CreateSet<Invoice>();
            foreach (Invoice test in rows)
            {
                IEnumerable<Invoice> Matches = this.Invoices
                    .Where(invoice => invoice.Number.Equals(test.Number))
                    .Where(invoice => invoice.TotalAmount.Equals(test.TotalAmount))
                    .Where(invoice => invoice.VatAmount.Equals(test.VatAmount));
                Assert.AreEqual(1, Matches.Count(), string.Format("Invoice {0} not correct in invoicestore", test.Number));
            }
        }

    }

}

Combining SpecFlow and Selenium

One of the cool things I like to do when testing webapps, is to define the testcases using SpecFlow (also known as Cucumber) and then use Selenium to actually execute the testcases against the web-application.

Here is an example testcase:

Feature: SpecFlowAndSelenium
	For demonstration purposes, I want to show how the human-readable
	SpecFlow testcases can be executed using Selenium to operate
	a real webbrowser

Scenario: SeleniumAndSpecFlow
	Given I navigate to 'https://www.google.com'
	Then there must be a control with 'id' 'gbqfq' 
	When I type 'Hello World!' into it
	Then there must be a control with 'id' 'gbqfb'
	When I click on it
	Then there must be a link with 'partialtext' 'Wikipedia' 
	When I click on it
	Then there must be a control with 'id' 'searchInput'
	When I type 'Selenium' into it
	Then there must be a control with 'id' 'searchButton' 
	When I click on it
	Then there must be a link with 'text' 'chemical element'
	And there must be a link with 'text' 'Selenium (software)'
	When I click on it
	Then there must be a link with 'text' 'web browsers'

And here you can see Selenium using Firefox to perform all the actions and checks:
https://youtu.be/3nIaD4-Qi10

Automatic conversion of Tables to your C# classes using Specflow.Assist.Dynamic

Using tables in .feature files is good practice for creating readable and maintainable testcases. When you create the step definitions, you’ll frequently find yourself creating code to convert the fields in the Table class to something more useful for your code. Using Specflow.Assist.Dynamic we can have automatic conversion between the Table class and a collection of our own classes. In this post we’ll see how to get this working for a basic example

Install the NuGet package

PM> Install-Package Specflow.Assist.Dynamic
Attempting to resolve dependency 'ImpromptuInterface (≥ 5.6.2)'.
Attempting to resolve dependency 'SpecFlow'.
Installing 'ImpromptuInterface 5.6.2'.
Successfully installed 'ImpromptuInterface 5.6.2'.
Installing 'SpecFlow.Assist.Dynamic 1.0.2'.
Successfully installed 'SpecFlow.Assist.Dynamic 1.0.2'.
Adding 'ImpromptuInterface 5.6.2' to TableConversion.
Successfully added 'ImpromptuInterface 5.6.2' to TableConversion.
Adding 'SpecFlow.Assist.Dynamic 1.0.2' to TableConversion.
Successfully added 'SpecFlow.Assist.Dynamic 1.0.2' to TableConversion.

This has updated our App.config with the following:

<specFlow>
    <!-- For additional details on SpecFlow configuration options see https://go.specflow.org/doc-config -->
  <stepAssemblies>
      <!-- This attribute is required in order to use StepArgument Transformation as described here; 
    https://github.com/marcusoftnet/SpecFlow.Assist.Dynamic/wiki/Step-argument-transformations  -->
      <stepAssembly assembly="SpecFlow.Assist.Dynamic" />
    </stepAssemblies>
  </specFlow>

Create a .feature file with a table in it

Create the following .feature file:

Feature: TableConversion
	In order to avoid lots of code, I want my Tables in the feature file 
	to be automatically converted to an object I can use in the code of my 
	step definitions.

Scenario: TableConversionExample
	Given the following table:
	| id | sometext        | somebool |
	| 1  | Hello World     | True     |
	| 2  | and again hello | False    |
	Then my binding should have the following objects:
	| id | sometext           | somebool |
	| 1  | Hello World        | True     |
	| 2  | and again hello    | False    |

Create the following binding

using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Collections.Generic;
using TechTalk.SpecFlow;
using TechTalk.SpecFlow.Assist;

namespace TableConversion
{
    class MyPocoClass
    {
        public int    id       { get; set; }
        public string sometext { get; set; }
        public bool?  somebool { get; set; }
    }

    [Binding]
    public class TableConversionSteps
    {
        private Dictionary<int, MyPocoClass> MyObjects;

        public TableConversionSteps()
        {
            this.MyObjects = new Dictionary<int, MyPocoClass>();

        }

        [Given(@"the following table:")]
        public void GivenTheFollowingTable(Table table)
        {
            var rows = table.CreateSet<MyPocoClass>();
            //Store each row/object in a dictionary using its id as key
            foreach (MyPocoClass poco in rows)
            {
                this.MyObjects.Add(poco.id, poco);
            }
        }

        [Then(@"my binding should have the following objects:")]
        public void ThenMyBindingShouldHaveTheFollowingObjects(Table table)
        {
            var ExpectedObjects = table.CreateSet<MyPocoClass>();

            //Check that each object that should be present, really is present
            foreach (MyPocoClass ExpectedObject in ExpectedObjects)
            {
                MyPocoClass ActualObject = this.MyObjects[ExpectedObject.id];
                if(false == ActualObject.sometext.Equals(ExpectedObject.sometext))
                {
                    Assert.Fail(String.Format(
                        "Expected sometext '{0}', actual text was {1}",
                        ExpectedObject.sometext,
                        ActualObject.sometext));
                }
                
                if(ActualObject.somebool != ExpectedObject.somebool)
                {
                    Assert.Fail(String.Format(
                        "Expected somebool '{0}', actual somebool was {1}",
                        ExpectedObject.somebool,
                        ActualObject.somebool));
                }
            }
        }
    }
}

Run the unit test

When you run this test, you’ll see that it passes. Somethings to remember:

  • The framework maps the name of the fields in the table header to a property in your poco class. If it cant find that property, no errors will be raised

  • Conversion of text in the .feature file to a C# property strips/trims white-space before and after the text in the .feature file. White-space in the middle will remain and be included in your C# property

  • The framework uses the .Net methods to parse the fields. Be careful, the text “1” will not result in a True value in your boolean fields in the poco class

Setting the language for your .feature files

The syntax of the .feature files is meant to be readable by your users, so of course its logical that we should be able to create .feature files in many different languages.

How to set the language to use

Setting the language for all .feature files

If you want to set the language for all .feature files, then you can specify this in the App.config:

  <specFlow>
    ...
    <language feature="nl-NL" />
    ...
   </specflow>

Setting the language per .feature file

If you want set the language for a single .feature file then you can use a comment in the header of your feature file:

#language: nl-NL
Functionaliteit: Optellen van getallen
...

How to find the syntax for your language

Now that you’ve set your desired language, you can start creating .feature files using the keywords in your user’s language. How do we find out what these keyword are? Take a look at the Gherkin i18n file and you’ll be able to figure it out.

"lv": {
"name": "Latvian",
"native": "latviešu",
"feature": "Funkcionalitāte|Fīča",
"background": "Konteksts|Situācija",
"scenario": "Scenārijs",
"scenario_outline": "Scenārijs pēc parauga",
"examples": "Piemēri|Paraugs",
"given": "*|Kad",
"when": "*|Ja",
"then": "*|Tad",
"and": "*|Un",
"but": "*|Bet"
},
"nl": {
"name": "Dutch",
"native": "Nederlands",
"feature": "Functionaliteit",
"background": "Achtergrond",
"scenario": "Scenario",
"scenario_outline": "Abstract Scenario",
"examples": "Voorbeelden",
"given": "*|Gegeven|Stel",
"when": "*|Als",
"then": "*|Dan",
"and": "*|En",
"but": "*|Maar"
},