Home > Programming, Test and Automation Stuffs > Gherkin BDD comparison

Gherkin BDD comparison

Hi guys, I will post a research about the BDD engines, it is a bit old (2012), but despite this I found very interesting. I have summarised the relevant informations. The actual language to write tests with BDD is called Gherkin. And it has different implementations adopted to different programming language, the most famous:

Cucumber for Ruby

JBehave for Java

NBehave and SpecFlow for C#

Freshen for Python Behat for PHP

All of them have some common set of supported features but there’re some restrictions and abilities specific to the actual engine. So, we will collect useful features for each listed above engine and present it in some comparable form. Key features to be mentioned are:

  • Documentation availability
  • Flexibility in passing parameters
  • Auto-complete
  • Steps, scenario and feature scoping
  • Complex steps
  • Hooks and pre-conditions
  • Binding to code
  • Formatting flexibility
  • Built-in reports
  • Input data sources support
Grade Criteria
0 No support at all
1 Functionality exists but with serious restrictions
2 Major functionality exists
3 Full-featured support

Documentation Availability

  • Cucumber:
    • Cucumber group on LinkedIn – quite populated place with big number of active discussions
    • Cukes Tutorial Site
  • Freshen – honestly speaking I didn’t find any specialized resourse dedicated to freshen only. Most likely it can be discussed in the more general forums dedicated to BDD in general.
  • JBehave:
  • SpecFlow:
  • Behat:


With this sources to look the documentation of each engine, we can evaluate the criteria and the grades:

GRADE CRITERIA
1 Documentation is available in general (it makes grade 1 at once)
2 Every feature is described and has examples (if it fits it makes grade 2)
3 There’re additional well-grown resources (forums, blogs, user groups) where we can find additional information about the engine

At the end we have:

Engine Documentation availability
Cucumber 3
Freshen 2
JBehave 3
NBehave 1
SpecFlow 3
Behat 3

Other grades:

Flexibility in passing parameters:

Engine Regular expressions support Tables support Multi-line input support Extra features
Cucumber 3 3 3 0
Freshen 3 3 3 0
JBehave 2 3 0 3
NBehave 2 3 0 2
SpecFlow 2 3 3 2
Behat 3 3 3 0

Auto-Complete:

Engine Auto-complete support
Cucumber 1
Freshen 1
JBehave 1
NBehave 1
SpecFlow 3
Behat 1

Step Scenario and feature scoping:

Engine Tagging support Scoped steps support
Cucumber 3 0
Freshen 3 0
JBehave 3 1
NBehave 0 0
SpecFlow 3 3
Behat 3 0

Complex Steps:

Engine Composite steps
Cucumber 3
Freshen 3
JBehave 3
NBehave 2
SpecFlow 2
Behat 3

Hooks and pre-conditions:

Engine Backgrounds Hooks
Cucumber 3 3
Freshen 3 3
JBehave 1 1
NBehave 0 1
SpecFlow 3 3
Behat 3 3

Binding Code:

Engine Binding to code
Cucumber 2
Freshen 3
JBehave 3
NBehave 3
SpecFlow 3
Behat 2

Formatting Flexibility:

Engine Formatting
Cucumber 3
Freshen 3
JBehave 1
NBehave 3
SpecFlow 3
Behat 3

Reports:

Engine Built-in reports
Cucumber 3
Freshen 2
JBehave 2
NBehave 2
SpecFlow 2
Behat 3

Input data sources support:

Engine External Data Inclusions
Cucumber 0 0
Freshen 2 3
JBehave 3 2
NBehave 0 0
SpecFlow 0 0
Behat 0 0

Conclusion:

Table


Source
http://mkolisnyk.blogspot.co.uk/2012/06/gherkin-bdd-engines-comparison.html

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: