News & Events

Agile Leadership

Category: Agile Leadership

Pytest-BDD vs. Behave

Date of Release: October 11th, 2021
Reading Time: 02 mins

Leslie

Written by Leslie Brooks
linkedinicon

Love it or loath it, Python is the language of choice for most Data Scientists – so if the work is done in Python and you are writing the requirements in Gherkin, what tool should you use to automate testing?

There is no official version of Cucumber for Python, so the options are Pytest-BDD, Behave, and Lettuce. Let’s eliminate Lettuce right up front; here is Github’s graph of commits for the project:

Agile Leadership

Lettuce is unfortunately a dead project; if you find a bug there is no community that will fix it for you. The last release was in 2016, so unless someone revives it, this isn’t a viable choice.

Pytest-BDD and Behave are both active, so both are worth considering. What are the differences?

Pytest-BDD

Advantages

Pytest-BDD has the advantage of being integrated with Pytest; your developers should find that part of it easy to work with. It can use Pytest fixtures, mocks, etc.

Disadvantages

There is no support for data tables – this is a HUGE problem. Data tables are used in many, many scenarios, like this one for a library catalog search:

Scenario: The catalog can be searched by author’s name; both first and last names are searched.

Given these books in the catalog

When a name search is performed for Stephen, then only these books will be returned

The ‘Given’ in this scenario uses a data table to specify the contents of the catalog, and the ‘Then’ uses a data table to show which titles will be returned by the search. How could you possibly write this scenario without data tables?

Some people will substitute a Pytest fixture for the data tables, but this defeats the primary reason for using SBE. The primary reason is not to automate testing, it is to make it incredibly easy of the Product Owner, Business Analyst, Developer, and QA to communicate clearly. Hiding the examples in a fixture destroys the expressiveness of the scenario – what would the Product Owner say if you asked them whether this requirement is correct?

Scenario: The catalog can be searched by author’s name; both first and last names are searched.

Given the books from Fixture1 in the catalog

When a name search is performed for Stephen

Then only the books from Fixture2 will be returned

This requirement does not facilitate communication; it isn’t Specification by Example because the examples are hidden in the fixtures!

It is possible to work around this by modifying the regex in the step definition so that the regex includes the data table, but that is rather clumsy.

Behave

Advantages

Behave supports most of the Gherkin syntax; in particular, it does support data tables (even though it thinks that all data tables have a title row).

Disadvantages

Behave doesn’t have a good parser; for example, it will be confused by this scenario title:

Scenario: The catalog can be searched by author’s name; both first is searched first,

and last names are searched last.

1) The second line begins with ‘and last names…’; Behave will think this is a Gherkin ‘And’ and will complain that there was no ‘Given’.

2) Behave doesn’t raise an error if your scenario outline has a with no matching column title in the Examples table; your scenario simply fails.

3) Behave doesn’t support some of the newer features of Gherkin, like ‘Rule’.

Conclusions

Overall, Behave is the better tool because it supports data tables – in my experience, easily more than half of the (several thousand) requirements I have written used data tables.

If your developers are making extensive use of Pytest mocks, you may be better off using Pytest-BDD. If you take that approach, DO NOT fall into the trap of using fixtures in place of data tables; you will lose all of the value of Specification by Example. Remember what Aslak once wrote in a blog post: “If you think Cucumber is a testing tool, please read on, because you are wron

Trending Posts

Creating a Culture of Accountability within Your Agile Team

Accountability serves as the vital link connecting tasks to objectives and aligning individuals across an organization.
Last updated on 13th July 2023

 

The QA Team Should NEVER Find a Bug!

Many companies use metrics to measure the performance of the Sales Team, the Developers, the QA Team, etc.
Last updated on 14th May 2023

 

Do You Want to Save $50,000 per Developer per Year?

A number of case studies have shown that a typical developer spends 30% to 40% of their time fixing bugs.
Last updated on 11th October 2021

 

Follow Us

Creating a Culture of Accountability within your Agile Team
July 13th, 2023
Writer: Stephen Otterstrom

Creating a Culture of Accountability within your Agile Team

Accountability serves as the vital link connecting tasks to objectives and aligning individuals across an organization.

Agile Leadership
October 11th, 2021
Writer: Leslie Brooks

Pytest-BDD vs. Behave

Love it or loath it, Python is the language of choice for most Data Scientists – so if the work is done in Python and you are writing the requirements in Gherkin, what tool should you use to automate testing?

Data Science
November 8th, 2021
Writer: Leslie Brooks

SBE BDD for Data Science

Data Science and Big Data are hot topics, and rightly so – companies can save huge sums, or dramatically increase sales, by analyzing the data that they already collect. Specification by Example is a great partner in this effort.

Be a part of SevenSparx community to enjoy free resources in your inbox!

Right Menu IconAll Courses