UI tests with Selenium - Testing on Multiple Browsers using Selenium Grid

Door: Harm Pauw & Robert van Vark,  04-03-2015
Onderwerp: Continuous delivery  CD  CI  Tooling  Testing  

This blog post is the second one in the series of “UI tests with Selenium and more”. This post will be about Selenium Grid, how it’s often used to test with multiple browsers and the challenges you face with supporting infrastructure.

In the previous blog we discussed Selenium WebDriver, why people often use it in combination with HtmlUnit instead of real browsers, and why it is not that smart to use HtmlUnit only. In case you’ve forgotten: none of your real customers actually uses HtmlUnit, as it doesn’t have a GUI. Also, it is not possible to take screenshots since there’s no rendered page to capture.

The main reason for teams to use HtmlUnit is that it can run easily on your headless build server. Also, it would be very time-consuming to manually create individual suites of automated tests for every OS / browser combination that your customers use. Fortunately, Selenium provides a framework of doing exactly that: Selenium Grid. With Selenium Grid, you can set up a grid of multiple browsers and OS combinations that you can use in your tests. What makes it better is that your tests can be completely browser and OS agnostic. This enables you to use the same test suite to test multiple browser and OS combinations.

Selenium Grid’s architecture consists of several Nodes and one central Hub. A Node can provide one or more browsers, e.g. Chrome and Internet Explorer, on a specific operating system. The Grid is controlled from a central place called the Hub.

Running your automated tests only requires that you create a RemoteWebdriver object, provide it with your wishes regarding the browser and OS and point it to the URL of the Hub. The Hub takes care of distributing your tests to a node that satisfies the browser/OS combination you requested. You can also specify the number of concurrent browser instances on each node, so that you can run your tests in parallel. There’s also a web based Console that - although it’s a little bit primitive - allows you to see the status of each node.

If you only want to test using one particular browser, it’s not necessary to have a separate Hub and Node. In this case, you can start a Standalone server which acts like both a Hub and Node. You can use it in exactly the same way as a grid with a hub and multiple nodes.

You’ll need some infrastructure to create a grid: either in the form of physical servers or virtual machines running on a server on-premise or in the cloud. The setup and maintenance of this grid costs effort and money, even if it’s only used for short periods. For this reason, companies often let multiple applications use the same grid: that way the grid is used more efficiently.

However, sharing a grid with multiple applications also has some downsides. Often the usage of a grid is not evenly distributed so during “rush hours” you can have a queue of tests that wait until they can be executed. This could lead to long execution times and false positives due to timeouts. Also, if the tests of one application are taking a long time to run or even hang (caused by timeouts or other problems), they could seriously affect tests from other applications.

An ideal solution for these problems is to have a completely new and fresh grid every time you start a new test run. This way, test runs for different applications don’t interfere with each other and when a test run goes bad, subsequent test runs are not affected by it. But booting physical and even virtual machines takes up quite some time, so this is not very practical.

In the next blog post we’ll take a look at Docker and see how we can apply this tool to provision on-demand Selenium Grids while avoiding long startup times and other problems.

The Team Consultant

Barry Overeem Door: Barry Overeem,  01-03-2015
Onderwerp: Scrum  Agile  

Currently I'm reading 'The Scrum Field Guide' written by Mitch Lacey. So far it's an excellent book that offers me some interesting, practical insights. One of them is the concept of the team consultant. This idea suits a matter I've encountered quite often. In this blog post I'll share some of the thoughts of Mitch Lacey on this topic and complement it with my own personal view and experiences.

The best team sizes are between five and nine people, all of whom are fully dedicated to a project for the duration of the project, and who work together in a cross-functional way to deliver working software at the end of every sprint.

Many organizations struggle to create teams that live up to these conditions, for example:

  • Some specialized knowledge might be thus scarcely available and highly wanted, that the people having this knowledge can't be allocated to just one team;
  • Some people have such highly specialized skills that a Scrum team can't offer them enough relevant and suitable tasks;
  • Some people don't have the personality to be the ideal Scrum team member. Ilan Goldstein wrote an excellent article in which he makes the distinction between 'rock stars' and 'studio musicians'.

So how can you compose a team that is dedicated to building the desired product, respects the fundamentals of teamwork but also takes the given realities as described above into account?

Technical Agile Coach

Marco Kroonwijk Door: Marco Kroonwijk ,  23-02-2015
Onderwerp: Technical Coach  Continuous Delivery  CD  Extreme Programming  XP  Agile Coach  Technical Debt  Architecture  Development Team  Role  Craftsmanship  Practices  
A while ago, I started working at Prowareness, and as new employees we choose our own role title best fitting our experience, skills, ambition and passion. I chose the title of Technical Agile Coach, without really having defined what it actually is. It just felt right!

However, occasionally, people ask me what a technical coach does and what the difference is with “other” Agile Coach titles. So here’s what that answer is, for me.

The User Story Mapping Game

Barry Overeem Door: Barry Overeem,  23-02-2015

Being an Agile Coach & Trainer for Prowareness, I use different types of games, tools and practices every week during meetings, workshops and trainings. Some of these I've invented myself, but most already existed and have only been changed slightly to a format that suits me best. The upcoming period I want to share some of these games, tools and practices. I will share the what-why-how-when and what worked well and what didn't. By sharing my experiences I hope to inspire you to give these tools and practices a try for yourself, improve it and hopefully share the perfected version in return.

What is user story mapping about?
User story mapping is a technique that allows you to add a second dimension to your backlog. The visualization enables you to see the big picture of the Product Backlog. It gives you a good opportunity for making decisions about refining and ordering the backlog. Or as Jeff Patton, the author of the book "User Story Mapping", puts it "a prioritized user story backlog helps to understand what to do next, but is a difficult tool for understanding what your whole system is intended to do. A user story map arranges user stories into a useful model to help understand the functionality of the system, identify holes and omissions in your backlog, and effectively plan holistic releases that deliver value to users and business with each release."

What is the user story mapping game about?
It is an exercise that gives you the opportunity to understand the strength of story mapping. By using an everyday real life example, a product backlog is created, visualized and prioritized in less than 30 minutes.

UI tests with Selenium - About Selenium

Door: Robert van Vark & Harm Pauw,  16-02-2015
Onderwerp: Testing  Tooling  Continuous delivery  CD  

This blog post is the first one in the series of “UI tests with Selenium”. This post will be about Selenium, how it’s often used and the problems that could arise when using it an a CI setup.

Selenium is one of the most used frameworks to create tests that interact with the browser. There are two flavors: Selenium IDE and Selenium WebDriver. The former is is not well suited for creating and maintaining test suites due to it’s record-and-playback setup: recorded tests are difficult to maintain and become outdated quickly. Selenium Webdriver provides a programmer with an API to interact with the browser in a uniform way. In combination with a test framework like jUnit, TestNG or Cucumber, a programmer can create UI tests in similar way as creating unit tests. Additionally, Selenium has features to record screenshots during the tests, which facilitates analyzing failures of your test runs.

A Projects Nightmare

Barry Overeem Door: Barry Overeem,  16-02-2015
Onderwerp: Scrum  Agile  

When developing a product, the given constraints and boundaries aren't always easy to handle. But as long as they are realistic and used wisely it shouldn't interfere in building a great product. The most common constraints are set on scope, budget, schedule or quality.

In traditional plan driven projects, the features are described and estimated up front. The time and cost necessary to deliver these features should be clear at the beginning of the project. After the requirement analyses phase the scope, schedule and budget get frozen and the phases of design, implementation and testing start. Everything with the big-bang deadline in mind at which everything must come together magically.

Scrum projects are value driven. Given the complexity and unpredictability of software development an up front fixed scope isn't possible. Therefore the budget and schedule are often fixed, but the scope isn't. Because Scrum always focuses on the features with the highest value, the features that aren't done at the end date are the ones with the lowest priority.

Another variety is one that can occur in both traditional and Scrum projects but for sure is a recipe for disaster. And I know, it may sound like a Product Owners dream, but in my experience it's a nightmare for a project:

A customer with an apparently unlimited budget.

5 urgent reasons to embrace Business Agility

Peter Koning Door: Peter Koning,  14-02-2015
Onderwerp: agile  Agile Principles  Agility in de bedrijfsvoering  

Searching for easy, flexible ways to unlock growth? Tired of endless meetings and e-mails all about internal process? Is your competitor more flexible, innovative and just one step ahead of you? Business Agility is a lightweight mindset that will empower synergy between marketing, sales and support employees. Unlike more process, managers, checklists and meetings; Business Agility is about small, empowered, cross-functional teams with one very smart goal, called Squads. These Squads have a lot of freedom and ownership to increase the customer experience and their success. They are on a mission to make their customers continuously more successful by continuously improving their products and services. After explaining what it is, I’ll give you 5 urgent reasons to start with Business Agility.

Scrum Webshop