Archive

Author Archive

Filling a ticket – Best practices

December 3, 2017 1 comment

Search for duplicates

Before reporting a bug it is necessary to make sure that the bug does not already exist.

Pick a good summary

When we’re looking through lists of lots of issues, the summaries are essential in helping us to understand what an issue is about. This means that you should err in favour of putting too much information in the summary, rather than too little.

Some examples:

Bad: Space export doesn’t work
Good: Space export fails with NullPointerException when performed by the anonymous user

Bad: Administrator attachment control
Good: Allow configuration of maximum attachment sizes per space or per page

Set the Priority, Severity and Impact

You need to select the priority, severity and impact of this bug.

Set the “Fix version”, release version

Set the fix version once you know the test has passed and this is going in the next release, unless the release is based on what needs to go and not in what is ready.

Choose appropriate components

You need to select all the components that are related to the ticket. So, if you need to make changes on different components, this is the field that you need to specify them. It will be useful for the QA engineer to know what are the components affected and all the components that need to be tested.

Attach images/logs/videos/files

Often, especially when describing bugs, an image/log/video/file will help tremendously to explain the issue. If you are testing on mobile platform, try to attach tablet logs, videos and images. For web an image or a video should be fine and for the server platform you can attach the response and of course the steps to reproduce.

If you need to include a large piece of information in an issue, such as a stack trace, please do not put it in the Description field. Instead, add it as an attachment or a comment. (This makes it a lot easier to view and parse issues in most browsers.) I suggest you to use Jing to capture the images and video.

Link issues

If you know that a feature request or a bug is similar (but not identical) to an existing one, please link it using a link type of “relates to”. This will help to get a broader view of the problem, and may be able to solve not only one but two issues.

Write a detailed description

For all issues, please provide a description that can be understood by all in the community, not just yourself or other developers. Also, the description should allow the QA engineer to understand the implementation details of the story, down to the code level. This informs their decisions on risks, what testing needs to be added, and what testing is unnecessary. If you need to include some techno-babble, in addition to the plain language, for other developer types to understand the full details that’s fine.

First and foremost, put yourself in the place of someone else trying to solve the issue based on information in the Jira ticket alone. That means put as much information as you can into the ticket so that the next person can work the issue without having to follow up. Even if the ticket is for an issue you plan to work yourself, the more information provided, the better.

Every ticket must include:

  • Detailed steps to test
  • Given/When/Then following BDD
  • Impacts, any integration with other components/features/functions. So, the tester will be able to think about edge cases
  • Write which environment this needs to be tested (QA is the default if not specified). For mobile platform, we test directly from each component’s master branch, unless written in the ticket that the app was deployed on qa/dev environment.

Every bug must include:

  • Detailed steps to reproduce
  • Impacts, any integration with other components/features/functions. So, the tester will be able to think about edge cases
  • What you expected to happen
  • What happened instead
  • Don’t forget to mention the environment you are testing. Make sure to include any information which might be relevant to the bug (eg. your screen resolution, browser, etc)

 

Don’t reopen issues that have been resolved and shipped

Don’t reopen resolved/closed issues once they have been shipped. If an issue similar to yours is marked as resolved, but you’re still experiencing a problem with it, create a new issue and provide a link to the original one. In the case of resolved bug-fix-issues, this will help to evaluate whether it is in fact the same problem, or actually a new issue with some of the same symptoms.

Testing a new story/feature

If you find a bug while testing a new feature:

  • Create a Story bug and link it to the main ticket, remember to prioritize this bug. Also, put back the story ticket to in progress again
  • Once the related Story bugs are fixed, the developer should move the story containing the bugs to Ready for QA, so any QA will be able to check if all bugs have been fixed and close the main story if there was not introduced new bugs.

Resources:

https://confluence.sakaiproject.org/display/MGT/Sakai+Jira+Guidelines#SakaiJiraGuidelines-Community
https://wiki.collectionspace.org/display/collectionspace/Guidelines+for+Filing+a+JIRA+Bug+Report
https://confluence.atlassian.com/jirakb/using-jira-for-test-case-management-136872198.html
http://www.3pillarglobal.com/insights/manage-test-cases-bugs-using-jira
http://blogs.atlassian.com/2013/12/jira-qa-process/
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=30746287
https://confluence.atlassian.com/display/DEV/JIRA+usage+guidelines

Advertisements

WebDriverIO framework first steps

November 26, 2017 1 comment

Hey guys, I have been studying a framework called WebdriverIO, it is a really good framework for Angular applications. It is compatible with BDD and TDD, so you can create your featuresimplement the step definition and it is also a synchronous command handling !!

Not only the hooks, but many useful parts are already in the config file, so you don’t need to create them separately, which is  always very handful.

I have pushed the project to github so you can have a look on my example, it is very simple for now.

https://github.com/rafaelaazevedo/webdriverio-study

 

– To run the project you need to run npm install and after ./node_modules/.bin/wdio wdio.conf.js

 

– In the login-spec.js you can add the scenario (describe) and the steps (it) directly like the step definitions, I don’t have the feature files for now.

– Look how you don’t need to add return for each step anymore. Since the steps are sync you don’t need to worry about it.

– The login-page.js groups all the common functions that will be used on the login scenarios.

– And the login-elements.js contains all the element getters that are on the login page, it works like a library for the elements and it will help you with the maintenance of the code.

If you want to give it a try as well you can add the webdriverio in the package of your project and follow these steps

Run Postman scripts on Jenkins with Docker

November 6, 2017 Leave a comment

You can run Postman scripts from the command line with Newman, but if you want to run these scripts as part of your Continuous Integration environment, you can run it with Docker or directly on Jenkins.  In case you prefer to use Docker, you can get started by downloading the Jenkins Docker instance and changing the Dockerfile to include node using the following node installation code found in the Docker/Jenkins Repository:

# Install Node

RUN curl -sL https://deb.nodesource.com/setup_4.x | bash

RUN apt-get -y install nodejs

RUN node -v

RUN npm -v

RUN npm install -g newman

Then, you will need to rebuild the Docker image and start the container with the same instructions as in the Jenkins Docker Repository.

You should now have a fully working Jenkins instance installed locally. Great! Now back to the task at hand using the newly-installed instance of Jenkins:

  • Create a new “Freestyle” job in Jenkins.

You will set it up to be able to upload the collection as a parameter. When you do this with your own projects, you should commit the Postman collection into whatever repository you’re using and pull directly from that repository to build by selecting “this project is parametrized” and then choosing “Add Parameter” with a “File Parameter.”

 

  • Select two file uploads – one for the collection and one for the environment.

  • Add a post-build step with “Execute Shell”. You’ll use the same command you used to run it from your own command line earlier (If you are using the same OS) except your path should be collection.js, as you named it ‘newman run collection.json’ in the File Parameter name field.

 

  • Now test it and run the build. I just uploaded the collection.json since I’m not using the environment file yet, but you can add it to the command line with:
    newman run collection.json –e environment.json

 

If you want to use the built in JUnit Jenkins viewer, you can archive the XML test result and point the tests to it. Below there is a sample of how you might archive and use the JUnit test results.

 

 

Resources: https://www.qasymphony.com/blog/automated-api-testing-tutorial/

API tests with newman and postman

October 21, 2017 Leave a comment

 

API is the most important part of your software because it has the highest security risks. What if someone were to hack your API? They could get production data, they could Bitcoin ransom the servers or they could hide on the machine until something interesting happens. The bottom line is, the stakes when using an API are much higher than if there is just a bug in the UI of your application — your data could be at risk and, by proxy, all of your users’ data.

API testing is not only the most vital testing to be done against your application, but it is also the easiest and quickest to execute

With your API tests and having a Continuous integration process, you can:

  • Test all of your endpoints no matter where they are hosted
  • Quickly ensure all of your services are running as expected
  • Confirm that all of your endpoints are secured from unauthorized AND unauthenticated users

 

Postman

Download postman for free

Now you need to have an API to test, you can select one from any-api or qa-symphony for this tutorial. In this example we are going to use this Oxford Dictionaries. Now on postman, you can create a collection and start to feed with your tests. Also, you can create an environment with environment variables and also global variables that can be used across the requests, for example a token.

 

Login with Token on Postman

  • Click on Manage Environments

 

  • Set the host

  • Type the request URL, change the host to the variable {{host}}
  • Select POST request and type the headers

 

  • Type the Body with the credentials for your site.

 

  • Open the Tests tab and write the tests to validate the request and set the access_token as an environment variable. You will need to use this variable in the header of all of your next requests.

var jsonData = JSON.parse(responseBody);

tests["Status code is 200"] = responseCode.code === 200;
tests["Access Token is not blank"] = jsonData.access_token !== "";
tests["Message success"] = jsonData.scope.indexOf("success") !== -1;

postman.setEnvironmentVariable ("access_token", jsonData.access_token);

 

  • Hit Send and if the status code is 200, it means the credentials are correct. Save the request in your collection.

 

  • Now you can create all the other requests and tests needed, just remember to add the {{access_token}} in your header and save the requests in your collection

 

 

Running your API automated tests

  • Now that you have a collection with all your tests, click on Runner and select your collection, environment. Type the number of interactions and delays in case you want to simulate more than multiple interactions.

  • You can also import a CSV file with data and substitute for variables in the body or header of the request.

 

  • The results report will be like this one:

 

Newman

 

To run your tests from the command line you need to have Newman, open your terminal and run npm install -g newman

  • Export your collection from Postman (Collection_v2) and download your environment (go to “Manage Environments” and click the download button) from Postman

 

  • Open your terminal an type the command to run the API tests: newman run /local/path_to_your_postman_collection.json

 

  • The result will be something like this:

 

  • To run the script on Jenkins you need to add the parameters  –reporters junit,json and the results should be created under a folder called “Newman” in your working directory, so: newman run -reporters junit,json /local/path_to_your_postman_collection.json

 

Resources: https://www.qasymphony.com/blog/automated-api-testing-tutorial/

https://api.qasymphony.com/#/login/postAccessToken

Usability Testing

October 12, 2017 Leave a comment

Hello guys,

Today I am going to write about some usability tests techniques that you can use in your day-to-day tests. Usability tests is where you can figure out if your design is working or not, what can be improved and what is not straight forward.

Jakob Nielsen created the 10 usability heuristics, which includes:

  • Visibility of system status
  • Match between system and the real world
  • User control and freedom
  • Consistency and standards
  • Error prevention
  • Recognition rather than recall
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
  • Help users recognize, diagnose, and recover from errors
  • Help and documentation

 

Know your user

When you know your users, you can focus on their goals, the characteristics that they have and the attitudes that they display. You should also examine what the user expects from the product.

User personas are created from other forms of user research and thus offer an real life portrait that is easy for the team to keep in mind when designing the products. User personas have a name and a story.

 

Simplify

We need to start with the idea of a user in your worst case scenario when doing UX testing, someone who knows nothing about your product, is distracted when they use the system, has bad cell reception, etc. By watching that person use and fumble through your product, you can quickly identify areas where the app is not simple, clear or fast enough.

 

Trust your intuition

It’s important to remember what specific problem you’re setting out to solve. Trust your guts: early on, it’s especially important when larger decisions are so fragile. Remember that you can’t build something that pleases everyone: trying to do so normally results in a weak release. Stay focused on the use-case you want to nail and avoid trying to solve all the use cases at once. The smallest details can be the difference between a product that has a good experience for the user and one that has not.

 

Efficiency

It lies in the time taken by the user to do a task. For instance, if you are an E-commerce site, the efficiency of your site depends on the time and the number of steps that the users take to complete the basic tasks like buying a product.

 

Recall

This is one of the most important aspects to examine the person’s memory concerning the browsing process and the interface they used a while ago. If your design is simple and straight forward it will be easier to remember how to complete a task.

 

Emotional response

This helps to analyze the user’s feelings after they have used the product. Some might feel happy while others might feel down and there are others who are so deeply impressed that they recommend the product to their friend.

 

Resources:

https://uxmastery.com/beginners-guide-to-usability-testing/

https://www.nngroup.com/articles/how-many-test-users/

https://thenextweb.com/dd/2013/08/10/13-ways-to-master-ux-testing-for-your-startup/#.tnw_3ngxAqEM

https://www.invisionapp.com/blog/ux-usability-research-testing/

https://www.interaction-design.org/literature/article/7-great-tried-and-tested-ux-research-techniques

https://uxdesign.cc/ux-tools-for-user-research-and-user-testing-a720131552e1

https://www.cso.com.au/article/626752/data-breach-notification-just-it-business/

Hiring QAs, Headless vs Real Browsers, Automated tests, Consumer contracting tests

September 30, 2017 Leave a comment

Hi guys, I went to the #18 Agile Roundabout meetup here in London and I found really interesting to share. The first video is a talk about some of the challenges that we have when hiring QAs and about automation on headless browsers vs real browsers. The second is about some of the challenges that we have when implementing automated tests in a company and the third one is about how to do consumer contracting tests with Pact.

I do recommend everyone to watch the videos since it was really interesting to hear these experiences and you might be facing one of these situations.

Common mistakes when you hire QAs

 

Automated Tests

 

Consumer Contract Testing

 

 

Thank you guys for this great meetup !

Selling automation to non-technical people

September 19, 2017 Leave a comment
%d bloggers like this: