Archive

Author Archive

How to measure exploratory tests ?

Hello guys,

Many people that are not from the QA area doesn’t know how to measure or what are the advantages of doing exploratory tests, but it is a technique really powerful when used correctly. Its effectiveness depends on several intangibles: the skill of the tester, their intuition, their experience, and their ability to follow hunches.

 

Value

  • detects subtle or complex bugs in a system (that are not detected in targeted testing)
  • provides user-oriented feedback to the team

Exploratory testing aims to find new and undiscovered problems. It contrasts with other more prescribed methods of testing, such as test automation, which aims to show scripted tests can complete successfully without defects. It will help you write new automated tests to ensure that problems aren’t repeated.

If you have any doubs about Exploratory tests, like examples and what are the advantages of doing it, have a look on the video below first:

99 Second Introduction to Exploratory Testing | MoT

 

When should you perform exploratory tests

Exploratory testing works best on a system that has enough functionality for you to interact with it in a meaningful way. This could be before you release your first minimum viable product in beta or before you release a major new feature in your service.

How to measure

Always test in sessions:
  1. Charter
  2. Time Box
  3. Debriefing
  4. Mind Maps

 

Charter

  • Mission for the session
  • What should be tested, how it should be tested, and what problems to look for
  • It is not meant to be a detailed plan
  • Specific charters provide better focus, but take more effort to design: “Test clip art insertion. Focus on stress and flow
  • techniques, and make sure to insert into a variety of documents. We’re concerned about resource leaks or anything else that might degrade performance over time.”

99 Second Introduction to Charters | MoT

 

Time Box

  • Focused test effort of fixed duration
  • Brief enough for accurate reporting
  • Brief enough to allow flexible scheduling
  • Brief enough to allow course correction
  • Long enough to get solid testing done
  • Long enough for efficient debriefings
  • Beware of overly precise timing
Sessions time:
  • Short: 60 minutes (+-15)
  • Normal: 90 minutes (+-15)
  • Long: 120 minutes (+-15)

Debriefing

  • Measurement begins with observation
  • Session metrics are checked
  • Charter may be adjusted
  • Session may be extended
  • New sessions may be chartered
  • Coaching happens

 

Mind maps

Mind maps can be useful to document exploratory testing in a diagram, instead of writing the scenarios. It is a visual thinking tool and are quick and easy to record as they don’t follow a linear approach.

 

Session metrics

The session metrics are the primary means to express the status of the exploratory test process. They contain the following elements:

  • Number of sessions completed
  • Number of problems found
  • Function areas covered
  • Percentage of session time spent setting up for testing
  • Percentage of session time spent testing
  • Percentage of session time spent investigating problems

 

Coverage

  • Coverage areas can include anything
  • Areas of the product
  • Test configuration
  • Test strategies
  • System configuration parameters
  • Use the debriefings to check the validity of the specified coverage areas

 

Reporting

  • Create a charter
  • Features you’ve tested
  • Notes on how you conducted the testing
  • Notes on any bugs you found
  • A list of issues (questions and concerns about the product or project that arose during testing)
  • Extra materials you used to support testing
  • How much time you spent creating and executing tests
  • How much time you were investigating and reporting bugs
  • How much time you were setting up the session

 

Tools

I like to use Katalon or Jing, but to be honest this is just to record and take screenshots of the test sessions. To do these kind of tests you just need a paper and a pen to write your notes, concerns and questions.

 

Resources:

http://www.satisfice.com/sbtm/

http://www.satisfice.com/presentations/htmaht.pdf

https://www.gov.uk/service-manual/technology/exploratory-testing

Advertisements

How to Develop a Growth Mindset in Tech

June 22, 2018 4 comments

I should have posted this ages ago when I went to the Growth Mindset workshop, but I was on holidays so I had to postpone it a bit. Growth mindset is how you approach failures and challenges in your life. I am going to focus on how to have this mindset in your workplace.

 

 

Have you ever found yourself in your comfort zone ?

 

If you are in your comfort zone you will be for some time until something happens.

Most of people love to be in the comfort zone, it is a safe place right ? But it is not in this zone that you have the chance to improve and be the best version of yourself. I started in the QA area as a manual tester as most of the QAs start and if I didn’t challenge myself I would still be there, doing the same job, unhappy but safe.

When you don’t push yourself to get out of this zone, you will need to wait until you are forced to get out. In these moments you will find yourself defeated because something happened out of your control. Sometimes if you have been in this place for many years you won’t even know where to start. I saw people working +10 years in the same company and when there was a cut they couldn’t find a job after more than 2 years and why is that ? I get that you are comfortable where you are, you have a good salary, quality of life, but and if you are outdated with the current market ? Do you think is really a good idea be completely dependent of a job ? Not sure if it is a good idea, for this reason I prefer not to wait for these moments and create myself new challenges.

 

How do you react a feedback ?

 

Embrace the challenges and be persistent, maybe will take more time than what you expected, but when you reach your objective you will feel fulfilled. There are some feedbacks that depending from who is coming you can ignore, because they are not there to add anything, these are called destructive feedbacks.

These feedbacks could be given by managers who don’t really know your work, they are not working with you daily or they don’t communicate with the team. This makes really hard for a person to give a realistic feedback, so in this case you can completely ignore because clearly this person has no clue of what you do.

They will give you a superficial feedback like you need to communicate more or have lunch with the team more often. Remember that social gathering is optional and you should do only if you feel comfortable enough. Don’t push yourself to go to places with people that you don’t trust or don’t add anything just for networking. It is better spending your time creating a quality network in another place instead.

This problem usually happens when the manager doesn’t know how to build bridges across the team, otherwise socializing in the workplace wouldn’t be a problem in the first place. During these 10 years of experience, I’ve had only 2 good managers which who I still have contact and they became my friends outside work.

The constructive feedbacks should be embraced, they come from these kind of managers. They really care about your development and your growth. They do regular catch ups (doesn’t need to be formal or even announced) with the team and they know how to build a trustworthy relationship.

Keep your mind open for these feedbacks, don’t take it personal and don’t be upset about it. I know we have a constant fight with our ego, but these feedbacks are the steps for your own development. Be thankful for having managers like this that can see yourself under the superficial impression and can give you a good feedback. My suggestion is to keep these kind of people always around, surround yourself with good professionals and be inspired by them.

 

Vision and values: Does the company has the same values as you ? Do you agree with the vision of the company ? Does the company has a vision ? Do you feel included in the company’s vision ?

Money: Do you know your value in the market ? If you don’t, are you doing some interviews to figure out ? Do you have goals where you need to sacrifice your work/life balance for an amount of time ? Is your company valuing you enough increasing your salary ?

Experience: Look for companies that look for generalist developers this means that the company will always follow the new technologies and you won’t be outdated.

Managers & Exec team: Do you trust in your manager ? Do they have your back and fight for the team ? Does he care about your happiness in the work ? Does he take your opinions into account or just ignore them ?

Peers & team: Are you respected in your team ? Do you trust in your team ? Do you have a good sense of collaboration across everybody ?

Work/Life & respect: Do you have a flexible work that understand your needs ? What is the police to work from home ? Do you need to feel the stress of commuting ?

Growth and empowerment: Can you learn from the people that you work with ? Are you growing ? Are you stuck doing the same thing over and over ?

 

How emotionally intelligent are you ?

 

 

Constant Learning

 

To summarize basically what was being written here, push yourself and accept the challenges. You won’t be successful on all of them, but be resilient and continue improving. Take advantage of learning with the previous mistakes. Sometimes you will need to change your strategy or even your goal, but what matters is the quality of the journey !

 

Thanks Joanna Chwastowska for sharing this workshop !

Using Artificial Intelligence to Speed up your Test Automation

Hey guys, today I am going to share with you this awesome webinar that I watched last weekend ! I suggest to follow IIST as they are always sharing good webinars about test.

 

What is the cost of a bug?

If you have wondered how much a bug cost and how to measure this, today I am going to show some research about this. You must be asking, but why invest in testing if you can just fix your mistake after? What’s the true cost of a software bug? The cost depends on when the bug, or defect, is found during the SDLC (Software Development LifeCycle.)

In 2016, the software failures cost the worldwide economy $1.1 trillion. These failures were found at 363 companies, affected 4.4 billion customers, and caused more than 315 years of time lost. Most of these incidents were avoidable, but the software was simply pushed to production without proper tests.

 

 

It’s time to pay attention to how much software errors cost to your company, and start taking steps to recoup those costs.

To illustrate: if a bug is found in the requirements-gathering phase, the cost could be $100. If the product owner doesn’t find that bug until the QA testing phase, then the cost could be $1500. If it’s not found until production, the cost could be $10,000. And if the bug is never found, it could be secretly costing the company money. A 2003 study commissioned by the Department of Commerce’s National Institute of Standards and Technology found that software bugs cost the US economy $59.5 billion annually.

The cost of a bug goes up based on how far down the SDLC (Software Development Life Cycle) the bug is found

Then there’s the domino effect to think about. The software development approach often need to change to accommodate the code fix, which can in turn bump back other code changes. So not only is the bug going to cost more to fix as it moves through a second round of SDLC, but a different code change could be delayed, which adds cost to that code change as well.

 

Test early, test often. Prevention is better than the cure

To ensure that bugs are fixed at an earlier stage, take advantage of the following security testing practices:

  • Get together the team to help identifying issues during the design phase of software development.
  • Implement code review stage
  • Create an automated regression or smoke tests pack and run them often

 

In development, you often have less data, use one browser and use the software exactly as intended. Plus you probably already have a debugger on the machine. The major problem with bugs in production is in the absence of supporting tools.

 

Better testing

The truth is when automated testing is still under-development you still need to do manual testing. A poor testing methodology costs a lot of money in wasted time and can result in a reducing the scale of return.

For instance, if your testing process is too thorough, your developers will be producing new features faster than you can test them, and you end up with a backlog of features waiting to be deployed to production. The same could happen if you don’t have the correct scale between how many people you have in your QA and in your DEV team.

Always add unit tests when software errors are found. It is a bit more painful when handling legacy codes without a good coverage of unit tests.

 

Prioritise your bug fixes

Prioritise software errors measuring the impact that they have on the end users. This way will allow you to allocate your resources accordingly, saving valuable costs throughout the SDLC and reduce the cost of software errors.

Software errors expose your end users to slow, buggy software, compromise the security and safety of your products. Many businesses don’t have visibility on their software errors, so measuring them and their impact can be hard.

Software errors were responsible for a majority of the software failures in Government, entertainment, finance, retail, services and transportation in 2016 thanks to research conducted by tricentis.com:

 

What’s worse is software errors have multiple consequences varying on impact, so you can’t always pinpoint the cause and effect. The effects trickle down ultimately to:

  • Developer costs spent on finding and fixing errors
  • Ongoing lost revenue from unsatisfied customers

Using a few industry averages, we can help you calculate the cost in lost development time for your company.

 

How to calculate the cost of developer labour caused by software errors

Taking the industry averages from 2018, we can estimate the financial costs to your company and investigate where the money is going.

  • The median developer wage for the UK April. 2018 £30,651

The errors in your applications need to be fixed or they will affect end users and cause lost revenue. This is where support costs start to mount. Fixing software errors is low cost, reactive work.

You should be aiming for around 20% reactive work (finding and fixing errors, support costs), 80% proactive work (building features and improving product) rather than vice versa. This is where you’ll add the true value to the business and your users according to Raygun.

Based on a 40 hour work week at the average wage of GBP £30,651, the average software developer could spend 8 hours each week (32 hours each month). Costing around £6,130.20 per year, fixing errors and replicating issues.

This is time spent away not building new and important features for your customers. So, think twice when building your QA team and not bringing key people to earlier stages of a new implementation. Share as early as possible, gather opinions and different perspectives of the platform.

 

Resources:

https://www.payscale.com/

http://blog.celerity.com/the-true-cost-of-a-software-bug

https://crossbrowsertesting.com/blog/development/software-bug-cost/

https://raygun.com/blog/cost-of-software-errors/

https://www.synopsys.com/blogs/software-security/cost-to-fix-bugs-during-each-sdlc-phase/

Building a sudoku game with Kotlin

Hi guys,

Today I will continue the series of posts related to the workshop I attended during the #TechKnowday here in London. This one is to learn Kotlin building a sudoku game. So, basically Ana left available on her Github the initial parts and the solutions.

The instructions with the steps are available here. Below you can find the exercise done in parts:

You can find the first part here: https://github.com/anikiki/sudoku-init-part1

The second part:  https://github.com/anikiki/sudoku-init-part2

And the third part:  https://github.com/anikiki/sudoku-init-part3

 

If you are wondering what you have done wrong, you can check the solution for each part below. Don’t cheat, this is for after you have tried the exercise by yourself first:

https://github.com/anikiki/sudoku-solution-part1

https://github.com/anikiki/sudoku-solution-part2

https://github.com/anikiki/sudoku-solution-part3

 

Her contacts:

 

Thanks Ana for this workshop !

 

Machine Learning in IOS apps

Hello guys,

I have been to the #TechKnowday here in London three weeks ago and one of the workshops that I joined was the Machine Learning in IOS Apps. So, in the end we had a face recognition app which showed what was the emotion of the person, if the picture was someone smiling than it should show the smile emoticon and so on.

You can find the slides and follow the explanations here:

https://github.com/costescv/MachineLearning/blob/master/MachineLearning.pdf

Then you will need to clone the repository with the project https://github.com/costescv/machinelearning and download the Sentiment Polarity model here

Step 1:

In the ViewController.swift you will need to create the face detection request, the request handler and the face detection action. So you will have something like this:

 

Step 2:

In the SentimentAnalysisService.swift you will need to create the model using the SentimentPolarity, pass the linguistic tagger options and create the input to receive and interpret the input with the sentiment. You can add, remove or change the sentiments in the Sentiment.swift class, but don’t forget to change in this class the sentiment as well.

 

 

So, after you build, run the app and type the name of the sentiment with a space in the end, so you should have something like this:

 

Thanks Vasilica for this workshop !

Leading by example

Hello guys,

Today I am going to post something interesting that I learned through time with my work experience. This time I’m not talking about technical skills, I’m talking about the soft skills and how you can be a better person every day by learning from the constructive critics and ditching the destructive ones that don’t give you any benefit.

I am also sharing what some successful companies are already doing in the Silicon Valley which Jacqueline Yumi Asano explained in her article (it is in Portuguese, but you can check it bellow in the Resources part).

 

Inspire people

 

Say sorry, you are not a machine and you are going to fail at some point

If you want to be a trustworthy person at your workplace, you need to show that you are fair and you recognise when you fail. Saying sorry will show that you are humble enough to have the trust of anyone.

 

Strength the individual

Nothing is more empowering than supporting people to learn something and improve themselves. Everybody wins when this happens. Ask yourself if 20% of your time is about learning or you just do the same task over and over again. Do you have challenges in your company ? Are you growing in this company ?

You might be asking how can you identify if a company will improve you or not before even starting there? Check how they write their specs and roles. Are they asking for specific technologies or are they looking for generalist professionals ? A good company will look for someone that is a machine learning person and not someone that has a fixed technology in their skillset. That means the company will follow the latest technologies and you will be learning most of the time.

We don’t’ assign the work for our developers. They assign up for work
LiveRamp

 

Don’t under estimate the power of the vision and direction

Do you have a clear version of your goal ? Does your company is taking you to this goal ? This is extremely important for your future and you don’t have time to lose. So, double check if your company helps you to achieve this goal and if you have a clear vision and direction to follow.

If not, there is not too much you can do apart from start looking for a place where you can clearly see it leading you to your goals in, let’s say, a years time.

 

Get away from tyrant bosses, look for a strong bond of trust on both sides

If you ever worked in a place where your manager use his lead position to decide something rather than arguments, what are you even doing in a place like this? It is definitely a toxic environment and with this lack of trust it is better to look for something new because discussions and arguments would be pointless. Your opinion will mean nothing, even if you have arguments, analysis and your own carrier experience. Your manager will always ignore what you are saying.

Free choices matter and you should work in a place where your opinion is not invalidated.


Our product is great because we show why we are recommending
an specific insight. This way we build trust.
François Lopitaux, Director of Product, Salesforce

 

Surround yourself with passionate, but not blind people

Passionate is different from blindness. I just want to highlight that I have found many people that come across as passionate people but in reality they were completely blind and never accepted to have failures on their ideas/projects/implementation. This is quite common among developers, for that reason there is this assumption that a QA will always be a developer enemy, since it is part of our job to show bugs on their implementations.

A smart person knows how beneficial is to have a constructive critic and feel glad to have them. So, surround yourself with passionate and smart people. Be smart, take responsibility, accept the failures and improve them.

 

Get others opinions to empower people

I can only see advantages on doing that. Share knowledge as soon as possible with everybody in your team. Is this a new feature ? So, get everybody together and expose your idea, get others opinions. Don’t ignore them, every idea has a value. You can find bugs in the early stages and also this builds an ownership feeling.

Does the company take your opinion into account ? Do you feel that you have autonomy ? There is nothing more empowering than showing that you care about everybody opinions. Everybody feels valued and this will increase the trust on your work since they know that they can be honest with you.

We don’t need as PMs to tell UX Designer what they need to look at
Pinterest

 

Resources:

https://medium.com/mulheres-de-produto/o-que-eu-aprendi-no-vale-do-sil%C3%ADcio-sobre-product-innovation-7f3128f33e3

http://qablog.practitest.com/leading-by-example/

http://www.soulcraft.co/essays/lead_by_example.html

%d bloggers like this: