Archive

Archive for May, 2015

Getting Started with Mobile Test Automation & Appium

May 27, 2015 1 comment

Hi guys, today I have another Webinar that I participate about Appium, how to choose the automation mobile tool, how should be the structure, etc. You can watch online or download, just click in this link to start.

You can jump the advertisement moving to 00:10:51 of the video. Also, you can jump to 00:23:12 and watch the demo. To jump the demo go to 00:40:00.

 

Difference between hybrid native and html

 

Screen Shot 2015-05-27 at 20.39.45

 

 

HTML: web applications which runs in web browser.

Native: just native elements, apps which doesn’t have html elements.

Hybrid: mix of native elements and html elements.

 

Mobile QA Solutions

 

Screen Shot 2015-05-27 at 20.44.35

– How choose the automation tool ?

– Verify how you inspect the elements.

– Scalability, How many devices supports (IOS, windows phone, tablet, kindle, ipad, etc).

Integration with other tools like Jenkins, JIRA, etc.

Cloud, If this tool supports cloud tests (Amazon cloud, Sauce Labs, etc).

Automation Support, How long do you take to find the solution for some problem ? (Particularly, this is the most important for me )

 

Appium

Open source automation test automation framework for native, hybrid and mobile apps.


Supports:
Real devices, Simulators, Native Apps, Hybrid Apps, Mobile Web Apps (IOS and Android only).

Environment: Appium, Android Studio (Sdks) and Eclipse.

Screen Shot 2015-05-27 at 20.47.56

Setting parameters: App device, launch device, android setting (path of sdks).


Screen Shot 2015-05-27 at 20.49.03

Architecture, Blueprint: 

Screen Shot 2015-05-27 at 20.46.16

Screen Shot 2015-05-27 at 20.50.50

Make sure that you are doing right

 

Screen Shot 2015-05-27 at 21.14.19

– Do some proof of concepts (The configuration it is the hard and most boring part for me).

– Share the right quantity of tests between emulators and devices.

– Distribute the right level of tests between unit (70%), integration (20%) and manual tests (10%).

– Remember that if you are using BDD, it is just 3 layers.

– Use CI since the beginning.

– Decide what must to have in the report.

– Considerate the language which the developers are using (You never know when you will need help).

Screen Shot 2015-05-27 at 21.18.30

Thank you guys ! See you next week 🙂

Source: link

Code example to catch url, Status code and time to load an url

Hi guys, today I will post a simple project that I did just to catch some information with Selenium and Java. So, I have a spreadsheet with some urls in the first column and when I run the project, it is being recorded the status code of the urls and how long is lasting to load this page in the following columns.

It is very simples. I know that there are many ways to do that and you don’t need to use selenium because will take more time to render the page and everything, but maybe someone is needing something similar. So, now the code and in the bottom the link to my project.

  • OPEN URL – Open the URL and write the StatusCode and the time to load the page in the following columns:
import jxl.read.biff.BiffException;
import org.openqa.selenium.WebDriver;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.FileWriter;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.Workbook;
import org.openqa.selenium.By;

public class OpenUrl {

    OpenDataBase opendatabase = new OpenDataBase();
    FileOutputStream fOut = null;

    public void OpenURL(WebDriver driver) throws BiffException, Exception {
        String tablename = ("tb");
        opendatabase.Open(tablename);
        File file = new File("BD.xls");
        int result = 0;
        String STR = "CRI -";
        int ARQ = 1;
        int WARNING = 2;
        int CRITICAL = 3;

        Workbook wb = new HSSFWorkbook(new FileInputStream(file.
getCanonicalPath()));

        for (int i = 0; i < opendatabase.sheet.getRows(); i++) {

            String url = opendatabase.sheet.getCell(0, i).getContents();

            double start = System.currentTimeMillis();
            driver.manage().window().maximize();
            driver.get(url);
            double finish = System.currentTimeMillis();
            double totalTime = finish - start;
            double totalTimeSec = totalTime / 1000;

            try {
                driver.findElement(By.xpath("/html/body/div/span[2]/ul/li/a")
).click();
            } catch (Exception e) {
                driver.findElement(By.xpath("/html/body/div/span/ul/li/a")
).click();
            }

            String titlepage = driver.getTitle().replace(" ", "");

            FileWriter fw = new FileWriter(titlepage + ".txt");
            int statusCode = Statuscode(driver.getCurrentUrl(), driver);
            String line = url + "," + totalTimeSec + "," + statusCode;

            fw.write(line);
            fw.close();


        }


    }

    public int Statuscode(String URLName, WebDriver driver) {
        try {
            int statuscode = Integer.parseInt(driver.findElement(By.xpath
("/html/body/div/span/span/pre[2]")).getText().substring(9, 12));
            return statuscode;
        } catch (Exception e) {
            e.printStackTrace();
            return 1;
        }
    }
}
  • OPEN DATABASE CODE – Read the columns in the Excel Spreadsheet:
import java.io.File;
import java.io.IOException;
import jxl.Sheet;
import jxl.Workbook;
import jxl.WorkbookSettings;
import jxl.read.biff.BiffException;

public class OpenDataBase{

    Sheet sheet;
    Workbook bdurl;
      
    public Sheet Open(String tablename) throws BiffException, IOException, 
Exception { 

        File file = new File("BD.xls");
            WorkbookSettings ws = new WorkbookSettings();
            ws.setEncoding("Cp1252");
            
            bdurl = Workbook.getWorkbook(new File(file.getCanonicalPath()), 
ws);             
            
            sheet = bdurl.getSheet(tablename);
            return sheet;
        }

    public void Close(Workbook bdurl) throws BiffException, IOException, 
Exception {

        bdurl.close();
    }
}

 

  • MAIN CLASS – statusCode:
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.firefox.internal.ProfilesIni;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

public class statusCode {

    public static void main(String[] args) throws Exception {
        ProfilesIni profilesIni = new ProfilesIni();
        FirefoxProfile profile = profilesIni.getProfile("default");
        WebDriver driver = new FirefoxDriver(profile);

        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
        
        profile.setPreference("browser.ssl_override_behavior", 1);
        profile.setPreference("network.proxy.type", 0);
        profile.setAcceptUntrustedCertificates(true);
        profile.setAssumeUntrustedCertificateIssuer(false);

        driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);

        OpenUrl Openurl = new OpenUrl();
        Openurl.OpenURL(driver);
        
        driver.close();
        driver.quit();
    }
}

Thanks guys ! See you next week, if you have any questions, suggestion please feel free to comment below.

Sourcehttps://github.com/rafaelaazevedo/statuscode_nfe

When should I do End-to-End Tests ?

About End-to-End tests:

“Focus on the user and all else will follow”

  • Developers like it because it offloads most, if not all, of the testing to others.
  • Managers and decision-makers like it because tests that simulate real user scenarios can help them easily determine how a failing test would impact the user.
  • Testers like it because they often worry about missing a bug or writing a test that does not verify real-world behaviour; writing tests from the user’s perspective often avoids both problems and gives the tester a greater sense of accomplishment.

 

Building the Right Feedback Loop

 

  • It’s fast. No developer wants to wait hours or days to find out if their change works.
  • It’s reliable. No developer wants to spend hours debugging a test, only to find out it was a flaky test.
  • It isolates failures. To fix a bug, developers need to find the specific lines of code causing the bug.

 

Unit Tests

 

Unit tests take a small piece of the product and test that piece in isolation. They tend to create that ideal feedback loop:

  • Unit tests are fast. We only need to build a small unit to test it, and the tests also tend to be rather small.
  • Unit tests are reliable. Simple systems and small units in general tend to suffer much less from flakiness. Furthermore, best practices for unit testing – in particular practices related to hermetic tests – will remove flakiness entirely.
  • Unit tests isolate failures. Even if a product contains millions of lines of code, if a unit test fails, you only need to search that small unit under test to find the bug.

 

Unit Tests vs. End-to-End Tests

 

With end-to-end tests, you have to wait: first for the entire product to be built, then for it to be deployed, and finally for all end-to-end tests to run.
image

 

Although end-to-end tests do a better job of simulating real user scenarios, this advantage quickly becomes outweighed by all the disadvantages of the end-to-end feedback loop.

 

 Integration Tests

 

Unit tests do have one major disadvantage: even if the units work well in isolation, you do not know if they work well together. For that, you can use an integration test (more simple than end-to-end tests). An integration test takes a small group of units, often two units, and tests their behaviour as a whole, verifying that they coherently work together.

 

piramid

Even with both unit tests and integration tests, you still need do a small number of end-to-end tests to verify the system as a whole. End to end tests are very important even that you let this for the last phase.

 

What is the right balance ?

 

To find the right balance between all three test types, the best visual aid to use is the testing pyramid. Here is a simplified version of the testing pyramid from the opening keynote of the 2014 Google Test Automation Conference.

 

The bulk of your tests are unit tests at the bottom of the pyramid. As you move up the pyramid, your tests gets larger, but at the same time the number of tests (the width of your pyramid) gets smaller.

Google often suggests a 70/20/10 split: 70% unit tests, 20% integration tests, and 10% end-to-end tests. The exact mix will be different for each team, but in general, it should retain that pyramid shape.

 

How can I choose what will be the tests for each phase ?

 

  • End to end tests: Choose the flows/functions most used for the client.
  • Integration tests: Focus in the integration between the functions. E.g. If you register a new client, you should be able to see the new client in the search of the admin site, etc.
  • Unit tests: This part you will see if the function is working (single function).You should test the validations, negative scenarios, positive scenarios… E.g. If you register a new client, you should be able to check if the message in the end is ok, if the client will appear in the database, if you can register an invalid client…

 

Hope you like it guys, I read the original article and it seemed to me that the guy doesn’t think it is too important end-to-end tests. I don’t agree with it, maybe for this reason I changed the way he was speaking about (If you still want to do end-to-end tests). End-to-End tests is important as any other test phase, the difference is that you don’t need to do all the small tests in this phase. It is just my thought… And you can write your comment, suggestion too in this post.


Thank you ! See you next week 🙂

Sourcehttp://googletesting.blogspot.com.br/2015/04/just-say-no-to-more-end-to-end-tests.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+blogspot/RLXA+(Google+Testing+Blog)&m=1

Webinar: Best Practices for Mobile CI – Sauce Labs

Hey guys, today I will post a webinar that I joined last week about Mobile CI.

It is very interesting, so you will see briefly when you need to use, why, how it works, etc. It lasts 43 minutes so, make sure that you have time to watch. Jump to 00:27:35 to watch the demo of Sauce Labs.

To download or watch you need follow the instructions in the link below:

https://saucelabsreview.webex.com/saucelabsreview/lsr.php?RCID=06e87f252b274b37b4500335b35ef868

 

What is CI for mobile ?

Screen Shot 2015-05-28 at 10.45.39

Automatically – every commit

How does it works?

Screen Shot 2015-05-28 at 10.46.57

Why ?

– Reduce the need for human-based testing (Save money and time)

– Faster Feedback

– Automate Everything (Very optimistic point of view)

– No touch configuration

– Automated OTA Distribution

– Code Validation: Automated Builds and Tests

 

How choose the right process ?

Screen Shot 2015-05-28 at 10.53.00

 

 

Depend the size of the

company.

 

 

Mobile is harder

– Infrastructure (If you are building in iOS – You will need XCode, a MacMini, etc.)

– Compilation/Code Signing (You have to configure your build machine, so you will need your key store file, mobile provisioning…)

– Testing (What is the right time to use, challenge to choose what is the better for the project- real devices, simulators…)

– Deployment

 

Simulators x Emulators x Devices

Simulators

– Used by iOS, included w/ XCode

– Execute i386 instruction set, not ARM

Upside: Very fast compared to Emulators

Downside : You don’t have access hardware functions (GPS,Bluetooth Radio, Accelerometers,etc)

Emulators

– Used by Android, included with Android sdks

– Execute ARM (or native device instruction set)

Upside: You can find free tools

– Downside: Very slow compared to Simulators

– Do not have access to certain hardware functions (GPS, Bluetooth Radio, Accelerometers, etc)

Devices

Upside: Reproduces the actual performance experienced by your users

– Upside: The only way to catch manufacturer/OEM bugs

Downside: Very expensive to configure and maintain compared

– Upside: Full access to hardware functions (GPS, Bluetooth Radio, Accelerometers, etc)

 

When to use devices vs simulators/emulators ?

– Emulators and Simulators are an excellent solution for running automated tests during development. They are inexpensive and will reliably catch most problems.

– Physical Devices can be used on a lower frequency (i.e. pre-release, weekly, daily, etc.). They are the only way to catch performance problems, test certain hardware features and find OEM issues. In the least, devices should be used before every release.

 

Unit Testing

– 77% of developers said app quality is “very important or mission critical”

– 80% of mobile developers test their apps manually

– Only 13% of mobile developers use automated testing

 

Unit vs Functional

Unit: Testing small pieces of code

Functional: Testing button clicks and UI interaction

Benefits:

– Instant Gratification !

– Repeatable

– Can automatically test every commit

Challenges:

– Unit Tests are not users

– Lot of work to write and maintain them

Which framework to use ?

– You need to remember:

– What is the language/framework do your developers know

– Open Source/Community Support

– 3rd party framework requirements

 

CI Tools/Services for mobile

Jenkins:

Open Source, Lots of plugins, iOS and Android 

 Self-Hosted, DYI solution (Setup all the environment, it isn’t easy this part)

Travis CI:

Hosted Solution, OS X support, lots of plugins, iOS and Android 

Tedious setup process

Ship.io:

Hosted solution, Designed specifically for mobile, Easy setup, iOS and Android 

Less flexible than other solutions

Xcode CI:

Integrated w/ Xcode, Apple-Supported

Self-hosted, iOS Only

Thank you guys, thank you Sauce Labs ! Hope you like it !

%d bloggers like this: