Software Testing

Types of Testing

2012-02-22T11:44:00.001+05:30

There are many types of Testing. Refer website "Guru99.com" for the list. But I would like to set our focus on few which we use often.

White Box Testing
Testing based on an analysis of internal workings and structure of a piece of software which is based on knowledge of internal logic of application's code is known as White box testing. It is also known as Glass box Testing. In White box testing one would have knowledge of development language and general OOPS concept.

Black Box Testing
Testing a feature without having access to code, instead testing functionality of an application with considering only requirements. In Black box testing one should have knowledge of the product in test and should compare with the requirements.

Smoke Testing
Smoke Test is performed to verify the stability of the build under test; it's a quick test that major functionality is tested without bothering with finer details. Smoke Test is generally automated and it takes between 20 minutes to 3 hours (depending upon functionality to be tested).

Sanity Testing
Sanity Test is performed on new software initially to decide if the build is stable enough and can be tested further.

Regression Testing
Regression is performed to verify the changes introduced during the release of new build are working as per expectation, the other functionality due to that fixed are not hampers and a new issue is not injected in the new build.

Performance Testing
Performance Test is the most effective way to gauge an application or an environment's capacity and scalability. This is useful to identify performance bottlenecks in application. Performance Testing is performed to verify whether system meets performance requirements.

Stress Testing
Stress Test is testing under heavy load beyond system capacity. The objective of Stress testing is to identify the load under which it fails and how. Stress Testing is similar to Performance testing but in Stress Testing we use a very high level of simulated load.

Load Testing
Load Test is monitoring the behavior of system under heavy load, wherein keeping the track of system response time and the graph which degrades or fail with time.

Unit Testing
Unit Test is often performed by developers and not testers, wherein it is a testing of software components or modules which is tested before it is integrated with other modules.

Compatibility Testing
Compatibility Test is often performed to determine whether the application is compatible with other systems that it needs to interface with. For a Windows application it can be tested across different OS, the look and feel as well as the functionality can be tested on the other hand for a web based application it can be tested across different browser and their sub version.

User Acceptance Testing
User Acceptance Test is black box is testing which is performed by client to verify the system functionality and usability prior to the system being moved to production. The acceptance test will be the responsibility of the client which is performed in presence of project team. A third party test team can work with the client to perform User Acceptance Testing.

I will try to blog more types of testing with more details in father, but if you want to know more about other types of testing please drop a comment and I will try to surely answer your queries as soon as possible.

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Smoke Testing and Sanity Testing

2012-02-02T18:06:00.000+05:30

Smoke Testing

In Smoketesting a set of test cases are executed on every new build to verify thatbuild is testable before test team accepts it for further testing. The testcases selected for smoke testing are the one which are of important for thebuild in test which is released. If any major functionality fails under smoketesting, the testing team rejects the build and conducts no further testing.Focus of Smoke testing should be on stability of the build released. In smoketesting mostly all features under test are tested, but not in much details.

SmokeTesting is also known as Build Verification Testing (BVT) or Build AcceptanceTesting (BAT).

Why weperform smoke testing?

The advantage of smoke testingis it saves the efforts of a test team considerably. Normally smoke testingshould not take more than 20-40 minutes.

What wedo in smoke testing?

In Smoke Testing important or identifiedtest cases are executed and if smoke test fails, build is rejected and a newbuild is released after the bug is fixed.

Sanity Testing

In Sanitytesting a set of important test cases for functionality in test are executed.The test cases selected for sanity testing are the one which are of importantfor a feature or two in test which is released. Sanity testing is a subset of regressiontesting. Focuses of Sanity testing are only one or feature in build releasedwhere some changes are made.

Why weperform smoke testing?

In Sanity testing we test onlythe few features which are released our focus is on the fixed issue and theiraffected area.

What wedo in smoke testing?

In Sanity testing a set ofimportant test cases for functionality in test are executed. The tester shouldfocus on the fixed issues and the components linked or affected due to thefixes.

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Verification and Validation

2012-01-31T14:34:00.001+05:30

Testing is the process which begins right from Requirement phase; hence the idea of catching defects at each stage, before the actual testing is started leads us to define Verification and Validation.
Verification and Validation are the activities performed to improve the quality and reliability of the system and assure the product satisfies the customer needs. Verification assures the product of each development phase meets their respective requirements. Validation assures the final product meets the client requirements.

When we ask the question "Are we producing the product right?” the activities which take care are part of Verification. Whereas when we ask "Are we producing the right product?" the activities which take care are part of Validation. Hence activities like Requirement Review, Design Review, Code Review, etc... Are the part of Verification and activities like Unit Testing, Integration Testing, System Testing are the part of Validation.

General Definition:
Verification is the process of evaluating a system or component to determine whether the product of a given phase satisfies the conditions imposed at the start of that phase.
Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

Verification is a Quality improvement process. It is a Low level activity performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards; it demonstrates consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.
Validation process ensures the functionality. It is a High level activity performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment, it determines correctness of the final software product by a development project with respect to the user needs and requirements

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Severity Vs Priority

2012-01-27T15:41:00.000+05:30

Severity and Priority are two completely different concepts when it comes to managing Software defects.
• Bug severity indicates how much effect a bug is doing on the application.
• Bug priority indicates how important or how quickly it is to fix that bug.

Severity defines the impact that a given defect has on the system.
They are classified in following sections

Blocker
Application crashes or fails to initialize. Database connectivity issue where system is unable to fetch data or fetches wrong data.
Critical
Important feature in test fails.
Major
Secondary feature in test is not operating as expected.
Normal
Secondary feature in test has some not so important issues. Minor feature in test is not operating as expected.
Minor
Secondary feature in test has some UI issues. Minor feature in test has some not so important issues.
Trivial
Minor feature in test has some UI issues or typos.
Enhancement
Any modification in the existing product which is not covered in requirements, but the stated modification shall be essential for look and feel or for business.

These days Normal and Trivial are not in much use, most of the bug reporting tool have Blocker, Critical, Major, Minor and Enhancement as severity values in their drop-down for Severity field.

Priority defines the order in which we should resolve a defect.
They are classified in following sections

Priority 1
-    Should be fixed as soon as possible.
-    The application hangs or crashes (Blockers, Critical and Show stopper Issues).
-    Data loss taking place.
-    Memory Leakages are identified.
-    Implementation is opposite to the requirement specified.
Priority 2
-    Should be considered to fix as soon as possible.
-    Having important feature some logical issues. (Wrong functionality implemented).
-    Performance Issues.
-    Major Cosmetic Issues
-    Important Typo errors to be fixed
Priority 3
-    Should be considered to fix.
-    Having secondary or other feature some logical issues. (Wrong functionality implemented).
-    Minor Cosmetic Issues
Priority 4
-    Should be considering to be fixed before build is released to clients.
-    Minor Cosmetic Issues.
-    Other Typo Errors.
Priority 5
- Enhancement and Suggestion.

Some bug reporting tool as High, Medium and Low as their Priority values in their drop-down for Priority values.

Following are the examples where different combination of Severity and Priority can be clubbed.

High severity, High priority
If we need to print some data on paper or display some data on screen, and you observed that the data is not printed. For example a Mobile number which is a mandatory field is to be entered in an application and it is to be printed, but it is not printed even though user has entered data. The scenario stated is of High Severity and High Priority.
High severity, Low priority
If we need to print some data on paper or display some data on screen, and you observed that the data is printed but not in order in which we expected. For example an address is to be printed as Building name, followed by Flat No but in actual implementation the order is not as per requirement. The scenario stated is of High Severity but with Low Priority.
Low severity, High priority
If we need to print some data on paper or display some data on screen, and you observed that the data is printed but it gets trimmed at the end. For example a Mobile number which is a mandatory field is to be entered in an application and it is to be printed, but it is not printed completed. For example user enters 10 digits and in actual implementation only 8 – 9 digits are printed even though there is more space which permits more digit to be printed. The scenario stated is of Low Severity and High Priority.
Low severity, Low priority
If we need to print some data on paper or display some data on screen, and you observed that the data is printed but some punctuation are missing as we expected. For example an address is to be printed as Building name followed by comma and Flat No but in actual implementation the comma is missing. The scenario stated is of Low Severity but with Low Priority.

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Never Under Estimate TESTERS

2012-01-27T12:17:00.000+05:30

Hello Friends,

I had received a mail few years back a converstaion between Testing Guru and University Scholar. I would like to share the story with you all.

A university scholar, Mr.John Smith approaches his friend a software-testing guru telling him that hehas a Bachelor in programming,

and now would like to learnthe software testing to complete his knowledge and to find a job as a softwaretester. After summing him up for a few

minutes, the software-testingguru told him "I seriously doubt that you are ready to study softwaretesting. It's the serious topic. If you wish

however I am willing toexamine you in logic, and if you pass the test I will help teach you softwaretesting. "

The young man agrees.Software testing guru holds up two fingers "Two men come down a chimney.

One comes with a clean faceand the other comes out with a dirty face. Which one washes his face?

The young man stares at thesoftware-testing guru. "Is that a test in Logic?" software testingguru nods.

"The one with the dirtyface washes his face," He answers wearily.

"Wrong. The one with theclean face washes his face. Examine the simple logic. The one with the dirtyface looks at the one with the clean face

and thinks his face is clean.The one with the clean face looks at the one with the dirty face and thinks hisface is dirty. So; the one with the

clean face washes hisface."

"Very clever" SaysSmith. "Give me another test"

The software-testing guruagain holds up two fingers "Two men come down a chimney. One comes outwith a clean face and the other comes out with a

dirty face. Which one washeshis face?

"We have alreadyestablished that. The one with the clean face washes his face"

"Wrong. Each one washeshis face. Examine the simple logic. The one with the dirty face looks at theone with the clean face and thinks his face is

clean. The one with the cleanface looks at the one with the dirty face and thinks his face is dirty. So; theone with the clean face washes his

face. When the one with thedirty face sees the one with the clean face washing his face, he also washeshis face. So each one washes his face"

"I didn't think ofthat!" Says Smith. " It's shocking to me that I could make an errorin logic. Test me again!."

The software-testing guruholds up two fingers "Two men come down a chimney. One comes out with aclean face and the other comes out with a

dirty face. Which one washeshis face?

"Each one washes hisface"

"Wrong. Neither onewashes his face. Examine the simple logic. The one with the dirty face looks atthe one with the clean face and thinks his

face is clean. The one withthe clean face looks at the one with the dirty face and thinks his face isdirty. But when the one with clean face sees

that the one with the dirtyface doesn't wash his face, he also doesn't wash his face So neither one washeshis face".

Smith is desperate. "Iam qualified to study software testing. Please give me one more test"

He groans when thesoftware-testing guru lifts his two fingers, "Two men come down a chimney.One comes out with a clean face and the other comes

out with a dirty face. Whichone washes his face?

"Neither one washes hisface"

"Wrong. Do you now see,John, why programming knowledge is an insufficient basis for studying thesoftware testing? Tell me, how is it possible for

two men to come down the samechimney, and for one to come out with a clean face and the other with a dirtyface? Don’t you see?

GUYS !!! Requirements fortesters will look the same but expectations from clients will vary every nowand then…

So never under estimateTESTERS..

I hope you all like the conversation, Please feel free to share it among your friends.
As Development is a core strength of Software Industry, Testing is the spine to it. Both goes hand in hand.

Don't forget testing is a creative and challenging task. Finally it depends on your skill and experience, how you handle this challenge.

Defect (Bug) Life Cycle

2012-01-23T19:45:00.001+05:30

A Defect/Bug Life Cycle is made up of series of phases.

1. New
    When the bug is identified and logged in bug tracking tool for the first time, its state will be "NEW".
2. Open / Closed
    After a QA has posted a bug, the QA lead validates the bug. If bug is valid then he changes the state as "OPEN" and if the bug is invalid then the lead changes its state to "CLOSED".
3. Assign
    After QA lead changes the state as "OPEN", he assigns the bug to corresponding developer or developer team lead and the bug status is changed to "ASSIGN".
4. Rejected:
    If the developer feels that the bug is not valid or it has some technical limitations and cannot be fixed, he rejects the bug. He changes the state of bug to "REJECTED".
5. Duplicate:
    If the bug is logged is repeated twice or the two bugs reported has alike results and steps to reproduce, then one bug status is changed to "DUPLICATE".
6. Deferred:
    If the development team lead decides to fix the bug in next release due to lack of time or the priority of the bug is low then he changes the state to "DEFERRED", which is later changes to "ASSIGN" when the bug is taken in consideration to be fixed.
7. InTest
    Once the developer fixes the bug, he assigns the bug to the testing team for next round of testing. Before that he changes the state of bug to "IN TEST". It specifies that the bug has been fixed and is released to testing team.
8. Verified
    Once the bug is fixed and the status is changed to "IN TEST", the tester tests the bug. If the bug is not reproducible in the software, he changes the status to "VERIFIED".
9. Reopened
    Once the bug is fixed and the status is changed to "IN TEST", the tester tests the bug. If the bug is reproducible in the software, he changes the status to "REOPENED".
10. Closed
    After the bug status is changes to "REJECTED" or "DUPLICATE" or "VERIFIED" the QA Lead verifies the comments added by the development or Testing team. When he is satisfied with the comments he changes the state to "CLOSED".

Note: There are different process and methodologies followedin various organizations. The above content is as per my experience in industryso far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Software Testing Life Cycle (STLC)

2012-01-16T19:04:00.000+05:30

A Software Testing Life Cycle is made up of series ofphases.

1. RequirementGathering and Analysis

Requirementsanalysis is critical to the success of any project.The scope andboundary of the proposed software solution is drawn. Identifying stakeholderssuch as customers, end-users, system administrators etc. is the next step inrequirements analysis. This is one of the most important steps in the wholeprocess as proper identification of stakeholders enables the business analystto draw a road map for gathering requirements. Testing Team identifies thetestable requirements. Requirement can be functional or non functional. Scopeof Automation (if any) is also considered.

Activities

· Prepare Requirement Traceability Matrix (RTM).

· Identifying the approach or setting thepriorities module wise.

· Identification of areas to be testedautomatically and those requiring manual testing

Out-Put After ThisPhase

· Requirement Traceability Matrix (RTM).

2. TestPlanning

A test plan is a document is created which provides detail of asystematic approach to testing. Testing Effort and Cost Estimation for theproject would be calculated.

Activities

· Project scope and expectations definition

· Technology and methodology selection

· Preparation of test plan document for varioustypes of testing to be performed.

· Testing effort estimation

· Definition of QA metrics

· Resource Planning and identifying the roles andresponsibility.

· Testing Tool Selection.

Out-Put After This Phase

· Test Plan

· Testing Effort Estimation

3. Test Case Development

After Requirement is freeze followed byinitial understanding, testing team starts creating Test Case or Test Scriptsalong with test data, which are later reviewed followed by rectifying theissues reported by reviewer. Categorization of Test Case as per their priorityis also set.

Activities

· Create test cases or Test scripts (ifapplicable).

· Categorization of Test Case.

· Create Test Data

Out-Put After This Phase

· Test Cases/Scripts

· Test Data

4. Test Environment Setup

A testing environment is a setup ofsoftware and hardware on which the testing team is going to perform the testingof the software product. This setup consists of the physical setup of hardwareand software setup that includes Server Operating system, client operatingsystem, database server, front end running environment, browser (if webapplication) and other software components required to run this softwareproduct. These setups are done by support staff. Testing team has to ensure theconditions are met before testing is taken off. Followed by Smoke Test whichverifies the Test Environment as well as stability of the build delivered toTest Team.

Activities

· Ensure the required Hardware and Software ordifferent combination of machines are ready

· Perform a Smoke Test on the build delivered forTesting

Out-Put After This Phase

· Test Machine or Test Lab

· Smoke Test Result

5. Test Case Execution

Test Case execution is a process of executingtest cases/scripts in a logical sequence with specific test data (ifavailable). If any Test Case/Script or Test Scenario fails, Testing Team logsthe bug in Bug Tracking tool (Bugzilla, Jira, TTPro, etc.). Simultaneously a dedicatedteam of developers would be fixing the bugs and release the fixes in nextbuild. Testing team shall verify the reported bug and accordingly update the Bugslogged. This cycle continues till all the bugs identified in product are fixed.Finally a Regression or Ad-hoc Testing is performed after which Test Report isprepared if all goes well.

Activities

· Execution of Test Cases

· Log bugs in bug tracking tool

· Create Test Result document

· Map defects to Test Cases in RTM

· Verify the bug logged in new build

· Update the bug status in bug reporting tool asper verification status

Out-Put After This Phase

· Test Result

· Defect Report

· RTM with execution status

6. Closure - Product Release

Once the test meets the exit criteria, theactivities such as capturing the key outputs, lessons learned, results, logs,documents related to the project are noted down and used as a reference forfuture projects. Later a Causal Analysis and Resolution (CAR) Report is preparedwhere in best practices and cause of failures occurred are noted down.

Activities

· Test Metrics based on Test coverage, Cost, Time,Quality etc is prepared

· Test Closure Report Preparation

· Test Result Analysis is performed to calculate thedefect distribution by type and severity.

· Causal Analysis and Resolution (CAR) Report isprepared

Out-Put After This Phase

· Test Closure Report

· Test Metrics

· Causal Analysis and Resolution (CAR) Report

Note: There are different process and methodologies followedin various organizations. The above content is as per my experience in industryso far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Tips for being a good Software Tester.

2012-01-13T14:31:00.000+05:30

Following are the mantra's or tips for being a good Tester.

One should be familiar with Software Development Life Cycle (SDLC) and Software Testing Life Cycle (STLC). Knowledge of database is a must too.
One should have good communication skills, where in Test case written by him should be easy to understand by other.
One should have or acquire good domain knowledge. He should know business and it's scenarios for which he is testing.
One should be a good listener, observer and analyzer to understand client's requirement and identify the loop holes or technical challenges one might face while testing.
One should have an open or creative mindset while writing Test Cases or executing the same to identify more new creative scenarios.
One should always try to identify negative test case against each positive test case to cover maximum possible test scenarios.
One should always explore the area thoroughly or one should always check related components for which a bug or defect is identified.
One should always think as a Customer. He should be able to identify the mistakes a user can do while performing a scenario and check if those are handled properly.
Always Test the tests First. Always execute the Test Case First followed by ad-hoc testing as per your domain knowledge.
One should have the knowledge of different bug reporting tool.
One should be wise in choosing Severity and Priority of any issue reported.
Bug's reported should be to the point and steps to reproduce should be descriptive.
Should always maintain a good reputation with development team to make them understand issue reported, One should be able to make understand the impact of the issue on the business.
Most important - One should have passion for the job and should enjoy the testing, while keep your personal grudge (if any) aside and be a honest critic.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.