Verification and Validation

Testing is the process which begins right from Requirement phase; hence the idea of catching defects at each stage, before the actual testing is started leads us to define Verification and Validation.
Verification and Validation are the activities performed to improve the quality and reliability of the system and assure the product satisfies the customer needs. Verification assures the product of each development phase meets their respective requirements. Validation assures the final product meets the client requirements.

When we ask the question "Are we producing the product right?” the activities which take care are part of Verification. Whereas when we ask "Are we producing the right product?" the activities which take care are part of Validation. Hence activities like Requirement Review, Design Review, Code Review, etc... Are the part of Verification and activities like Unit Testing, Integration Testing, System Testing are the part of Validation.

General Definition:
Verification is the process of evaluating a system or component to determine whether the product of a given phase satisfies the conditions imposed at the start of that phase.
Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

Verification is a Quality improvement process. It is a Low level activity performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards; it demonstrates consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.
Validation process ensures the functionality. It is a High level activity performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment, it determines correctness of the final software product by a development project with respect to the user needs and requirements

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

 

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Severity Vs Priority

Severity and Priority are two completely different concepts when it comes to managing Software defects.
•    Bug severity indicates how much effect a bug is doing on the application.
•    Bug priority indicates how important or how quickly it is to fix that bug.

Severity defines the impact that a given defect has on the system.
They are classified in following sections

1. Blocker
   Application crashes or fails to initialize. Database connectivity issue where system is unable to fetch data or fetches wrong data.
2. Critical
   Important feature in test fails.
3. Major
   Secondary feature in test is not operating as expected.
4. Normal
   Secondary feature in test has some not so important issues. Minor feature in test is not operating as expected.
5. Minor
   Secondary feature in test has some UI issues. Minor feature in test has some not so important issues.
6. Trivial
   Minor feature in test has some UI issues or typos.
7. Enhancement
   Any modification in the existing product which is not covered in requirements, but the stated modification shall be essential for look and feel or for business.

These days Normal and Trivial are not in much use, most of the bug reporting tool have Blocker, Critical, Major, Minor and Enhancement as severity values in their drop-down for Severity field.

Priority defines the order in which we should resolve a defect.
They are classified in following sections

1. Priority 1
   -    Should be fixed as soon as possible.
   -    The application hangs or crashes (Blockers, Critical and Show stopper Issues).
   -    Data loss taking place.
   -    Memory Leakages are identified.
   -    Implementation is opposite to the requirement specified.
2. Priority 2
   -    Should be considered to fix as soon as possible.
   -    Having important feature some logical issues. (Wrong functionality implemented).
   -    Performance Issues.
   -    Major Cosmetic Issues
   -    Important Typo errors to be fixed
3. Priority 3
   -    Should be considered to fix.
   -    Having secondary or other feature some logical issues. (Wrong functionality implemented).
   -    Minor Cosmetic Issues
4. Priority 4
   -    Should be considering to be fixed before build is released to clients.
   -    Minor Cosmetic Issues.
   -    Other Typo Errors.
5. Priority 5
   -    Enhancement and Suggestion.

Some bug reporting tool as High, Medium and Low as their Priority values in their drop-down for Priority values.

Following are the examples where different combination of Severity and Priority can be clubbed.

• High severity, High priority
  If we need to print some data on paper or display some data on screen, and you observed that the data is not printed. For example a Mobile number which is a mandatory field is to be entered in an application and it is to be printed, but it is not printed even though user has entered data. The scenario stated is of High Severity and High Priority. 
• High severity, Low priority
  If we need to print some data on paper or display some data on screen, and you observed that the data is printed but not in order in which we expected. For example an address is to be printed as Building name, followed by Flat No but in actual implementation the order is not as per requirement. The scenario stated is of High Severity but with Low Priority. 
• Low severity, High priority
  If we need to print some data on paper or display some data on screen, and you observed that the data is printed but it gets trimmed at the end. For example a Mobile number which is a mandatory field is to be entered in an application and it is to be printed, but it is not printed completed. For example user enters 10 digits and in actual implementation only 8 – 9 digits are printed even though there is more space which permits more digit to be printed. The scenario stated is of Low Severity and High Priority. 
• Low severity, Low priority
  If we need to print some data on paper or display some data on screen, and you observed that the data is printed but some punctuation are missing as we expected. For example an address is to be printed as Building name followed by comma and Flat No but in actual implementation the comma is missing. The scenario stated is of Low Severity but with Low Priority.

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

A Story: Software Developer Versus Software Testing Guru

Hello Friends,

I had received a mail few years back a converstaion between Testing Guru and University Scholar. I would like to share the story with you all.

---

A university scholar, Mr. John Smith approaches his friend a software-testing guru telling him that he has a Bachelor in programming,

and now would like to learn the software testing to complete his knowledge and to find a job as a software tester. After summing him up for a few

minutes, the software-testing guru told him "I seriously doubt that you are ready to study software testing. It's the serious topic. If you wish

however I am willing to examine you in logic, and if you pass the test I will help teach you software testing. "

The young man agrees. Software testing guru holds up two fingers "Two men come down a chimney.

One comes with a clean face and the other comes out with a dirty face. Which one washes his face?

The young man stares at the software-testing guru. "Is that a test in Logic?" software testing guru nods.

"The one with the dirty face washes his face," He answers wearily.

"Wrong. The one with the clean face washes his face. Examine the simple logic. The one with the dirty face looks at the one with the clean face

and thinks his face is clean. The one with the clean face looks at the one with the dirty face and thinks his face is dirty. So; the one with the

clean face washes his face."

"Very clever" Says Smith. "Give me another test"

The software-testing guru again holds up two fingers "Two men come down a chimney. One comes out with a clean face and the other comes out with a

dirty face. Which one washes his face?

"We have already established that. The one with the clean face washes his face"

"Wrong. Each one washes his face. Examine the simple logic. The one with the dirty face looks at the one with the clean face and thinks his face is

clean. The one with the clean face looks at the one with the dirty face and thinks his face is dirty. So; the one with the clean face washes his

face. When the one with the dirty face sees the one with the clean face washing his face, he also washes his face. So each one washes his face"

"I didn't think of that!" Says Smith. " It's shocking to me that I could make an error in logic. Test me again!."

The software-testing guru holds up two fingers "Two men come down a chimney. One comes out with a clean face and the other comes out with a

dirty face. Which one washes his face?

"Each one washes his face"

"Wrong. Neither one washes his face. Examine the simple logic. The one with the dirty face looks at the one with the clean face and thinks his

face is clean. The one with the clean face looks at the one with the dirty face and thinks his face is dirty. But when the one with clean face sees

that the one with the dirty face doesn't wash his face, he also doesn't wash his face So neither one washes his face".

Smith is desperate. "I am qualified to study software testing. Please give me one more test"

He groans when the software-testing guru lifts his two fingers, "Two men come down a chimney. One comes out with a clean face and the other comes

out with a dirty face. Which one washes his face?

"Neither one washes his face"

"Wrong. Do you now see, John, why programming knowledge is an insufficient basis for studying the software testing? Tell me, how is it possible for

two men to come down the same chimney, and for one to come out with a clean face and the other with a dirty face? Don’t you see?

GUYS !!! Requirements for testers will look the same but expectations from clients will vary every now and then…

So never under estimate TESTERS.. 

---

I hope you all like the conversation, Please feel free to share it among your friends.
As Development is a core strength of Software Industry, Testing is the spine to it. Both goes hand in hand.

Don't forget testing is a creative and challenging task. Finally it depends on your skill and experience, how you handle this challenge.

Defect (Bug) Life Cycle

A Defect/Bug Life Cycle is made up of series of phases.

1. New
    When the bug is identified and logged in bug tracking tool for the first time, its state will be "NEW".
2. Open / Closed
    After a QA has posted a bug, the QA lead validates the bug. If bug is valid then he changes the state as "OPEN" and if the bug is invalid then the lead changes its state to "CLOSED".
3. Assign
    After QA lead changes the state as "OPEN", he assigns the bug to corresponding developer or developer team lead and the bug status is changed to "ASSIGN".
4. Rejected:
    If the developer feels that the bug is not valid or it has some technical limitations and cannot be fixed, he rejects the bug. He changes the state of bug to "REJECTED".
5. Duplicate:
    If the bug is logged is repeated twice or the two bugs reported has alike results and steps to reproduce, then one bug status is changed to "DUPLICATE".   
6. Deferred:
    If the development team lead decides to fix the bug in next release due to lack of time or the priority of the bug is low then he changes the state to "DEFERRED", which is later changes to "ASSIGN" when the bug is taken in consideration to be fixed.
7. InTest
    Once the developer fixes the bug, he assigns the bug to the testing team for next round of testing. Before that he changes the state of bug to "IN TEST". It specifies that the bug has been fixed and is released to testing team.
8. Verified
    Once the bug is fixed and the status is changed to "IN TEST", the tester tests the bug. If the bug is not reproducible in the software, he changes the status to "VERIFIED".
9. Reopened
    Once the bug is fixed and the status is changed to "IN TEST", the tester tests the bug. If the bug is reproducible in the software, he changes the status to "REOPENED".
10. Closed
    After the bug status is changes to "REJECTED" or "DUPLICATE" or "VERIFIED" the QA Lead verifies the comments added by the development or Testing team. When he is satisfied with the comments he changes the state to "CLOSED".

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Software Testing Life Cycle (STLC)

A Software Testing Life Cycle is made up of series of phases.

1.       Requirement Gathering and Analysis

Requirements analysis is critical to the success of any project. The scope and boundary of the proposed software solution is drawn. Identifying stakeholders such as customers, end-users, system administrators etc. is the next step in requirements analysis. This is one of the most important steps in the whole process as proper identification of stakeholders enables the business analyst to draw a road map for gathering requirements. Testing Team identifies the testable requirements. Requirement can be functional or non functional. Scope of Automation (if any) is also considered.

Activities

·         Prepare Requirement Traceability Matrix (RTM).

·         Identifying the approach or setting the priorities module wise.

·         Identification of areas to be tested automatically and those requiring manual testing

Out-Put After This Phase

·         Requirement Traceability Matrix (RTM).

2.       Test Planning

A test plan is a document is created which provides detail of a systematic approach to testing. Testing Effort and Cost Estimation for the project would be calculated.

Activities

·         Project scope and expectations definition

·         Technology and methodology selection

·         Preparation of test plan document for various types of testing to be performed.

·         Testing effort estimation 

·         Definition of QA metrics

·         Resource Planning and identifying the roles and responsibility. 

·         Testing Tool Selection.

Out-Put After This Phase

·         Test Plan

·         Testing Effort Estimation

3.       Test Case Development

After Requirement is freeze followed by initial understanding, testing team starts creating Test Case or Test Scripts along with test data, which are later reviewed followed by rectifying the issues reported by reviewer. Categorization of Test Case as per their priority is also set.

Activities

·         Create test cases or Test scripts (if applicable).

·         Categorization of Test Case.

·         Create Test Data

Out-Put After This Phase

·         Test Cases/Scripts

·         Test Data

4.       Test Environment Setup

A testing environment is a setup of software and hardware on which the testing team is going to perform the testing of the software product. This setup consists of the physical setup of hardware and software setup that includes Server Operating system, client operating system, database server, front end running environment, browser (if web application) and other software components required to run this software product. These setups are done by support staff. Testing team has to ensure the conditions are met before testing is taken off. Followed by Smoke Test which verifies the Test Environment as well as stability of the build delivered to Test Team.

Activities

·         Ensure the required Hardware and Software or different combination of machines are ready

·         Perform a Smoke Test on the build delivered for Testing

Out-Put After This Phase

·         Test Machine or Test Lab

·         Smoke Test Result

5.       Test Case Execution

Test Case execution is a process of executing test cases/scripts in a logical sequence with specific test data (if available). If any Test Case/Script or Test Scenario fails, Testing Team logs the bug in Bug Tracking tool (Bugzilla, Jira, TTPro, etc.). Simultaneously a dedicated team of developers would be fixing the bugs and release the fixes in next build. Testing team shall verify the reported bug and accordingly update the Bugs logged. This cycle continues till all the bugs identified in product are fixed. Finally a Regression or Ad-hoc Testing is performed after which Test Report is prepared if all goes well.

Activities

·         Execution of Test Cases

·         Log bugs in bug tracking tool

·         Create Test Result document

·         Map defects to Test Cases in RTM

·         Verify the bug logged in new build

·         Update the bug status in bug reporting tool as per verification status

Out-Put After This Phase

·         Test Result

·         Defect Report

·         RTM with execution status

6.       Closure - Product Release

Once the test meets the exit criteria, the activities such as capturing the key outputs, lessons learned, results, logs, documents related to the project are noted down and used as a reference for future projects. Later a Causal Analysis and Resolution (CAR) Report is prepared where in best practices and cause of failures occurred are noted down.

 Activities

·         Test Metrics based on Test coverage, Cost, Time, Quality etc is prepared

·         Test Closure Report Preparation

·         Test Result Analysis is performed to calculate the defect distribution by type and severity.

·         Causal Analysis and Resolution (CAR) Report is prepared

Out-Put After This Phase

·         Test Closure Report 

·         Test Metrics

·         Causal Analysis and Resolution (CAR) Report

Note: There are different process and methodologies followed in various organizations. The above content is as per my experience in industry so far.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.

Tips for being a good Software Tester.

Following are the mantra's or tips for being a good Tester.

1. One should be familiar with Software Development Life Cycle (SDLC) and Software Testing Life Cycle (STLC). Knowledge of database is a must too.
2. One should have good communication skills, where in Test case written by him should be easy to understand by other.
3. One should have or acquire good domain knowledge. He should know business and it's scenarios for which he is testing.
4. One should be a good listener, observer and analyzer to understand client's requirement and identify the loop holes or technical challenges one might face while testing.
5. One should have an open or creative mindset while writing Test Cases or executing the same to identify more new creative scenarios.
6. One should always try to identify negative test case against each positive test case to cover maximum possible test scenarios.
7. One should always explore the area thoroughly or one should always check related components for which a bug or defect is identified.
8. One should always think as a Customer. He should be able to identify the mistakes a user can do while performing a scenario and check if those are handled properly.
9. Always Test the tests First. Always execute the Test Case First followed by ad-hoc testing as per your domain knowledge.
10. One should have the knowledge of different  bug reporting tool.
11. One should be wise in choosing Severity and Priority of any issue reported.
12. Bug's reported should be to the point and steps to reproduce should be descriptive.
13. Should always maintain a good reputation with development team to make them understand issue reported, One should be able to make understand the impact of the issue on the business.
14. Most important - One should have passion for the job and should enjoy the testing, while keep your personal grudge (if any) aside and be a honest critic.

Please feel free to write comments if you like my post or else if your opinion is different or you may like to add any more points in them.