Sunday, 31 July 2011

Practical Tips and Insight on How to Test Database

Database is one of the inevitable parts of a software application these days. It does not matter at all whether it is web or desktop, client server or peer to peer, enterprise or individual business, database is working at back-end. Similarly, whether it is health-care of finance, leasing or retail, mailing application or controlling spaceship, behind the scene a database is always in action.

Moreover, as the complexity of application increases the need of stronger and secure database emerges. In the same way, for the applications with high frequency of transactions (e.g. banking or finance application), necessity of fully featured DB Tool is coupled.

Currently, several database tools are available in the market e.g. MS-Access2010, MS SQL Server 2008 r2, Oracle 10g, Oracle Financial, MySQL, PostgreSQL, DB2 etc.  All of these vary in cost, robustness, features and security. Each of these DBs possesses its own benefits and drawbacks. One thing is certain; a business application must be built using one of these or other DB Tools.

Before I start digging into the topic, let me comprehend the foreword. When the application is under execution, the end user mainly utilizes the ‘CRUD’ operations facilitated by the DB Tool.

C: Create – When user ‘Save’ any new transaction, ‘Create’ operation is performed.
R: Retrieve – When user ‘Search’ or ‘View’ any saved transaction, ‘Retrieve’ operation is performed.
U: Update – when user ‘Edit’ or ‘Modify’ an existing record, the ‘Update’ operation of DB is performed.
D: Delete – when user ‘Remove’ any record from the system, ‘Delete’ operation of DB is performed.

It does not matter at all, which DB is used and how the operation is preformed. End user has no concern if any join or sub-query, trigger or stored-procedure, query or function was used to do what he wanted. But, the interesting thing is that all DB operations performed by user, from UI of any application, is one of the above four, acronym as CRUD.

Database Testing

As a database tester one should be focusing on following DB testing activities:
What to test in database testing:
1) Ensure data mapping:

Make sure that the mapping between different forms or screens of AUT and the Relations of its DB is not only accurate but is also according to design documents. For all CRUD operations, verify that respective tables and records are updated when user clicks ‘Save’, ‘Update’, ‘Search’ or ‘Delete’ from GUI of the application.
2) Ensure ACID Properties of Transactions:

ACID properties of DB Transactions refer to the ‘Atomicity’, ‘Consistency’, ‘Isolation’ and ‘Durability’. Proper testing of these four properties must be done during the DB testing activity. This area demands more rigorous, thorough and keen testing when the database is distributed.
3) Ensure Data Integrity:

Consider that different modules (i.e. screens or forms) of application use the same data in different ways and perform all the CRUD operations on the data. In that case, make it sure that the latest state of data is reflected everywhere. System must show the updated and most recent values or the status of such shared data on all the forms and screens. This is called the Data Integrity.
4) Ensure Accuracy of implemented Business Rules:

Today, databases are not meant only to store the records. In fact, DBs have been evolved into extremely powerful tools that provide ample support to the developers in order to implement the business logic at DB level. Some simple examples of powerful features of DBs are ‘Referential Integrity’, relational constrains, triggers and stored procedures. So, using these and many other features offered by DBs, developers implement the business logic on DB level. Tester must ensure that the implemented business logic is correct and works accurately.

Above points describe the four most important ‘What Tos’ of database testing. Now, I will put some light on ‘How Tos’ of DB Testing. But, first of all I feel it better to explicitly mention an important point. DB Testing is a business critical task, and it should never be assigned to a fresh or inexperienced resource without proper training.
How To Test Database:
1. Create your own Queries

In order to test the DB properly and accurately, first of all a tester should have very good knowledge of SQL and specially DML (Data Manipulation Language) statements. Secondly, the tester should acquire good understanding of internal DB structure of AUT. If these two pre-requisites are fulfilled, then the tester is ready to test DB with complete confidence. (S)He will perform any CRUD operation from the UI of application, and will verify the result using SQL query.

This is the best and robust way of DB testing especially for applications with small to medium level of complexity. Yet, the two pre-requisites described are necessary. Otherwise, this way of DB testing cannot be adopted by the tester.

Moreover, if the application is very complex then it may be hard or impossible for the tester to write all of the needed SQL queries himself or herself. However, for some complex queries, tester may get help from the developer too. I always recommend this method for the testers because it does not only give them the confidence on the testing they have performed but, also enhance their SQL skill.
2. Observe data table by table

If the tester is not good in SQL, then he or she may verify the result of CRUD operation, performed using GUI of the application, by viewing the tables (relations) of DB. Yet, this way may be a bit tedious and cumbersome especially when the DB and tables have large amount of data.

Similarly, this way of DB testing may be extremely difficult for tester if the data to be verified belongs to multiple tables. This way of DB testing also requires at least good knowledge of Table structure of AUT.
3. Get query from developer

This is the simplest way for the tester to test the DB. Perform any CRUD operation from GUI and verify its impacts by executing the respective SQL query obtained from the developer. It requires neither good knowledge of SQL nor good knowledge of application’s DB structure.

So, this method seems easy and good choice for testing DB. But, its drawback is havoc. What if the query given by the developer is semantically wrong or does not fulfill the user’s requirement correctly? In this situation, the client will report the issue and will demand its fix as the best case. While, the worst case is that client may refuse to accept the application.


Conclusion:

Database is the core and critical part of almost every software application. So DB testing of an application demands keen attention, good SQL skills, proper knowledge of DB structure of AUT and proper training.

In order to have the confident test report of this activity, this task should be assigned to a resource with all the four qualities stated above. Otherwise, shipment time surprises, bugs identification by the client, improper or unintended application’s behavior or even wrong outputs of business critical tasks are more likely to be observed. Get this task done by most suitable resources and pay it the well-deserved attention.

If you have queries related to database testing, database testing interview questions, database testing tools or concepts please put it below. We will be happy to address all the queries in comments or part II of database testing article.

Sunday, 17 July 2011

outlin :test plan

1. Introduction

      Description of this Document
      Related Documents
      Schedule and Milestones

2. Resource Requirements

      Hardware
      Software
          o Test Tools
      Staffing
          o Responsibilities
          o Training

3. Features To Be Tested / Test Approach

      New Features Testing
      Regression Testing

4. Features Not To Be Tested

5. Test Deliverable

6. Dependencies/Risks

7. Entrance/Exit Criteria

Saturday, 16 July 2011

Levels of Testing:Acceptance Testing

Acceptance Testing is a level of the software testing process where a system is tested for acceptability.
The purpose of this test is to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery.
acceptance testing image
ANALOGY
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed. Once the System Testing is complete, Acceptance Testing is performed so as to confirm that the ballpoint pen is ready to be made available to the end-users.
METHOD
Usually, Black Box Testing method is used in Acceptance Testing.
Testing does not usually follow a strict procedure and is not scripted but is rather ad-hoc.
TASKS
  • Acceptance Test Plan
    • Prepare
    • Review
    • Rework
    • Baseline
  • Acceptance Test Cases/Checklist
    • Prepare
    • Review
    • Rework
    • Baseline
  • Acceptance Test
    • Perform
When is it performed?
Acceptance Testing is performed after System Testing and before making the system available for actual use.
Who performs it?
  • Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales and/or Customer Support.
  • External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
    • Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software for them. [This is in the case of the software not being owned by the organization that developed it.]
    • User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.

Levels of Testing:System Testing

System Testing is a level of the software testing process where a complete, integrated system/software is tested.
The purpose of this test is to evaluate the system’s compliance with the specified requirements.
system testing image
ANALOGY
During the process of manufacturing a ballpoint pen, the cap, the body, the tail, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed.
METHOD
Usually, Black Box Testing method is used.
TASKS
  • System Test Plan
    • Prepare
    • Review
    • Rework
    • Baseline
  • System Test Cases
    • Prepare
    • Review
    • Rework
    • Baseline
  • System Test
    • Perform
When is it performed?
System Testing is performed after Integration Testing and before Acceptance Testing.
Who performs it?
Normally, independent Testers perform System Testing.

Levels of Testings:Integrated Testing

DEFINITION
Integration Testing is a level of the software testing process where individual units are combined and tested as a group.
integration testing image
The purpose of this level of testing is to expose faults in the interaction between integrated units.
Test drivers and test stubs are used to assist in Integration Testing.
Note: The definition of a unit is debatable and it could mean any of the following:
  1. the smallest testable part of a software
  2. a ‘module’ which could consist of many of ‘1’
  3. a ‘component’ which could consist of many of ’2′
ANALOGY
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. For example, whether the cap fits into the body or not.
METHOD
Any of Black Box Testing, White Box Testing, and Gray Box Testing methods can be used. Normally, the method depends on your definition of ‘unit’.
TASKS
  • Integration Test Plan
    • Prepare
    • Review
    • Rework
    • Baseline
  • Integration Test Cases/Scripts
    • Prepare
    • Review
    • Rework
    • Baseline
  • Integration Test
    • Perform
When is Integration Testing performed?
Integration Testing is performed after Unit Testing and before System Testing.
Who performs Integration Testing?
Either Developers themselves or independent Testers perform Integration Testing.
APPROACHES
  • Big Bang is an approach to Integration Testing where all or most of the units are combined together and tested at one go. This approach is taken when the testing team receives the entire software in a bundle. So what is the difference between Big Bang Integration Testing and System Testing? Well, the former tests only the interactions between the units while the latter tests the entire system.
  • Top Down is an approach to Integration Testing where top level units are tested first and lower level units are tested step by step after that. This approach is taken when top down development approach is followed. Test Stubs are needed to simulate lower level units which may not be available during the initial phases.
  • Bottom Up is an approach to Integration Testing where bottom level units are tested first and upper level units step by step after that. This approach is taken when bottom up development approach is followed. Test Drivers are needed to simulate higher level units which may not be available during the initial phases.
  • Sandwich/Hybrid is an approach to Integration Testing which is a combination of Top Down and Bottom Up approaches.
TIPS
  • Ensure that you have a proper Detail Design document where interactions between each unit are clearly defined. In fact, you will not be able to perform Integration Testing without this information.
  • Ensure that you have a robust Software Configuration Management system in place. Or else, you will have a tough time tracking the right version of each unit, especially if the number of units to be integrated is huge.
  • Make sure that each unit is first unit tested before you start Integration Testing.
  • As far as possible, automate your tests, especially when you use the Top Down or Bottom Up approach, since regression testing is important each time you integrate a unit, and manual regression testing can be inefficient.

Levels of Testings:Unit Testing

Unit Testing is a level of the software testing process where individual units/components of a software/system are tested. The purpose is to validate that each unit of the software performs as designed.
unit testing
A unit is the smallest testable part of software. It usually has one or a few inputs and usually a single output. In procedural programming a unit may be an individual program, function, procedure, etc. In object-oriented programming, the smallest unit is a method, which may belong to a base/super class, abstract class or derived/child class. (Some treat a module of an application as a unit. This is to be discouraged as there will probably be many individual units within that module.)
Unit testing frameworks, drivers, stubs and mock or fake objects are used to assist in unit testing.
METHOD
Unit Testing is performed by using the White Box Testing method.
When is it performed?
Unit Testing is the first level of testing and is performed prior to Integration Testing.
Who performs it?
Unit Testing is normally performed by software developers themselves or their peers. In rare cases it may also be performed by independent software testers.
TASKS
  • Unit Test Plan
    • Prepare
    • Review
    • Rework
    • Baseline
  • Unit Test Cases/Scripts
    • Prepare
    • Review
    • Rework
    • Baseline
  • Unit Test
    • Perform
BENEFITS

  • Unit testing increases confidence in changing/maintaining code. If good unit tests are written and if they are run every time any code is changed, the likelihood of any defects due to the change being promptly caught is very high. If unit testing is not in place, the most one can do is hope for the best and wait till the test results at higher levels of testing are out. Also, if codes are already made less interdependent to make unit testing possible, the unintended impact of changes to any code is less.
  • Codes are more reusable. In order to make unit testing possible, codes need to be modular. This means that codes are easier to reuse.
  • Development is faster. How? If you do not have unit testing in place, you write your code and perform that fuzzy ‘developer test’ (You set some breakpoints, fire up the GUI, provide a few inputs that hopefully hit your code and hope that you are all set.) In case you have unit testing in place, you write the test, code and run the tests. Writing tests takes time but the time is compensated by the time it takes to run the tests. The test runs take very less time: You need not fire up the GUI and provide all those inputs. And, of course, unit tests are more reliable than ‘developer tests’. Development is faster in the long run too. How? The effort required to find and fix defects found during unit testing is peanuts in comparison to those found during system testing or acceptance testing.
  • The cost of fixing a defect detected during unit testing is lesser in comparison to that of defects detected at higher levels. Compare the cost (time, effort, destruction, humiliation) of a defect detected during acceptance testing or say when the software is live.
  • Debugging is easy. When a test fails, only the latest changes need to be debugged. With testing at higher levels, changes made over the span of several days/weeks/months need to be debugged.
  • Codes are more reliable. Why? I think there is no need to explain this to a sane person.
TIPS
  • Find a tool/framework for your language.
  • Do not create test cases for ‘everything’: some will be handled by ‘themselves’. Instead, focus on the tests that impact the behavior of the system.
  • Isolate the development environment from the test environment.
  • Use test data that is close to that of production.
  • Before fixing a defect, write a test that exposes the defect. Why? First, you will later be able to catch the defect if you do not fix it properly. Second, your test suite is now more comprehensive. Third, you will most probably be too lazy to write the test after you have already ‘fixed’ the defect.
  • Write test cases that are independent of each other. For example if a class depends on a database, do not write a case that interacts with the database to test the class. Instead, create an abstract interface around that database connection and implement that interface with mock object.
  • Aim at covering all paths through the unit. Pay particular attention to loop conditions.
  • Make sure you are using a version control system to keep track of your code as well as your test cases.
  • In addition to writing cases to verify the behavior, write cases to ensure performance of the code.
  • Perform unit tests continuously and frequently.
ONE MORE REASON
Lets say you have a program comprising of two units. The only test you perform is system testing. [You skip unit and integration testing.] During testing, you find a bug. Now, how will you determine the cause of the problem?
  • Is the bug due to an error in unit 1?
  • Is the bug due to an error in unit 2?
  • Is the bug due to errors in both units?
  • Is the bug due to an error in the interface between the units?
  • Is the bug due to an error in the test or test case?
Unit testing is often neglected but it is, in fact, the most important level of testing.

DEFINITION OF TEST

Google Dictionary:
A Test is a deliberate action or experiment to find out how well something works.

Cambridge Advanced Learner’s Dictionary:
A Test is an act of using something to find out whether it is working correctly or how effective it is.

The Free Dictionary by Farlex:
A Test is a procedure for critical evaluation; a means of determining the presence, quality, or truth of something.

Meriam Webster:

A Test is a critical examination, observation, or evaluation.

Dictionary.com:
A Test is the means by which the presence, quality, or genuineness of anything is determined.

WordWeb:
A Test is trying something to find out about it.

Longman Dictionary of Contemporary English:

A Test is a process used to discover whether equipment or a product works correctly, or to discover more about it.

SOFTWARE DEVELOPMENT LIFE CYCLE

Software Development Life Cycle, or Software Development Process, defines the steps/stages/phases in the building of software.
There are various kinds of software development models like:
  • Waterfall model
  • Spiral model
  • Iterative and incremental development (like ‘Unified Process’ and ‘Rational Unified Process’)
  • Agile development (like ‘Extreme Programming’ and ‘Scrum’)
Models are evolving with time and the development life cycle can vary significantly from one model to the other. It is beyond the scope of this particular article to discuss each model. However, each model comprises of all or some of the following phases/activities/tasks.
SDLC IN SUMMARY
  • Project Planning
  • Requirements Development
  • Estimation
  • Scheduling
  • Design
  • Coding
  • Test Build/Deployment
  • Unit Testing
  • Integration Testing
  • User Documentation
  • System Testing
  • Acceptance Testing
  • Production Build/Deployment
  • Release
  • Maintenance

Software Quality Assurance

Software Quality Assurance (SQA) is a set of activities for ensuring quality in software engineering processes (that ultimately result in quality in software products).
It includes the following activities:
  • Process definition and implementation
  • Auditing
  • Training
The quality management system under which the software system is created is normally based on one or more of the following models/standards:
  • CMMI
  • Six Sigma
  • ISO 9000
Note: There are many other models/standards for quality management but the ones mentioned above are the most popular.
Software Quality Assurance encompasses the entire software development life cycle and the goal is to ensure that the development and/or maintenance processes are continuously improved to produce products that meet specifications/requirements.
The process of Software Quality Control (SQC) is also governed by Software Quality Assurance (SQA).
SQA is generally shortened to just QA.


Differences between Software Quality Assurance (SQA) and Software Quality Control (SQC):


Many people still use the term Quality Assurance (QA) and Quality Control (QC) interchangeably but this should be discouraged.


CriteriaSoftware Quality Assurance (SQA)
Software Quality Control (SQC)
DefinitionSQA is a set of activities for ensuring quality in software engineering processes (that ultimately result in quality in software products). The activities establish and evaluate the processes that produce products.SQC is a set of activities for ensuring quality in software products. The activities focus on identifying defects in the actual products produced.
FocusProcess focusedProduct focused
OrientationPrevention orientedDetection oriented
BreadthOrganization wideProduct/project specific
ScopeRelates to all products that will ever be created by a processRelates to specific product
Activities
  • Process Definition and Implementation
  • Audits
  • Training
  • Reviews
  • Testing

Software Quality Definition, Explanation, Dimensions:

Software quality is the degree of conformance to explicit or implicit requirements and expectations.

Explanation:
  • Explicit: clearly defined and documented
  • Implicit: not clearly defined and documented but indirectly suggested
  • Requirements: business/product/software requirements
  • Expectations: mainly end-user expectations
Note: Some people tend to accept quality as compliance to only explicit requirements and not implicit requirements. We tend to think of such people as lazy.

Definition by IEEE
  • The degree to which a system, component, or process meets specified requirements.
  • The degree to which a system, component, or process meets customer or user needs or expectations.

DIMENSIONS
Below are some dimensions of software quality:
  • Accessibility: The degree to which software can be used comfortably by a wide variety of people, including those who require assistive technologies like screen magnifiers or voice recognition.
  • Compatibility: The suitability of software for use in different environments like different Operating Systems, Browsers, etc.
  • Concurrency: The ability of software to service multiple requests to the same resources at the same time.
  • Efficiency: The ability of software to perform well or achieve a result without wasted energy, resources, effort, time or money.
  • Functionality: The ability of software to carry out the functions as specified or desired.
  • Install-ability: The ability of software to be installed in specified environment.
  • Localization: The ability of software to be used in different languages, time zones etc.
    Maintainability: The ease with which software can be modified (adding features, enhancing features, fixing bugs, etc)
  • Performance: The speed at which software performs under a particular load.
  • Portability: The ability of software to be transferred easily from one location to another.
  • Reliability: The ability of software to perform a required function under stated conditions for stated period of time without any errors.
  • Scalability: The measure of software’s ability to increase or decrease in performance in response to changes in software’s processing demands.
  • Security: The extent of protection of software against unauthorized access, invasion of privacy, theft, loss of data, etc.
  • Testability: The ability of software to be easily tested.
  • Usability: The degree of software’s ease of use.